Video signal processing method and apparatus

Abstract

The video signal processing method of the present disclosure includes the steps of parsing higher-level MMVD activation information (sps_mmvd_enabled_flag) from a bitstream, which indicates whether an upper-level MMVD (Merge with MVD) including a current block is used; parsing MMVD merge information (mmvd_merge_flag) from the bitstream, which indicates whether an MMVD is to be used for the current block if the higher-level MMVD activation information indicates MMVD activation; parsing MMVD distance-related information (mmvd_distance_idx) and MMVD direction-related information (mmvd_direction_idx) if the MMVD merge information indicates that an MMVD is to be used for the current block; and obtaining information about the MMVD (mMvdLX) based on the MMVD distance-related information and the MMVD direction-related information, wherein the information about the MMVD is greater than or equal to -2^17 and less than or equal to 2^17-1.

Description

[Technical field]

[0001] The present disclosure relates to a method and apparatus for processing a video signal, and more particularly to a method and apparatus for processing a video signal for encoding or decoding a video signal. [Background technology]

[0002] Compression coding refers to a series of signal processing techniques for transmitting digitized information via communication lines or storing it in a form suitable for storage media. The objects of compression coding include voice, video, and text, and the technology for compressing and coding video in particular is called video compression. Compression coding of video signals is performed by removing redundant information by taking into account spatial correlation, temporal correlation, probabilistic correlation, and the like. However, with the recent development of various media and data transmission media, there is a demand for more efficient video signal processing methods and devices. Summary of the Invention [Problem to be solved by the invention]

[0003] An object of the present disclosure is to increase the coding efficiency of video signals. [Means for solving the problem]

[0004] A method for decoding a video signal according to one embodiment of the present disclosure includes the steps of parsing higher-level MMVD activity information (sps_mmvd_enabled_flag) from a bitstream indicating whether an upper-level MMVD (Merge with MVD) including a current block is used, parsing MMVD merge information (mmvd_merge_flag) from the bitstream indicating whether an MMVD is to be used for the current block when the higher-level MMVD activity information indicates MMVD activity, parsing MMVD distance-related information (mmvd_distance_idx) and MMVD direction-related information (mmvd_direction_idx) when the MMVD merge information indicates the use of an MMVD for the current block, and obtaining information about the MMVD (mMvdLX) based on the MMVD distance-related information and the MMVD direction-related information, wherein the information about the MMVD is greater than or equal to -2^17 and less than or equal to 2^17-1.

[0005] In a method for decoding a video signal according to one embodiment of the present disclosure, the upper level is characterized in that it is any one of a coding tree unit, a slice, a tile, a tile group, a picture, or a sequence unit.

[0006] A method for decoding a video signal according to one embodiment of the present disclosure further includes a step of generating a merge candidate list for a current block, a step of selecting a motion vector from the merge candidate list based on a merge index parsed from a bitstream, a step of adding information about the MMVD to the motion vector to obtain a modified motion vector when the MMVD merge information indicates that an MMVD is to be used for the current block, and a step of restoring the current block based on the modified motion vector, wherein the modified motion vector is greater than or equal to -2^17 and less than or equal to 2^17-1.

[0007] In a method for decoding a video signal according to one embodiment of the present disclosure, the step of acquiring information (mMvdLX) related to the MMVD includes a step of acquiring an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD; when a first reference list and a second reference list are used, acquiring a difference in POC (Picture Order Count) between a current picture including a current block and a first reference picture based on the first reference list as a first POC difference, and acquiring a difference in POC (Picture Order Count) between the current picture and a second reference picture based on the second reference list as a second POC difference; and acquiring information related to a first MMVD associated with the first reference list and information related to a second MMVD associated with the second reference list based on at least one of the MMVD offset, the first POC difference, and the second POC difference, wherein the information related to the MVD includes information related to the first MMVD and information related to the second MMVD.

[0008] A method for decoding a video signal according to one embodiment of the present disclosure is characterized in that it includes a step of obtaining an MMVD offset as information about the first MMVD and obtaining an MMVD offset as information about the second MMVD when the first POC difference and the second POC difference are the same.

[0009] A method for decoding a video signal according to one embodiment of the present disclosure is characterized in that it includes a step of obtaining an MMVD offset as information about a first MMVD when the absolute value of the first POC difference is greater than or equal to the absolute value of the second POC difference, a step of scaling information about the first MMVD to obtain information about a second MMVD when the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, and a step of obtaining information about the second MMVD without scaling the absolute value of information about the first MMVD when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture.

[0010] A method for decoding a video signal according to one embodiment of the present disclosure is characterized in that it includes a step of obtaining an MMVD offset as information about a second MMVD when the absolute value of the first POC difference is smaller than the absolute value of the second POC difference, a step of scaling information about the second MMVD to obtain information about the first MMVD when the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, and a step of obtaining information about the first MMVD without scaling the absolute value of the information about the second MMVD when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture.

[0011] In a method for decoding a video signal according to one embodiment of the present disclosure, the step of acquiring information (mMvdLX) related to the MMVD includes a step of acquiring an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD, a step of not scaling the MMVD offset and acquiring it as information related to the first MMVD related to the first reference list when only the first reference list is used, and a step of not scaling the MMVD offset and acquiring it as information related to the second MMVD related to the second reference list when only the second reference list is used.

[0012] A method for decoding a video signal according to one embodiment of the present disclosure includes the steps of: obtaining chroma component format information from an upper level bitstream; obtaining width information (SubWidthC) and height information (SubHeightC) based on the chroma component format information; obtaining x-axis scale information based on width information or information about a color component of a current block; obtaining y-axis scale information based on height information or information about a color component of the current block; determining a position of a left block based on the y-axis scale information; determining a position of an upper block based on the x-axis scale information; determining a weighting value based on the left block and the upper block; obtaining a first sample predicting the current block as a merge mode; obtaining a second sample predicting the current block as an intra mode; and obtaining a combined prediction sample for the current block based on the weighting value, the first sample, and the second sample.

[0013] In a method for decoding a video signal according to one embodiment of the present disclosure, the step of determining a weighting value includes a step of setting code information (isIntraCodedNeighbourA) for the left block to TRUE if the left block is available and the prediction mode of the left block is intra prediction, a step of setting code information for the left block to FALSE if the left block is not available or the prediction mode of the left block is not intra prediction, a step of setting code information (isIntraCodedNeighbourB) for the upper block to TRUE if the upper block is available and the prediction mode of the upper block is intra prediction, and a step of setting code information for the upper block to FALSE if the upper block is not available or the prediction mode of the upper block is not intra prediction.

[0014] In a method for decoding a video signal according to one embodiment of the present disclosure, the step of determining a weighting value includes a step of determining a weighting value as 3 when the code information regarding the left block and the code information regarding the upper block are both TRUE, a step of determining a weighting value as 1 when the code information regarding the left block and the code information regarding the upper block are both FALSE, and a step of determining a weighting value as 2 when only one of the code information regarding the left block and the code information regarding the upper block is TRUE.

[0015] In a method for decoding a video signal according to one embodiment of the present disclosure, the step of obtaining a combination prediction sample includes a step of predicting a current block based on predSamplesComb[x][y]=(w*predSamplesIntra[x][y]+(4-w)*predSamplesInter[x][y]+2)>>2, where predSamplesComb refers to a combination prediction sample, w refers to a weighting value, predSamplesIntra refers to a second sample, predSamplesInter refers to a first sample, [x] refers to the x-axis coordinate of a sample included in the current block, and [y] refers to the y-axis coordinate of a sample included in the current block.

[0016] In a method for decoding a video signal according to one embodiment of the present disclosure, the step of acquiring x-axis scale information includes a step of determining the x-axis scale information to be 0 when a color component of the current block is 0 or information related to width is 1, and a step of determining the x-axis scale information to be 1 when a color component of the current block is not 0 and information related to width is not 1; and the step of acquiring y-axis scale information includes a step of determining the y-axis scale information to be 0 when a color component of the current block is 0 or information related to height is 1, and a step of determining the y-axis scale information to be 1 when a color component of the current block is not 0 and information related to height is not 1.

[0017] In a method for decoding a video signal according to an embodiment of the present disclosure, the position of the left block is (xCb - 1, yCb - 1+(cbHeight << scallFactHeight)), where xCb is the x-axis coordinate of the upper left sample of the current luma block, yCb is the y-axis coordinate of the upper left sample of the current luma block, cbHeight is the size of the height of the current block, scallFactHeight is the scale information of the y-axis, the position of the upper block is (xCb - 1+(cbWidth << scallFactWidth), yCb - 1), where xCb is the x-axis coordinate of the upper left sample of the current luma block, yCb is the y-axis coordinate of the upper left sample of the current luma block, cbWidth is the size of the width of the current block, and scallFactWidth is the scale information of the x-axis, which is characterized by the above.

[0018] An apparatus for decoding a video signal according to an embodiment of the present disclosure includes a processor and a memory. The processor parses from a bit stream a higher-level MMVD (Merge with MVD) active information (sps_mmvd_enabled_flag) indicating whether a higher-level MMVD including the current block is used based on an instruction word stored in the memory. When the higher-level MMVD active information indicates the activation of the MMVD, the processor parses from the bit stream an MMVD merge information (mmvd_merge_flag) indicating whether to use the MMVD for the current block. When the MMVD merge information indicates to use the MMVD for the current block, the processor parses information related to the distance of the MMVD (mmvd_distance_idx) and information related to the direction of the MMVD (mmvd_direction_idx), and obtains information regarding the MMVD (mMvdLX) based on the information related to the distance of the MMVD and the information related to the direction of the MMVD. The information regarding the MMVD is greater than or equal to -2^17 and less than or equal to 2^17 - 1, which is characterized by the above.

[0019] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, the upper level is characterized in that it is any one of a coding tree unit, a slice, a tile, a tile group, a picture, or a sequence unit.

[0020] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor generates a merge candidate list for a current block based on an instruction stored in a memory, selects a motion vector from the merge candidate list based on a merge index parsed from a bitstream, and when the MMVD merge information indicates that an MMVD is to be used for the current block, adds information about the MMVD to the motion vector to obtain a modified motion vector, and restores the current block based on the modified motion vector, wherein the modified motion vector is greater than or equal to -2^17 and less than or equal to 2^17-1.

[0021] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor acquires an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD based on an instruction stored in a memory; when a first reference list and a second reference list are used, acquires a difference in POC (Picture Order Count) between a current picture including a current block and a first reference picture based on the first reference list as a first POC difference, and acquires a difference in POC (Picture Order Count) between the current picture and a second reference picture based on the second reference list as a second POC difference; and acquires information regarding a first MMVD associated with the first reference list and information regarding a second MMVD associated with the second reference list based on at least one of the MMVD offset, the first POC difference, and the second POC difference, wherein the information regarding the MMVD includes information regarding the first MMVD and information regarding the second MMVD.

[0022] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor is characterized in that, when the first POC difference and the second POC difference are identical, based on an instruction word stored in a memory, the processor obtains an MMVD offset as information about the first MMVD and obtains an MMVD offset as information about the second MMVD.

[0023] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor, based on an instruction word stored in a memory, obtains an MMVD offset as information about the first MMVD when the absolute value of the first POC difference is greater than or equal to the absolute value of the second POC difference, scales information about the first MMVD to obtain information about the second MMVD when the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, and obtains information about the second MMVD without scaling the absolute value of the information about the first MMVD when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture.

[0024] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor, based on an instruction word stored in a memory, obtains an MMVD offset as information about a second MMVD when the absolute value of the first POC difference is smaller than the absolute value of the second POC difference, scales information about the second MMVD to obtain information about the first MMVD when the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, and obtains information about the first MMVD without scaling the absolute value of the information about the second MMVD when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture.

[0025] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor obtains an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD based on an instruction word stored in a memory, and when only a first reference list is used, the MMVD offset is not scaled and is obtained as information related to a first MMVD associated with the first reference list, and when only a second reference list is used, the MMVD offset is not scaled and is obtained as information related to a second MMVD associated with the second reference list.

[0026] An apparatus for decoding a video signal according to one embodiment of the present disclosure includes a processor and a memory, and the processor obtains chroma component format information from an upper level bitstream based on an instruction stored in the memory, obtains width information (SubWidthC) and height information (SubHeightC) based on the chroma component format information, obtains x-axis scale information based on information about the width or information about a color component of a current block, obtains y-axis scale information based on information about the height or information about a color component of the current block, determines a position of a left block based on the y-axis scale information, determines a position of an upper block based on the x-axis scale information, determines a weighting value based on the left block and the upper block, obtains a first sample predicting the current block as a merge mode, obtains a second sample predicting the current block as an intra mode, and obtains a combined prediction sample for the current block based on the weighting value, the first sample, and the second sample.

[0027] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor is characterized in that, based on an instruction word stored in a memory, if the left block is available and the prediction mode of the left block is intra prediction, the processor sets code information for the left block (isIntraCodedNeighbourA) to TRUE, if the left block is unavailable or the prediction mode of the left block is not intra prediction, the processor sets code information for the left block to FALSE, if the upper block is available and the prediction mode of the upper block is intra prediction, the processor sets code information for the upper block (isIntraCodedNeighbourB) to TRUE, and if the upper block is unavailable or the prediction mode of the upper block is not intra prediction, the processor sets code information for the upper block to FALSE.

[0028] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor determines a weighted value to be 3 when the code information for the left block and the code information for the upper block are both TRUE, determines a weighted value to be 1 when the code information for the left block and the code information for the upper block are both FALSE, and determines a weighted value to be 2 when only one of the code information for the left block and the code information for the upper block is TRUE, based on an instruction word stored in a memory.

[0029] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor predicts a current block based on predSamplesComb[x][y]=(w*predSamplesIntra[x][y]+(4-w)*predSamplesInter[x][y]+2)>>2 based on an instruction stored in a memory, where predSamplesComb means a combination prediction sample, w means a weighting value, predSamplesIntra means a second sample, predSamplesInter means a first sample, [x] means the x-axis coordinate of a sample included in the current block, and [y] means the y-axis coordinate of a sample included in the current block.

[0030] In an apparatus for decoding a video signal according to one embodiment of the present disclosure, a processor determines, based on an instruction word stored in a memory, x-axis scale information to be 0 if a color component of a current block is 0 or information regarding width is 1, determines x-axis scale information to be 1 if a color component of a current block is not 0 and information regarding width is not 1, determines y-axis scale information to be 0 if a color component of a current block is 0 or information regarding height is 1, and determines y-axis scale information to be 1 if a color component of a current block is not 0 and information regarding height is not 1.

[0031] The position of the left block of the apparatus for decoding a video signal according to an embodiment of the present disclosure is (xCb - 1, yCb - 1+(cbHeight<<scallFactHeight)), where xCb is the x-axis coordinate of the upper left sample of the current luma block, yCb is the y-axis coordinate of the upper left sample of the current luma block, cbHeight is the size of the height of the current block, scallFactHeight is the scale information of the y-axis, the position of the upper block is (xCb - 1+(cbWidth<<scallFactWidth), yCb - 1), where xCb is the x-axis coordinate of the upper left sample of the current luma block, yCb is the y-axis coordinate of the upper left sample of the current luma block, cbWidth is the size of the width of the current block, and scallFactWidth is the scale information of the x-axis, which is characterized by the above.

[0032] A method for encoding a video signal according to an embodiment of the present disclosure includes generating information (mMvdLX) regarding the MMVD of the current block, generating information related to the distance of the MMVD and information related to the direction of the MMVD based on the information (mMvdLX) regarding the MMVD, generating MMVD merge information (mmvd_merge_flag) indicating whether to use the MMVD for the current block, generating upper level MMVD activation information (sps_mmvd_enabled_flag) indicating whether to use the upper level MMVD (Merge with MVD) including the current block, and generating a bitstream based on the information related to the distance of the MMVD, the information related to the direction of the MMVD, the MMVD merge information (mmvd_merge_flag), and the upper level MMVD activation information (sps_mmvd_enabled_flag), where the information regarding the MMVD is greater than or equal to -2^17 and less than or equal to 2^17 - 1, which is characterized by the above.

[0033] An apparatus for encoding a video signal according to one embodiment of the present disclosure includes a processor and a memory, wherein the processor generates information (mMvdLX) regarding an MMVD of a current block based on an instruction word stored in the memory, generates information related to the distance of the MMVD and information related to the direction of the MMVD based on the information (mMvdLX) regarding the MMVD, generates MMVD merge information (mmvd_merge_flag) indicating whether an MMVD is used for the current block, generates higher level MMVD activity information (sps_mmvd_enabled_flag) indicating whether an upper level MMVD (Merge with MVD) including the current block is used, and generates a bitstream based on the MMVD distance-related information, the MMVD direction-related information, the MMVD merge information (mmvd_merge_flag), and the higher level MMVD activity information (sps_mmvd_enabled_flag), wherein the information regarding the MMVD is greater than or equal to -2^17 and less than or equal to 2^17-1.

[0034] A method for encoding a video signal according to one embodiment of the present disclosure includes the steps of: generating higher-level chroma component format information; obtaining width information (SubWidthC) and height information (SubHeightC) based on the chroma component format information; obtaining x-axis scale information based on width information or information about a color component of a current block; obtaining y-axis scale information based on height information or information about a color component of the current block; determining a position of a left block based on the y-axis scale information; determining a position of an upper block based on the x-axis scale information; determining a weighting value based on the left block and the upper block; obtaining a first sample predicting the current block as a merge mode; obtaining a second sample predicting the current block as an intra mode; and obtaining a combined prediction sample for the current block based on the weighting value, the first sample, and the second sample.

[0035] An apparatus for encoding a video signal according to one embodiment of the present disclosure includes a processor and a memory, and the processor generates higher-level chroma component format information based on an instruction stored in the memory, obtains width information (SubWidthC) and height information (SubHeightC) based on the chroma component format information, obtains x-axis scale information based on information about the width or information about the color components of the current block, obtains y-axis scale information based on information about the height or information about the color components of the current block, determines a position of a left block based on the y-axis scale information, determines a position of an upper block based on the x-axis scale information, determines a weighting value based on the left block and the upper block, obtains a first sample predicting the current block as a merge mode, obtains a second sample predicting the current block as an intra mode, and obtains a combined prediction sample for the current block based on the weighting value, the first sample, and the second sample. Effect of the Invention

[0036] According to an embodiment of the present invention, the coding efficiency of a video signal can be improved. [Brief description of the drawings]

[0037] [Figure 1] 1 is a schematic block diagram of a video signal encoding device according to an embodiment of the present disclosure. [Diagram 2] 1 is a schematic block diagram of a video signal decoding device according to an embodiment of the present disclosure. [Diagram 3] A diagram showing an example in which coding tree units are divided into coding units within a picture. [Figure 4] FIG. 1 illustrates one embodiment of a method for signaling splits in quadtrees and multitype trees. [Diagram 5] FIG. 2 is a diagram illustrating an intra prediction method according to an embodiment of the present disclosure in more detail. [Figure 6]FIG. 2 is a diagram illustrating an intra prediction method according to an embodiment of the present disclosure in more detail. [Figure 7] FIG. 1 is a diagram illustrating an inter-prediction method according to an embodiment of the present disclosure. [Figure 8] 4 illustrates how a motion vector of a current block is signaled according to one embodiment of the present disclosure. [Figure 9] 4 illustrates how a motion vector difference value of a current block is signaled according to one embodiment of the present disclosure. [Figure 10] FIG. 2 illustrates adaptive motion vector resolution signaling according to one embodiment of the present disclosure. [Figure 11] FIG. 1 is a diagram illustrating an inter prediction related syntax according to an embodiment of the present disclosure. [Figure 12] FIG. 2 illustrates a multi-hypothesis prediction according to one embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram illustrating a multi-hypothesis prediction-related syntax according to an embodiment of the present disclosure. [Figure 14] FIG. 13 is a diagram illustrating a multi-hypothesis prediction-related syntax according to an embodiment of the present disclosure. [Figure 15] FIG. 13 is a diagram illustrating a multi-hypothesis prediction-related syntax according to an embodiment of the present disclosure. [Figure 16] FIG. 13 is a diagram illustrating a multi-hypothesis prediction mode decision method according to an embodiment of the present disclosure. [Figure 17] FIG. 2 is a diagram illustrating a method for generating a candidate list according to an embodiment of the present disclosure. [Figure 18] FIG. 13 is a diagram showing surrounding positions referenced in multi-hypothesis prediction according to an embodiment of the present disclosure. [Figure 19] FIG. 13 is a diagram illustrating a method for referencing a surrounding mode according to one embodiment of the present disclosure. [Figure 20] FIG. 1 illustrates a method for generating a candidate list according to an embodiment of the present disclosure. [Figure 21]FIG. 1 illustrates a method for generating a candidate list according to an embodiment of the present disclosure. [Figure 22] FIG. 13 is a diagram showing surrounding positions referenced in multi-hypothesis prediction according to an embodiment of the present disclosure. [Figure 23] FIG. 13 is a diagram illustrating a method for referencing a surrounding mode according to one embodiment of the present disclosure. [Figure 24] FIG. 13 illustrates the use of peripheral samples according to one embodiment of the present disclosure. [Diagram 25] FIG. 2 illustrates a transform mode according to one embodiment of the present disclosure. [Figure 26] FIG. 13 is a diagram illustrating the relationship between multi-hypothesis prediction and transform modes according to one embodiment of the present disclosure. [Figure 27] FIG. 2 is a diagram illustrating the relationship between color components according to one embodiment of the present disclosure. [Figure 28] FIG. 2 is a diagram illustrating a relationship between color components according to an embodiment of the present disclosure. [Figure 29] FIG. 13 is a diagram illustrating peripheral reference positions according to an embodiment of the present disclosure. [Diagram 30] FIG. 2 illustrates a weighted sample prediction process according to one embodiment of the present disclosure. [Diagram 31] FIG. 13 is a diagram illustrating peripheral reference positions according to an embodiment of the present disclosure. [Diagram 32] FIG. 2 is a diagram illustrating a weighted sample prediction process according to an embodiment of the present disclosure. [Diagram 33] FIG. 2 is a diagram illustrating a weighted sample prediction process according to an embodiment of the present disclosure. [Diagram 34] FIG. 2 is a diagram illustrating a weighted sample prediction process according to an embodiment of the present disclosure. [Diagram 35] FIG. 2 is a diagram illustrating a CIIP weight derivation according to one embodiment of the present disclosure. [Diagram 36] A diagram showing a CIIP process according to one embodiment of the present disclosure. [Figure 37] A diagram showing the range of MV and MVD according to one embodiment of the present disclosure. [Figure 38] FIG. 2 is a diagram showing an MMVD according to one embodiment of the present disclosure. [Figure 39] FIG. 1 is a diagram showing MVD derivation of MMVD according to one embodiment of the present disclosure. [Diagram 40] FIG. 13 is a diagram illustrating the derivation of MVD and MV according to one embodiment of the present disclosure. [Diagram 41] FIG. 13 is a diagram showing derivation of MV and CPMV according to an embodiment of the present disclosure. [Diagram 42] A diagram showing the range of MV and MVD according to one embodiment of the present disclosure. [Diagram 43] FIG. 13 is a diagram illustrating MVD derivation of MMVD according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] The terms used in this specification are currently widely used and common terms as much as possible, taking into consideration the functions in this disclosure, but these may change depending on the intentions of the technicians in this field, customs, or the emergence of new technologies. In addition, in certain cases, the applicant may arbitrarily select terms, and in this case, the meanings will be described in the description of the relevant invention. Therefore, it is clear that the terms used in this specification should be interpreted based on the substantial meaning of the terms and the contents of the entire specification, rather than simply the names of the terms.

[0039] In this specification, some terms may be interpreted as follows. Coding may be interpreted as encoding or decoding depending on the case. In this specification, a device that encodes a video signal to generate a video signal bitstream is referred to as an encoding device or encoder, and a device that decodes the video signal bitstream to restore a video signal is referred to as a decoding device or decoder. In this specification, a video signal processing device is used as a concept term including both an encoder and a decoder. Information is a term including any of values, parameters, coefficients, elements, etc., and may be interpreted differently depending on the case, so the present disclosure is not limited thereto. A "unit" is used to mean a basic unit of image processing or a specific position of a picture, and refers to an image area including both luma components and chroma components. In addition, a "block" refers to an image area including a specific component among luma components and chroma components (i.e., Cb and Cr). However, depending on the embodiment, terms such as "unit", "block", "partition" and "region" may be used with the same meaning. In addition, in this specification, a unit may be used as a concept including any of a coding unit, a prediction unit and a transform unit. A picture refers to a field or a frame, and these terms may be used with the same meaning depending on the embodiment.

[0040] 1 is a schematic block diagram of a video signal encoding device according to an embodiment of the present disclosure. Referring to FIG. 1, the encoding device 100 of the present disclosure includes a transform unit 110, a quantization unit 115, an inverse quantization unit 120, an inverse transform unit 125, a filtering unit 130, a prediction unit 150, and an entropy coding unit 160.

[0041] The transform unit 110 transforms a residual signal, which is a difference between a received video signal and a predicted signal generated by the prediction unit 150, to obtain transform coefficient values. For example, a discrete cosine transform (DCT), a discrete sine transform (DST), or a wavelet transform may be used. The discrete cosine transform and the discrete sine transform divide an input picture signal into blocks and perform the transform. In the transform, the coding efficiency may change depending on the distribution and characteristics of values ​​in a transform domain. The quantizer 115 quantizes the transform coefficient values ​​output from the transform unit 110.

[0042] In order to improve coding efficiency, instead of coding the picture signal as it is, the prediction unit 150 predicts a picture using an already coded region, and adds the residual values ​​of the original picture and the predicted picture to the predicted picture to obtain a restored picture. In order to prevent mismatch between the encoder and the decoder, when the encoder performs prediction, information that can be used by the decoder must be used. To this end, the encoder performs a process of restoring the current block that has been coded. The inverse quantization unit 120 inverse quantizes the transform coefficient values, and the inverse transform unit 125 restores the residual values ​​using the inverse quantized transform coefficient values. Meanwhile, the filtering unit 130 performs a filtering operation to improve the quality of the restored picture and the coding efficiency. For example, the filtering unit 130 may include a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter. The filtered picture is output or stored in a decoded picture buffer (DPB) 156 to be used as a reference picture.

[0043] The prediction unit 150 includes an intra prediction unit 152 and an inter prediction unit 154. The intra prediction unit 152 performs intra prediction in a current picture, and the inter prediction unit 154 performs inter prediction to predict the current picture using a reference picture stored in a decoded picture buffer 156. The intra prediction unit 152 performs intra prediction from a restored sample in the current picture, and transmits intra coding information to the entropy coding unit 160. The intra coding information may include at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra coding information may include information about the reference sample. The inter prediction unit 154 may include a motion estimation unit 154a and a motion compensation unit 154b. The motion estimation unit 154a obtains a motion vector value of the current region by referring to a specific region of the restored reference picture. The motion estimation unit 154a transmits a motion information set (such as a reference picture index and motion vector information) related to the reference region to the entropy coding unit 160. The motion compensation unit 154b performs motion compensation using the motion vector values ​​transmitted from the motion estimation unit 154a. The inter prediction unit 154 transmits to the entropy coding unit 160 inter coding information including motion information related to the reference region.

[0044] According to a further embodiment, the prediction unit 150 may include an intra block copy (BC) prediction unit (not shown). The intra BC prediction unit performs intra BC prediction from reconstructed samples in the current picture and transmits intra BC coding information to the entropy coding unit 160. The intra BC prediction unit refers to a specific region in the current picture to obtain block vector values ​​indicating a reference region used for predicting the current region. The intra BC prediction unit may perform intra BC prediction using the obtained block vector values. The intra BC prediction unit transmits the intra BC coding information to the entropy coding unit 160. The intra BC coding information may include block vector information.

[0045] When the above picture prediction is performed, the transform unit 110 converts residual values ​​between the original picture and the predicted picture to obtain transform coefficient values. At this time, the transform may be performed in units of a specific block within the picture, and the size of the specific block may vary within a preset range. The quantization unit 115 quantizes the transform coefficient values ​​generated by the transform unit 110 and transfers the quantized transform coefficient values ​​to the entropy coding unit 160.

[0046] The entropy coding unit 160 generates a video signal bitstream by entropy coding information indicating quantized transform coefficients, intra-coding information, and inter-coding information. The entropy coding unit 160 may use a variable length coding (VLC) scheme, an arithmetic coding scheme, and the like. The variable length coding (VLC) scheme converts input symbols into continuous codewords, and the length of the codewords may be variable. For example, frequently occurring symbols are represented by short codewords, and infrequently occurring symbols are represented by long codewords. As the variable length coding scheme, a context-based adaptive variable length coding (CAVLC) scheme may be used. The arithmetic coding converts continuous data symbols into one prime number, and the arithmetic coding can obtain optimal prime number bits required to represent each symbol. As the arithmetic coding, a context-based adaptive binary arithmetic code (CABAC) may be used. For example, the entropy coding unit 160 may binarize information indicating the quantized transform coefficients, and may arithmetically code the binarized information to generate a bitstream.

[0047] The generated bitstream is encapsulated with Network Abstraction Layer (NAL) units as basic units. An NAL unit includes an integer number of coded coding tree units. In order for a video decoder to decode the bitstream, the bitstream must first be separated into NAL units and then each separated NAL unit must be decoded. Meanwhile, information required for decoding the video signal bitstream may be transmitted in Raw Byte Sequence Payload (RBSP) of a higher level set such as a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), a Video Parameter Set (VPS), etc.

[0048] 1 shows an encoding apparatus 100 according to an embodiment of the present disclosure, with separate blocks illustrating elements of the encoding apparatus 100 in a logically distinct manner. Thus, the elements of the encoding apparatus 100 described above may be implemented as one chip or multiple chips depending on the device design. According to an embodiment, the operation of each element of the encoding apparatus 100 described above may be performed by a processor (not shown).

[0049] 2 is a schematic block diagram of a video signal decoding device 200 according to an embodiment of the present disclosure. Referring to FIG. 2, the decoding device 200 of the present disclosure includes an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 225, a filtering unit 230, and a prediction unit 250.

[0050] The entropy decoding unit 210 entropy decodes the video signal bitstream to extract transform coefficient information, intra-coding information, inter-coding information, etc. for each region. For example, the entropy decoding unit 210 may obtain a binary code for transform coefficient information of a specific region from the video signal bitstream. The entropy decoding unit 210 also de-binarizes the binary code to obtain quantized transform coefficients. The inverse quantization unit 220 inverse quantizes the quantized transform coefficients, and the inverse transform unit 225 restores residual values ​​using the inverse quantized transform coefficients. The video signal processing device 200 combines the residual values ​​obtained from the inverse transform unit 225 with the predicted values ​​obtained from the prediction unit 250 to restore original pixel values.

[0051] Meanwhile, the filtering unit 230 performs filtering on the picture to improve image quality, which may include a deblocking filter for reducing block distortion and / or an adaptive loop filter for removing distortion of the entire picture, etc. The filtered picture is output or stored in a decoded picture buffer (DPB) 256 to be used as a reference picture for the next picture.

[0052] The prediction unit 250 includes an intra prediction unit 252 and an inter prediction unit 254. The prediction unit 250 generates a predicted picture using the coding type, the transform coefficient for each region, the intra / inter coding information, etc., decoded by the entropy decoding unit 210. A decoded region of the current picture including the current block or another picture may be used to reconstruct the current block to be decoded. A picture (or a tile / slice) that uses only the current picture for reconstruction, i.e., performs only intra prediction (or intra prediction and intra BC prediction), is called an intra picture or an I picture (or a tile / slice), and a picture (or a tile / slice) that can perform both intra prediction and inter prediction is called an inter picture (or a tile / slice). Among interpictures (or tiles / slices), a picture (or tile / slice) that uses at most one motion vector and reference picture index to predict sample values ​​of each block is called a predictive picture or P picture (or tile / slice), and a picture (or tile / slice) that uses at most two motion vectors and reference picture indexes is called a bi-predictive picture or B picture (or tile / slice). In other words, a P picture (or tile / slice) uses at most one motion information set to predict each block, and a B picture (or tile / slice) uses at most two motion information sets to predict each block. Here, a motion information set includes one or more motion vectors and one reference picture index.

[0053] The intra prediction unit 252 may generate a prediction block using intra coding information and reconstructed samples in a current picture. As described above, the intra coding information may include at least one of an intra prediction mode, a Most Probable Mode (MPM) flag, and an MPM index. The intra prediction unit 252 predicts sample values ​​of a current block using reconstructed samples located to the left and / or above the current block as reference samples. In the present disclosure, the reconstructed samples, the reference samples, and the samples of the current block may represent pixels. Also, the sample values ​​may represent pixel values.

[0054] According to an embodiment, the reference sample may be a sample included in a neighboring block of the current block. For example, the reference sample may be a sample adjacent to the left boundary and / or a sample adjacent to the top boundary of the current block. Also, the reference sample may be a sample located on a line within a preset distance from the left boundary of the current block and / or a sample located on a line within a preset distance from the top boundary of the current block, among samples of neighboring blocks of the current block. In this case, the neighboring blocks of the current block may include at least one of a left (L) block, an upper (A) block, a below left (BL) block, an above right (AR) block, or an above left (AL) block adjacent to the current block.

[0055] The inter prediction unit 254 generates a prediction block using the reference picture and inter coding information stored in the decoded picture buffer 256. The inter coding information may include a motion information set (reference picture index, motion vector information, etc.) of a current block with respect to a reference block. Inter prediction may include L0 prediction, L1 prediction, and bi-prediction. L0 prediction means prediction using one reference picture included in the L0 picture list, and L1 prediction means prediction using one reference picture included in the L1 picture list. For this purpose, one set of motion information (e.g., motion vector and reference picture index) may be required. In the bi-prediction method, up to two reference regions may be used, and the two reference regions may exist in the same reference picture or may exist in different pictures. That is, in the bi-prediction method, up to two sets of motion information (e.g., motion vector and reference picture index) may be used, and two motion vectors may correspond to the same reference picture index or may correspond to different reference picture indexes. In this case, the reference picture may be displayed (or output) either before or after the current picture in terms of time. In addition, in a bi-prediction method, two reference regions may exist in two reference picture lists.

[0056] The inter prediction unit 254 may obtain a reference block of the current block using the motion vector and the reference picture index. The reference block exists in a reference picture corresponding to the reference picture index. Also, a sample value of a block identified by the motion vector or an interpolated value thereof may be used as a predictor of the current block. For motion prediction with pixel accuracy in sub-pel units, for example, an 8-tap interpolation filter may be used for the luma signal and a 4-tap interpolation filter may be used for the chroma signal. However, the interpolation filter for the sub-pel unit motion prediction is not limited thereto. In this way, the inter prediction unit 254 performs motion compensation to predict the texture of the current unit from a previously restored picture. At this time, the inter prediction unit may use a motion information set.

[0057] According to a further embodiment, the prediction unit 250 may include an intra BC prediction unit (not shown). The intra BC prediction unit performs intra BC prediction from reconstructed samples in the current picture and transmits intra BC coding information to the entropy coding unit 160. The intra BC prediction unit obtains block vector values ​​of a current region indicating a specific region in the current picture. The intra BC prediction unit may perform intra BC prediction using the obtained block vector values. The intra BC prediction unit transmits the intra BC coding information to the entropy coding unit 160. The intra BC coding information may include block vector information.

[0058] A reconstructed video picture is generated by adding the predicted value output from the intra prediction unit 252 or the inter prediction unit 254 and the residual value output from the inverse transformation unit 225. That is, the video signal decoding apparatus 200 reconstructs a current block using the predicted block generated by the prediction unit 250 and the residual obtained from the inverse transformation unit 225.

[0059] Meanwhile, the block diagram of Fig. 2 shows a decoding device 200 according to an embodiment of the present disclosure, with the separately shown blocks logically distinguishing the elements of the decoding device 200. Thus, the elements of the decoding device 200 described above may be implemented as one chip or multiple chips depending on the device design. According to an embodiment, the operation of each element of the decoding device 200 described above may be performed by a processor (not shown).

[0060] FIG. 3 illustrates an embodiment in which a coding tree unit (CTU) is divided into coding units (CUs) in a picture. In a video signal coding process, a picture may be divided into a sequence of coding tree units (CTUs). A coding tree unit is composed of an NXN block of luma samples and two blocks of corresponding chroma samples. A coding tree unit may be divided into multiple coding units. A coding tree unit may not be divided and may become a leaf node. In this case, the coding tree unit itself may be a coding unit. A coding unit refers to a basic unit for processing a picture in the above-mentioned video signal processing process, i.e., in the process of intra / inter prediction, transformation, quantization, and / or entropy coding. The size and shape of a coding unit in one picture may not be constant. A coding unit may have a square or rectangular shape. A rectangular coding unit (or rectangular block) includes a vertical coding unit (or vertical block) and a horizontal coding unit (or horizontal block). As used herein, a vertical block is a block whose height is greater than its width, and a horizontal block is a block whose width is greater than its height. Also, as used herein, a non-square block may refer to a rectangular block, although this disclosure is not limited thereto.

[0061] Referring to Fig. 3, a coding tree unit is first divided into a quad tree (QT) structure. That is, one node having a size of 2N x 2N in the quad tree structure may be divided into four nodes having a size of N x N. In this specification, a quad tree may also be referred to as a quaternary tree. The quad tree division may be performed recursively, and all nodes do not need to be divided to the same depth.

[0062] Meanwhile, the leaf node of the quad tree may be further divided into a multi-type tree (MTT) structure. According to an embodiment of the present disclosure, in the multi-type tree structure, one node may be divided into a horizontally or vertically divided binary or ternary tree structure. That is, the multi-type tree structure has four division structures, namely, vertical binary division, horizontal binary division, vertical ternary division, and horizontal ternary division. According to an embodiment of the present disclosure, in each of the tree structures, the width and height of the node may have a power of 2 value. For example, in a binary tree (BT) structure, a node of size 2N×2N may be divided into two N×2N nodes by vertical binary division and divided into two 2N×N nodes by horizontal binary division. In addition, in a Ternary Tree (TT) structure, a node of size 2N×2N may be divided into (N / 2)×2N, N×2N, and (N / 2)×2N nodes by vertical ternary division, and into 2N×(N / 2), 2N×N, and 2N×(N / 2) nodes by horizontal binary division. Such multi-type tree division may be performed recursively.

[0063] A leaf node of a multi-type tree can be a coding unit. If no division is indicated for a coding unit or the coding unit is not larger than the maximum transform length, the coding unit is used as a unit of prediction and transformation without further division. Meanwhile, in the above-mentioned quad tree and multi-type tree, at least one of the following parameters may be predefined or transmitted in the RBSP of a higher level set such as PPS, SPS, VPS, etc. 1) CTU size: root node size of the quad tree, 2) Min QT size (MinQtSize): minimum allowed QT leaf node size, 3) Max BT size (MaxBtSize): maximum allowed BT root node size, 4) Max TT size (MaxTtSize): maximum allowed TT root node size, 5) Max MTT depth (MaxMttDepth): maximum allowed depth of MTT division from the leaf node of QT, 6) Min BT size (MinBtSize): minimum allowed BT leaf node size, 7) Min TT size (MinTtSize): minimum allowed TT leaf node size.

[0064] FIG 4 illustrates an embodiment of a method for signaling the split of a quadtree and a multitype tree. In order to signal the split of a quadtree and a multitype tree as described above, a previously set flag may be used. Referring to FIG 4, at least one of a flag 'qt_split_flag' indicating whether a quadtree node is split, a flag 'mtt_split_flag' indicating whether a multitype tree node is split, a flag 'mtt_split_vertical_flag' indicating the split direction of a multitype tree node, or a flag 'mtt_split_binary_flag' indicating the split shape of a multitype tree node may be used.

[0065] According to an embodiment of the present disclosure, a coding tree unit is a root node of a quad tree and may be first split into a quad tree structure. In the quad tree structure, 'qt_split_flag' is signaled for each node 'QT_node'. If the value of 'qt_split_flag' is 1, the corresponding node is split into four square nodes, and if the value of 'qt_split_flag' is 0, the corresponding node becomes a leaf node 'QT_leaf_node' of the quad tree.

[0066] Each quad tree leaf node 'QT_leaf_node' may be further split into a multitype tree structure. In the multitype tree structure, 'mtt_split_flag' is signaled for each node 'MTT_node'. If the value of 'mtt_split_flag' is 1, the corresponding node is split into multiple rectangular nodes, and if the value of 'mtt_split_flag' is 0, the corresponding node becomes a leaf node 'MTT_leaf_node' of the multitype tree. If the multitype tree node 'MTT_node' is split into multiple rectangular nodes (i.e., if the value of 'mtt_split_flag' is 1), 'mtt_split_vertical_flag' and 'mtt_split_binary_flag' for the node 'MTT_node' may be further signaled. A vertical split of the node 'MTT_node' is indicated when the value of 'mtt_split_vertical_flag' is 1, and a horizontal split of the node 'MTT_node' is indicated when the value of 'mtt_split_vertical_flag' is 0. In addition, when the value of 'mtt_split_binary_flag' is 1, the node 'MTT_node' is split into two rectangular nodes, and when the value of 'mtt_split_binary_flag' is 0, the node 'MTT_node' is split into three rectangular nodes.

[0067] Picture prediction (motion compensation) for coding is performed on a coding unit that is not further divided (i.e., a leaf node of a coding unit tree). A basic unit for performing such prediction is hereinafter referred to as a prediction unit or a prediction block.

[0068] Hereinafter, the term "unit" used in this specification may be used as an alternative term to the prediction unit, which is a basic unit for performing prediction, although the present disclosure is not limited thereto and may be understood as a concept including the coding unit in a broader sense.

[0069] 5 and 6 show the intra prediction method according to the embodiment of the present disclosure in more detail. As described above, the intra prediction unit predicts sample values ​​of the current block using reconstructed samples located to the left and / or above the current block as reference samples.

[0070] First, Fig. 5 shows an example of reference samples used for prediction of a current block in an intra prediction mode. According to an example, the reference samples may be samples adjacent to the left boundary and / or the top boundary of the current block. As shown in Fig. 5, when the size of the current block is W x H and a sample of a single reference line adjacent to the current block is used for intra prediction, the reference samples may be set using up to 2W + 2H + 1 neighboring samples located to the left and / or top of the current block.

[0071] According to a further embodiment of the present disclosure, samples on a plurality of reference lines may be used for intra prediction of a current block. The plurality of reference lines may be composed of n lines located within a preset distance from a boundary of the current block. In this case, separate reference line information indicating at least one reference line used for intra prediction of the current block may be signaled. Specifically, the reference line information may include an index indicating any one of the plurality of reference lines.

[0072] Also, when at least some samples used as reference samples have not yet been restored, the intra prediction unit may perform a reference sample padding process to obtain reference samples. Also, the intra prediction unit may perform a reference sample filtering process to reduce an error in intra prediction. That is, the intra prediction unit may perform filtering on the neighboring samples and / or the reference samples obtained by the reference sample padding process to obtain filtered reference samples. The intra prediction unit predicts samples of the current block using the reference samples obtained in this manner. The intra prediction unit predicts samples of the current block using unfiltered reference samples or filtered reference samples. In the present disclosure, the neighboring samples may include samples on at least one reference line. For example, the neighboring samples may include neighboring samples on a line adjacent to a boundary of the current block.

[0073] Next, FIG. 6 illustrates an example of prediction modes used in intra prediction. For intra prediction, intra prediction mode information indicating an intra prediction direction may be signaled. The intra prediction mode information indicates one of a plurality of intra prediction modes constituting an intra prediction mode set. If the current block is an intra prediction block, the decoder receives the intra prediction mode information of the current block from the bitstream. The intra prediction unit of the decoder performs intra prediction on the current block based on the extracted intra prediction mode information.

[0074] According to an embodiment of the present disclosure, the intra prediction mode set may include all intra prediction modes (e.g., a total of 67 intra prediction modes) used in intra prediction. More specifically, the intra prediction mode set may include a planar mode, a DC mode, and a plurality of (e.g., 65) angle modes (i.e., direction modes). Each intra prediction mode may be indicated by a pre-set index (i.e., intra prediction mode index). For example, as shown in FIG. 6, an intra prediction mode index 0 indicates a planar mode, and an intra prediction mode index 1 indicates a DC mode. In addition, intra prediction mode indexes 2 to 66 may indicate different angle modes, respectively. The angle modes may indicate different angles within a pre-set angle range, respectively. For example, the angle modes may indicate angles within an angle range of 45° to −135° in a clockwise direction (i.e., a first angle range). The angle modes may be defined based on the 12 o'clock direction. At this time, intra-prediction mode index 2 indicates horizontal diagonal (HDIA) mode, intra-prediction mode index 18 indicates horizontal (HOR) mode, intra-prediction mode index 34 indicates diagonal (DIA) mode, intra-prediction mode index 50 indicates vertical (VER) mode, and intra-prediction mode index 66 indicates vertical diagonal (VDIA) mode.

[0075] Meanwhile, the previously set angle range may be set differently depending on the shape of the current block. For example, if the current block is a rectangular block, a wide angle mode indicating an angle greater than 45° or less than −135° in a clockwise direction may be further used. If the current block is a horizontal block, the angle mode may indicate an angle within an angle range of (45+offset1)° to (−135+offset1)° in a clockwise direction (i.e., a second angle range). In this case, an angle mode 67 to 80 outside the first angle range may be further used. Also, if the current block is a vertical block, the angle mode may indicate an angle within an angle range of (45−offset2)° to (−135−offset2)° in a clockwise direction (i.e., a third angle range). In this case, an angle mode -10 to -1 outside the first angle range may be further used. According to an embodiment of the present disclosure, the values ​​of offset1 and offset2 may be determined individually according to a ratio between the width and height of the rectangular block. Additionally, offset1 and offset2 may be positive numbers.

[0076] According to a further embodiment of the present disclosure, the plurality of angle modes constituting the intra prediction mode set may include a base angle mode and an extended angle mode, where the extended angle mode may be determined based on the base angle mode.

[0077] According to an embodiment, the basic angle mode may be a mode corresponding to an angle used in intra prediction of the existing HEVC (High Efficiency Video Coding) standard, and the extended angle mode may be a mode corresponding to an angle newly added in intra prediction of the next-generation video codec standard. More specifically, the basic angle mode may be an angle mode corresponding to any one of intra prediction modes {2, 4, 6, ..., 66}, and the extended angle mode may be an angle mode corresponding to any one of intra prediction modes {3, 5, 7, ..., 65}. That is, the extended angle mode may be an angle mode between the basic angle modes within the first angle range. Therefore, the angle indicated by the extended angle mode may be determined based on the angle indicated by the basic angle mode.

[0078] According to another embodiment, the basic angle mode may be a mode corresponding to an angle within a first angle range that has already been set, and the extension angle mode may be a wide angle mode outside the first angle range. That is, the basic angle mode may be an angle mode corresponding to any one of intra prediction modes {2, 3, 4, ..., 66}, and the extension angle mode may be an angle mode corresponding to any one of intra prediction modes {-10, -9, ..., -1} and {67, 68, ..., 76}. An angle indicated by the extension angle mode may be determined as an opposite angle to an angle indicated by the corresponding basic angle mode. Thus, an angle indicated by the extension angle mode may be determined based on an angle indicated by the basic angle mode. Meanwhile, the number of extension angle modes is not limited thereto, and further extension angles may be defined according to the size and / or shape of the current block. For example, the extension angle mode may be defined as an angle mode corresponding to any one of intra prediction modes {-14, -13, ..., -1} and {67, 68, ..., 80}. Meanwhile, the total number of intra prediction modes included in the intra prediction mode set may vary depending on the configuration of the base angle mode and the extended angle mode.

[0079] In the above embodiment, the interval between the extension angle modes may be set based on the interval between the corresponding basic angle modes. For example, the interval between the extension angle modes {3, 5, 7, ..., 65} may be determined based on the interval between the corresponding basic angle modes {2, 4, 6, ..., 66}. In addition, the interval between the extension angle modes {-10, -9, ..., -1} may be determined based on the interval between the corresponding opposite basic angle modes {56, 57, ..., 65}, and the interval between the extension angle modes {67, 68, ..., 76} may be determined based on the interval between the corresponding opposite basic angle modes {3, 4, ..., 12}. The angle interval between the extension angle modes may be set to be the same as the angle interval between the corresponding basic angle modes. In addition, the number of the extension angle modes in the intra prediction mode set may be set to be equal to or less than the number of the basic angle modes.

[0080] According to an embodiment of the present disclosure, the extended angle mode may be signaled based on the basic angle mode. For example, the wide angle mode (i.e., the extended angle mode) may replace at least one angle mode (i.e., the basic angle mode) within the first angle range. The replaced basic angle mode may be an angle mode corresponding to the opposite side of the wide angle mode. That is, the replaced basic angle mode is an angle mode corresponding to an angle in the opposite direction of the angle indicated by the wide angle mode or an angle that is offset from the opposite angle by a previously set offset index. According to an embodiment of the present disclosure, the previously set offset index is 1. The intra prediction mode index corresponding to the replaced basic angle mode may be remapped to the wide angle mode to signal the corresponding wide angle mode. For example, the wide angle modes {-10, -9, ..., -1} may be signaled by intra prediction mode indexes {57, 58, ..., 66}, respectively, and the wide angle modes {67, 68, ..., 76} may be signaled by intra prediction mode indexes {2, 3, ..., 11}, respectively. In this way, by making the intra prediction mode index for the base angular mode signal the extended angular mode, the same set of intra prediction mode indexes can be used to signal the intra prediction mode even if the configurations of the angular modes used for intra prediction of each block are different, and therefore, signaling overhead due to changes in the intra prediction mode configuration can be minimized.

[0081] Meanwhile, whether or not to use the extended angle mode may be determined based on at least one of the shape and size of the current block. According to one embodiment, if the size of the current block is larger than a pre-set size, the extended angle mode may be used for intra prediction of the current block, and if not, only the basic angle mode may be used for intra prediction of the current block. According to another embodiment, if the current block is a block other than a square, the extended angle mode may be used for intra prediction of the current block, and if the current block is a square block, only the basic angle mode may be used for intra prediction of the current block.

[0082] The intra prediction unit determines a reference sample and / or an interpolated reference sample to be used for intra prediction of the current block based on the intra prediction mode information of the current block. When the intra prediction mode index indicates a specific angle mode, a reference sample or an interpolated reference sample corresponding to the specific angle from the current sample of the current block is used for predicting the current sample. Thus, different sets of reference samples and / or interpolated reference samples may be used for intra prediction depending on the intra prediction mode. After intra prediction of the current block is performed using the reference sample and the intra prediction mode information, the decoder restores sample values ​​of the current block by adding a residual signal of the current block obtained from the inverse transform unit to an intra prediction value of the current block.

[0083] Hereinafter, an inter prediction method according to an embodiment of the present disclosure will be described with reference to Fig. 7. In the present disclosure, the inter prediction method may include a general inter prediction method optimized for translation motion and an affine model-based inter prediction method. Also, the motion vector may include at least one of a general motion vector for motion compensation according to the general inter prediction method and a control point motion vector for affine motion compensation.

[0084] FIG. 7 illustrates an inter-prediction method according to an embodiment of the present disclosure. As described above, the decoder can predict a current block by referring to a reconstructed sample of another decoded picture. Referring to FIG. 7, the decoder obtains a reference block 702 in a reference picture 720 based on a motion information set of a current block 701. At this time, the motion information set may include a reference picture index and a motion vector 703. The reference picture index indicates a reference picture 720 including a reference block for inter-prediction of the current block in a reference picture list. According to an embodiment, the reference picture list may include at least one of the above-mentioned L0 picture list or L1 picture list. The motion vector 703 indicates an offset between the coordinate value of the current block 701 in the current picture 710 and the coordinate value of the reference block 702 in the reference picture 720. The decoder obtains a predictor of the current block 701 based on a sample value of the reference block 702, and reconstructs the current block 701 using the predictor.

[0085] Specifically, the encoder may search for a block similar to the current block from a picture that is earlier in the reconstruction order, and obtain the reference block. For example, the encoder may search for a reference block that minimizes the sum of differences between the current block and sample values ​​within a previously set search area. At this time, at least one of SAD (Sum Of Absolute Difference) and SATD (Sum of Hadamard Transformed Difference) may be used to measure the similarity between samples of the current block and the reference block. Here, the SAD may be a value obtained by adding up all absolute values ​​of differences between sample values ​​included in two blocks. Also, the SATD may be a value obtained by adding up all absolute values ​​of Hadamard transform coefficients obtained by performing Hadamard transform on differences between sample values ​​included in two blocks.

[0086] Meanwhile, the current block may be predicted using one or more reference regions. As described above, the current block may be inter-predicted by a bi-prediction scheme using two or more reference regions. According to an embodiment, the decoder may obtain two reference blocks based on two sets of motion information of the current block. Also, the decoder may obtain a first predictor and a second predictor of the current block based on respective sample values ​​of the obtained two reference blocks. Also, the decoder may reconstruct the current block using the first predictor and the second predictor. For example, the decoder may reconstruct the current block based on a sample-by-sample average of the first predictor and the second predictor.

[0087] As described above, one or more motion information sets may be signaled for motion compensation of the current block. At this time, similarity between the motion information sets for each of the multiple blocks may be used. For example, the motion information set used for predicting the current block may be derived from the motion information set used for predicting any one of the other samples that have already been reconstructed. This allows the encoder and decoder to reduce signaling overhead.

[0088] For example, there may be a plurality of candidate blocks that may be predicted based on a motion information set that is the same as or similar to the motion information set of the current block. The decoder may generate a merge candidate list. The decoder may generate the merge candidate list based on the plurality of candidate blocks. Here, the merge candidate list may include candidates corresponding to samples that may be predicted based on a motion information set related to the motion information set of the current block among samples reconstructed before the current block. The merge candidate list may include spatial candidates or temporal candidates. The merge candidate list may be generated based on the position of a sample reconstructed before the current block. The sample reconstructed before the current block may be a neighboring block of the current block. The neighboring block of the current block may mean a block adjacent to the current block. The encoder and the decoder may configure the merge candidate list of the current block according to a predefined rule. In this case, the merge candidate lists respectively constructed by the encoder and the decoder may be the same as each other. For example, the encoder and the decoder may configure the merge candidate list of the current block based on the position of the current block in the current picture. In this disclosure, the position of a particular block refers to the relative position of the top-left sample of the particular block within the picture that contains the particular block.

[0089] 8 is a diagram illustrating a method for signaling a motion vector of a current block according to an embodiment of the present disclosure. According to an embodiment of the present disclosure, the motion vector of the current block may be derived from a motion vector predictor (MVP) of the current block. According to an embodiment, the motion vector predictor referenced to derive the motion vector of the current block may be obtained using a motion vector predictor (MVP) candidate list. The MVP candidate list may include a preset number of MVP candidates (Candidate 1, Candidate 2, ..., Candidate N).

[0090] According to an embodiment, the MVP candidate list may include at least one of spatial candidates and temporal candidates. The spatial candidates may be a motion information set used for predicting a neighboring block within a certain range from the current block in the current picture. The spatial candidates may be configured based on available neighboring blocks of the current block. Also, the temporal candidates may be a motion information set used for predicting a block in the current picture and another picture. For example, the temporal candidates may be configured based on a specific block corresponding to the position of the current block in a specific reference picture. In this case, the position of the specific block represents the position of the top-left sample of the specific block in the reference picture. According to a further embodiment, the MVP candidate list may include a zero motion vector. According to a further embodiment, a rounding process may be performed on the MVP candidates included in the MVP candidate list of the current block. In this case, a resolution of a motion vector difference value of the current block, which will be described later, may be used. For example, each of the MVP candidates of the current block may be rounded based on the resolution of the motion vector difference value of the current block.

[0091] In this disclosure, the MVP candidate list may include an advanced temporal motion vector prediction (ATMVP, or subblock-based temporal motion vector prediction, SbTMVP) list, a merge candidate list for merge inter prediction, a control point motion vector candidate list for affine motion compensation, a subblock-based temporal motion vector prediction (STMVP) list for subblock-based motion compensation, and combinations thereof.

[0092] According to an embodiment, the encoder 810 and the decoder 820 may construct an MVP candidate list for motion compensation of the current block. For example, among samples reconstructed prior to the current block, there may be candidates corresponding to samples that may be predicted based on the same or similar motion information set as the current block. The encoder 810 and the decoder 820 may construct an MVP candidate list for the current block based on a plurality of corresponding candidate blocks. In this case, the encoder 810 and the decoder 820 may construct the MVP candidate list according to a rule predefined between the encoder 810 and the decoder 820. That is, the MVP candidate lists constructed in the encoder 810 and the decoder 820 may be the same as each other.

[0093] Also, the predefined rule may be different depending on the prediction mode of the current block. For example, when the prediction mode of the current block is an affine model-based affine prediction mode, the encoder and decoder may configure the MVP candidate list of the current block using a first method based on the affine model. The first method may be a method of obtaining a control point motion vector candidate list. Meanwhile, when the prediction mode of the current block is a general inter prediction mode not based on the affine model, the encoder and decoder may configure the MVP candidate list of the current block using a second method not based on the affine model. In this case, the first method and the second method may be different methods.

[0094] The decoder 820 may derive a motion vector for the current block based on any one of at least one MVP candidates included in the MVP candidate list for the current block. For example, the encoder 810 may signal an MVP index indicating a motion vector predictor to be referenced to derive a motion vector for the current block. Signaling may mean that the encoder generates a signal as a bitstream and the decoder 820 parses it from the bitstream. The MVP index may include a merge index for the merge candidate list. The decoder 820 may obtain a motion vector predictor for the current block based on the signaled MVP index. The decoder 820 may derive a motion vector for the current block using the motion vector predictor. According to an embodiment, the decoder 820 may use the motion vector predictor obtained from the MVP candidate list as the motion vector for the current block without a separate motion vector difference value. For example, the decoder 820 may select a motion vector from the merge list based on the merge index.

[0095] The decoder 820 can reconstruct the current block based on the motion vector of the current block. The inter prediction mode in which the motion vector predictor obtained from the MVP candidate list is used as the motion vector of the current block without a separate motion vector difference value can be called a merge mode.

[0096] According to another embodiment, the decoder 820 may obtain another motion vector difference value for the motion vector of the current block. The decoder 820 may obtain the motion vector of the current block by summing the motion vector predictor obtained from the MVP candidate list and the motion vector difference value of the current block. In this case, the encoder 810 may signal a motion vector difference value (MV difference) representing the difference between the motion vector of the current block and the motion vector predictor. A method of signaling the motion vector difference value will be described in detail with reference to FIG. 9. The decoder 820 may obtain the motion vector of the current block based on the motion vector difference value (MV difference). The decoder 820 may restore the current block based on the motion vector of the current block.

[0097] Further, a reference picture index for motion compensation of the current block may be signaled. The encoder 810 may signal a reference picture index indicating a reference picture including a reference block. The decoder 820 may obtain a POC of a reference picture referenced for reconstruction of the current block based on the signaled reference picture index. At this time, the POC of the reference picture may be different from the POC of a reference picture corresponding to an MVP referenced to derive a motion vector of the current block. In this case, the decoder 820 may perform motion vector scaling. That is, the decoder 820 may scale the MVP to obtain an MVP'. At this time, the motion vector scaling may be performed based on the POC of the current picture, the POC of the signaled reference picture of the current block, and the POC of the reference picture corresponding to the MVP. Also, the decoder 820 may use the MVP' as a motion vector predictor of the current block.

[0098] As described above, the motion vector of the current block may be obtained by summing the motion vector predictor of the current block and the motion vector differential value. At this time, the motion vector differential value may be signaled from the encoder. The encoder may encode the motion vector differential value to generate information indicating the motion vector differential value and signal the information. Hereinafter, a method for signaling the motion vector differential value according to an embodiment of the present disclosure will be described.

[0099] 9 is a diagram illustrating a method for signaling a motion vector difference value of a current block according to an embodiment of the present disclosure. According to an embodiment, the information indicating the motion vector difference value may include at least one of absolute value information of the motion vector difference value or sign information of the motion vector difference value. The absolute value and sign of the motion vector difference value may be encoded separately.

[0100] According to one embodiment, the absolute value of the motion vector difference value may not be signaled by the value itself. The encoder may reduce the size of the signaled value using at least one flag that indicates a characteristic of the absolute value of the motion vector difference value. The decoder may derive the absolute value of the motion vector difference value from the signaled value using the at least one flag.

[0101] For example, the at least one flag may include a first flag indicating whether the absolute value of the motion vector difference value is greater than N, where N may be an integer. If the size of the absolute value of the motion vector difference value is greater than N, a value (absolute value of the motion vector difference value-N) may be signaled together with the activated first flag. In this case, the activated flag may indicate that the size of the absolute value of the motion vector difference value is greater than N. The decoder may obtain the absolute value of the motion vector difference value based on the activated first flag and the signaled value.

[0102] 9, a second flag (abs_mvd_greater0_flag) indicating whether the absolute value of the motion vector difference value is greater than '0' may be signaled. If the second flag (abs_mvd_greater0_flag[]) indicates that the absolute value of the motion vector difference value is not greater than '0', the absolute value of the motion vector difference value may be '0'. Also, if the second flag (abs_mvd_greater0_flag) indicates that the absolute value of the motion vector difference value is greater than '0', the decoder may obtain the absolute value of the motion vector difference value using other information on the motion vector difference value.

[0103] According to one embodiment, a third flag (abs_mvd_greater1_flag) may be signaled to indicate whether the absolute value of the motion vector differential value is greater than '1.' If the third flag (abs_mvd_greater1_flag) indicates that the absolute value of the motion vector differential value is not greater than '1,' the decoder may determine that the absolute value of the motion vector differential value is '1.'

[0104] Conversely, when the third flag (abs_mvd_greater1_flag) indicates that the absolute value of the motion vector difference value is greater than '1', the decoder can obtain the absolute value of the motion vector difference value using further information on the motion vector difference value. For example, a value (absolute value of the motion vector difference value - 2) (abs_mvd_minus2) may be signaled. This is because when the absolute value of the motion vector difference value is greater than '1', the absolute value of the motion vector difference value may be a value of 2 or more.

[0105] As described above, the absolute value of the motion vector difference value of the current block may be modified to at least one flag. For example, the modified absolute value of the motion vector difference value may be indicated as (absolute value of the motion vector difference value-N) depending on the size of the motion vector difference value. According to an embodiment, the modified absolute value of the motion vector difference value may be signaled by at least one bit. In this case, the number of bits signaled to indicate the modified absolute value of the motion vector difference value may be variable. The encoder may encode the modified absolute value of the motion vector difference value using a variable length binarization method. For example, the encoder may use at least one of truncated unary binarization, unary binarization, truncated rice, or exp-Golomb binarization as the variable length binarization method.

[0106] Alternatively, the sign of the motion vector difference value may be signaled by a sign flag (mvd_sign_flag), or alternatively, the sign of the motion vector difference value may be signaled implicitly by sign-bit-hiding (sign-bit-hiding).

[0107] In addition, in FIG. 9, [0] and [1] can indicate component indexes. For example, they can indicate the x-component and the y-component.

[0108] FIG. 10 is a diagram illustrating adaptive motion vector resolution signaling according to one embodiment of the present disclosure.

[0109] According to an embodiment of the present disclosure, the resolution at which a motion vector or a motion vector difference is indicated may vary. In other words, the resolution at which a motion vector or a motion vector difference is coded may vary. For example, the resolution may be indicated based on pixels. For example, a motion vector or a motion vector difference may be signaled in units of 1 / 4, 1 / 2, 1, 2, 4 pixels, etc. For example, when it is desired to indicate 16, it may be coded by 64 in units of 1 / 4 (1 / 4*64=16), by 1 unit (1*16=16), or by 4 units (4*4=16). That is, the value may be determined as follows:

[0110] valueDetermined=resolution*valuePerResolution

[0111] Here, valueDetermined may be a value to be transmitted, which in this embodiment may be a motion vector or a motion vector differential value, and valuePerResolution may be a value indicated in [ / resolution] units.

[0112] In this case, if the value signaled by the motion vector or motion vector differential value is not divisible by the resolution, an inaccurate value may be sent instead of the motion vector or motion vector differential value with the best prediction performance due to rounding, etc. Using a high resolution can reduce inaccuracy, but since the value to be coded is large, more bits may be used, and using a low resolution can increase inaccuracy, but since the value to be coded is small, fewer bits may be used.

[0113] Moreover, the resolution can be set individually for each unit such as a block, a CU, a slice, etc. Therefore, the resolution can be adaptively applied according to the unit.

[0114] The resolution may be signaled from the encoder to the decoder. In this case, the signaling for the resolution may be binarized to the variable length as described above. In this case, if the resolution is signaled with an index corresponding to the smallest value (the most forward value), the signaling overhead can be reduced.

[0115] As an example, the signaling indexes can be matched from high resolution (fine signaling) to low resolution.

[0116] Figure 10 shows signaling for three resolutions. In such a case, the three signalings can be 0, 10, and 11, and each of the three signalings can correspond to resolution 1, resolution 2, and resolution 3. One bit is required to signal resolution 1, and two bits are required to signal the remaining resolutions, so there is less signaling overhead when signaling resolution 1. In the example of Figure 10, resolution 1, resolution 2, and resolution 3 are 1 / 4, 1, and 4 pels, respectively.

[0117] In the following invention, motion vector resolution can mean the resolution of motion vector difference values.

[0118] FIG. 11 is a diagram illustrating an inter prediction related syntax according to an embodiment of the present disclosure.

[0119] According to an embodiment of the present disclosure, the inter prediction method may include a skip mode, a merge mode, an inter mode, etc. According to an embodiment, a residual signal may not be transmitted in the skip mode. Also, an MV determination method such as the merge mode may be used in the skip mode. Whether or not the skip mode is used may be determined by a skip flag. Referring to FIG. 11, whether or not the skip mode is used may be determined by a cu_skip_flag value.

[0120] According to an embodiment, in the merge mode, the motion vector difference value may not be used. The motion vector may be determined based on the motion vector index. Whether the merge mode is used may be determined by a merge flag. Referring to FIG. 11, whether the merge mode is used may be determined by a merge_flag value. Also, the merge mode can be used when the skip mode is not used.

[0121] In the skip mode or the merge mode, one or more types of candidate list may be selectively used. For example, a merge candidate or a subblock merge candidate may be used. The merge candidate may include a spatial neighboring candidate, a temporal candidate, etc. The merge candidate may include a candidate using a motion vector for the entire current block (CU). That is, the motion vector of each subblock belonging to the current block may include a candidate having the same motion vector. The subblock merge candidate may include a subblock-based temporal MV, an affine merge candidate, etc. The subblock merge candidate may include a candidate capable of using a different motion vector for each subblock of the current block (CU). The affine merge candidate may be a method of determining a control point motion vector of the affine motion prediction without using a motion vector difference value. The subblock merge candidate may include a method of determining a motion vector in a subblock unit in the current block. For example, the sub-block merging candidates may include planar MV, regression based MV, STMVP, etc. in addition to the sub-block-based temporal MV and affine merging candidates described above.

[0122] According to an embodiment, a motion vector difference value may be used in the inter mode. A motion vector predictor may be determined based on a motion vector index, and a motion vector may be determined based on the motion vector predictor and the motion vector difference value. Whether the inter mode is used may be determined based on whether other modes are used. In yet another embodiment, whether the inter mode is used may be determined by a flag. FIG. 11 illustrates an example in which the inter mode is used when the other modes, skip mode and merge mode, are not used.

[0123] The inter mode may include an AMVP mode, an affine inter mode, etc. The inter mode may be a mode in which a motion vector is determined based on a motion vector predictor and a motion vector difference value. The affine inter mode may be a method in which a motion vector difference value is used when determining a control point motion vector of affine motion prediction.

[0124] Referring to FIG. 11, after the skip mode or the merge mode is determined, it may be determined whether to use a subblock merge candidate or a merge candidate. For example, a merge_subblock_flag indicating whether to use a subblock merge candidate when a specific condition is satisfied may be parsed. In addition, the specific condition may be a condition related to a block size. For example, the condition may be a condition related to a width, height, area, etc., and may be used in combination. Referring to FIG. 11, for example, the condition may be a condition when the width and height of a current block (CU) are equal to or greater than a specific value. When parsing merge_subblock_flag, the value may be inferred to be 0. If merge_subblock_flag is 1, a subblock merge candidate may be used, and if it is 0, a merge candidate may be used. When a subblock merge candidate is used, a candidate index merge_subblock_idx may be parsed, and when a merge candidate is used, a candidate index merge_idx may be parsed. In this case, if the maximum number of candidate lists is 1, parsing may not be necessary. If merge_subblock_idx or merge_idx are not parsed, then it can be inferred to be 0.

[0125] FIG. 11 shows the coding_unit function, but the intra prediction related content may be omitted, and FIG. 11 may show the case where inter prediction is determined.

[0126] FIG. 12 is a diagram illustrating multi-hypothesis prediction according to one embodiment of the present disclosure.

[0127] As described above, encoding and decoding may be performed based on a prediction block. According to an embodiment of the present disclosure, a prediction block may be generated based on multiple predictions. This may be called multi-hypothesis (MH) prediction. The prediction may refer to a block generated by a certain prediction method. The prediction method in the multiple predictions may include intra prediction, inter prediction, and the like. Alternatively, the prediction method in the multiple predictions may be further subdivided to refer to a merge mode, an AMVP mode, a specific mode of intra prediction, and the like.

[0128] In addition, as a method for generating a prediction block based on multiple predictions, a weighted sum of multiple predictions can be used.

[0129] Or, the maximum number of multiple predictions may already be set. For example, the maximum number of multiple predictions may be 2. Therefore, it is possible to generate a prediction block from two predictions in the case of uni prediction, and from two predictions (when multiple predictions are used for prediction from only one reference list) or four predictions (when multiple predictions are used for prediction from two reference lists) in the case of bi prediction.

[0130] Alternatively, prediction modes available in a large number of predictions may already be set. Alternatively, prediction mode combinations available in a large number of predictions may already be set. For example, it is possible to use predictions generated by inter prediction and intra prediction. At this time, it is possible to use only some modes of inter prediction and intra prediction for multi-hypothesis prediction. For example, it is possible to use only merge modes of inter prediction for multi-hypothesis prediction. Alternatively, it is possible to use only merge modes other than sub-block merge modes of inter prediction for multi-hypothesis prediction. Alternatively, it is possible to use only specific intra modes of intra prediction for multi-hypothesis prediction. For example, it is possible to use only modes including planar, DC, vertical, and horizontal modes of intra prediction for multi-hypothesis prediction.

[0131] Thus, for example, it is possible to generate a prediction block based on a merge mode and prediction to intra prediction, where only planar, DC, vertical, and horizontal modes are allowed for intra prediction.

[0132] 12, a predicted block is generated based on prediction 1 and prediction 2. At this time, the predicted block is generated by a weighted sum of prediction 1 and prediction 2, and the weights of prediction 1 and prediction 2 are w1 and w2, respectively.

[0133] In addition, according to an embodiment of the present disclosure, when generating a prediction block based on multiple predictions, the weights of the multiple predictions may be based on the positions within the block, or the weights of the multiple predictions may be based on the mode of generating the predictions.

[0134] For example, if one of the modes for generating a prediction is intra prediction, the weighting value may be determined based on the prediction mode. For example, if one of the modes for generating a prediction is intra prediction and a directional mode, the weighting value of a position far from the reference sample may be increased. More specifically, if one of the modes for generating a prediction is intra prediction and a directional mode, and the other mode for generating a prediction is inter prediction, the weighting value of a prediction generated based on intra prediction on the far side of the reference sample may be increased. In the case of inter prediction, motion compensation may be performed based on a spatial neighbor candidate, in which case there is a probability that the motion of the current block and the spatial neighbor block referenced for MC are the same or similar, and there is a probability that the prediction of the area including an object having a motion with the prediction of the vicinity of the spatial neighbor block is more accurate than other parts. Then, more residual signals may remain near the opposite side of the spatial neighbor block than other parts, but this can be offset by using intra prediction in multi-hypothesis prediction. Since the reference sample position of intra prediction may be near the spatial neighbor candidate of inter prediction, the weighting value on the far side from there may be increased.

[0135] As another example, when one of the modes for generating a prediction is intra prediction and a direction mode, the weight of a position close to the reference sample may be increased. More specifically, when one of the modes for generating a prediction is intra prediction and a direction mode, and the other mode for generating a prediction is inter prediction, the weight of a prediction generated based on an intra prediction close to the reference sample may be increased. This is because intra prediction may have high prediction accuracy near the reference sample.

[0136] As yet another example, if one of the modes for generating the prediction is intra prediction and not a directional mode (e.g., planar, DC mode), the weighting value may be constant regardless of position in the block.

[0137] Also, in multi-hypothesis prediction, the weighting value for prediction 2 can be determined based on the weighting value for prediction 1.

[0138] The following is a formula that illustrates an example of determining the predicted samples, pbSamples, based on multiple predictions.

[0139] pbSamples[ x ][ y ] = Clip3( 0, ( 1 << bitDepth ) - 1, ( w * predSamples [ x ][ y ] + (8- w) * predSamplesIntra [ x ][ y ] ) >> 3 )

[0140] In the above equation, x and y may indicate the coordinates of a sample within a block and may be in the following range: x=0..nCbW-1 and y=0..nCbH-1. Also, nCbW and nCbH may be the width and height of the current block, respectively. Also, predSamples may be blocks / samples generated by inter prediction, and predSamplesIntra may be blocks / samples generated by intra prediction. Also, a weight value w may be determined as follows:

[0141] - If predModeIntra is INTRA_PLANAR or INTRA_DC or nCbW < 4 or nCbH < 4 or cIdx > 0, w is set equal to 4.

[0142] - Otherwise, if predModeIntra is INTRA_ANGULAR50 and y < (nCbH / 4), w is set equal to 6.

[0143] - Otherwise, if predModeIntra is INTRA_ANGULAR50 and (nCbH / 4) <= y < (nCbH / 2), w is set equal to 5.

[0144] - Otherwise, if predModeIntra is INTRA_ANGULAR50 and (nCbH / 2) <= y < (3 * nCbH / 4), w is set equal to 4.

[0145] - Otherwise, if predModeIntra is INTRA_ANGULAR50 and (3 * nCbH / 4) <= y < nCbH, w is set equal to 3.

[0146] - Otherwise, if predModeIntra is INTRA_ANGULAR18 and x < (nCbW / 4), w is set equal to 6.

[0147] - Otherwise, if predModeIntra is INTRA_ANGULAR18 and (nCbW / 4) <= x < (nCbW / 2), w is set equal to 5.

[0148] - Otherwise, if predModeIntra is INTRA_ANGULAR18 and (nCbW / 2) <= x < (3 * nCbW / 4), w is set equal to 4.

[0149] - Otherwise, if predModeIntra is INTRA_ANGULAR18 and (3 * nCbW / 4) <= x < nCbW, w is set equal to 3.

[0150] FIG. 13 is a diagram illustrating a multi-hypothesis prediction related syntax according to an embodiment of the present disclosure.

[0151] Referring to FIG. 13, mh_intra_flag may be a flag indicating whether or not multi-hypothesis prediction is used. According to an embodiment, multi-hypothesis prediction can be used only when mh_conditions are satisfied, and when mh_conditions are not satisfied, mh_intra_flag can be inferred as 0 without parsing. For example, mh_conditions can include a condition on a block size. Also, mh_conditions can include a condition on whether or not to use a specific mode. For example, when merge_flag is 1 and subblock_merge_flag is 0, mh_intra_flag can be parsed.

[0152] Also, according to an embodiment of the present disclosure, in order to determine a mode in multi-hypothesis prediction, it is possible to divide candidate modes into multiple lists and signal which list to use. With reference to FIG. 13, mh_intra_luma_mpm_flag may be a flag indicating which list among multiple lists to use. If mh_intra_luma_mpm_flag does not exist, it is possible to infer 1. Also, as an embodiment of the present disclosure, the multiple lists may be an MPM list and a non-MPM list.

[0153] Also, according to an embodiment of the present disclosure, an index can be signaled to indicate which index candidate is used in which list of the multiple lists. With reference to FIG. 13, mh_intra_luma_mpm_idx can be the index. Also, as an embodiment, the index can be signaled only when a specific list is selected. With reference to FIG. 13, mh_intra_luma_mpm_idx can be parsed only when any list is determined by mh_intra_luma_mpm_flag.

[0154] For example, as in one embodiment described in FIG. 12, multi-hypothesis prediction can be performed based on a prediction generated by inter prediction and a prediction generated by intra prediction. Also, multi-hypothesis prediction can be performed only when it is signaled that inter prediction is to be used. Or, multi-hypothesis prediction can be performed only when it is signaled that a specific mode of inter prediction, such as merge mode, is to be used. In that case, signaling for inter prediction does not need to be performed separately. Also, as one embodiment, when generating a prediction by intra prediction, there may be a total of four candidate modes. And, the total of four candidate modes may be divided into List1 and List2, three and one. At this time, when List2 is selected, there is no need to signal an index. Also, when List1 is selected, an index can be signaled, but since there are three candidates included in List1, it may be signaled by variable length coding, and one or two bits may be required for index signaling.

[0155] FIG. 14 is a diagram illustrating a multi-hypothesis prediction related syntax according to an embodiment of the present disclosure.

[0156] As described in FIG. 13, there may be signaling indicating which list to use, and in FIG. 13 and FIG. 14, this may be mh_intra_luma_mpm_flag.

[0157] According to an embodiment of the present disclosure, signaling indicating which list to use may be explicitly signaled only in specific cases. Also, if it is not explicitly signaled, the value of the signaling may be inferred according to a previously set method. With reference to FIG. 14, if the condition of mh_mpm_infer_condition is satisfied, there may be no explicit signaling, and if the condition is not satisfied, there may be explicit signaling. Also, if the condition of mh_mpm_infer_condition is satisfied, there may be no mh_intra_luma_mpm_flag, and in that case, it may be inferred to be 1. That is, it may be inferred that an MPM list is used.

[0158] FIG. 15 is a diagram illustrating a multi-hypothesis prediction related syntax according to an embodiment of the present disclosure.

[0159] As explained in Figures 13-14, there may be signaling indicating which list to use, and its value can be inferred if certain conditions are met.

[0160] According to one embodiment of the present disclosure, the condition for inferring the signaling value indicating which list to use may be based on the current block size. For example, it may be based on the width and height of the current block. More specifically, it is possible to infer the signaling value if the larger of the width and height of the current block is greater than n times the smaller of the width and height of the current block. For example, n may be 2.

[0161] Referring to Figure 15, a condition for inferring a signaling value indicating which list to use may be that the larger of the width and height of the current block is greater than twice the smaller. If the width and height of the current block are cbWidth and cbHeight, respectively, the Abs(Log2(cbWidth / cbHeight)) value is 0 when cbWidth and cbHeight are the same, and is 1 when there is a two-fold difference. Therefore, when the difference between cbWidth and cbHeight is greater than twice, the Abs(Log2(cbWidth / cbHeight)) value is greater than 1 (greater than 2).

[0162] FIG. 16 is a diagram illustrating a multihypothesis prediction mode determination method according to an embodiment of the present disclosure.

[0163] As described in FIG. 13 to FIG. 15, the mode decision may be made based on multiple lists. The mode may refer to an intra mode that generates a prediction based on intra prediction. The multiple lists may include List1 and List2. Referring to FIG. 16, whether List1 is used or not may be determined by List1_flag. There may be multiple candidates that may belong to List1, and there may be one candidate that may belong to List2. The multiple lists may be two lists.

[0164] If List1_flag is inferred, it can be inferred that the value is List1 to be used. In that case, List1_index, which is an index indicating which candidate in List1 is to be used, can be parsed. Also, if List1_flag is not inferred, List1_flag can be parsed. If List1_flag is 1, List1_index can be parsed, and if List1_flag is not 1, the index does not need to be parsed. Also, if List1_flag is 1, the mode to be actually used can be determined from the candidate modes in List1 based on the index. Also, if List1_flag is not 1, the candidate mode in List2 can be determined as the mode to be actually used without the index. That is, in List1, the mode can be determined based on the flag and index, and in List2, the mode can be determined based on the flag.

[0165] FIG. 17 is a diagram illustrating a candidate list generation method according to an embodiment of the present disclosure.

[0166] According to an embodiment of the present disclosure, in order to improve coding efficiency when variable length coding an index that determines a candidate mode in a list, a method for determining an order of modes included in a candidate list may exist. For example, a method for determining an order of modes included in List1 may exist. In this case, modes around the current block may be referenced to determine the order of modes. Also, List2 may be determined without reference to modes around the current block. For example, List1 may be generated with reference to modes around the current block, and modes not included in List1 may be included in List2.

[0167] In addition, List 1 may be an MPM mode and List 2 may be a non-MPM mode. In addition, there may be four candidate modes in total, with List 1 including three modes and List 2 including one mode.

[0168] Referring to FIG. 17, there may be List1_flag, which is signaling indicating whether List1 is used. If List1 is used, List1 may be generated and a mode in List1 may be selected. At this time, the generation of List1 and confirmation of whether List1 is used may be in any order. However, in a situation where List1 is used, List1 may be generated before or after confirmation of whether List1 is used. Also, when List1 is used, the process of generating List2 does not need to be performed. If List1 is not used, List2 may be generated and a mode in List2 may be selected. At this time, it is possible to generate List1 in order to generate List2. And, it is possible to include candidates not included in List1 among the candidate modes in List2.

[0169] Furthermore, according to an embodiment of the present disclosure, the method of generating List1 may be the same regardless of whether List1 is used (List1_flag value), whether it is inferred whether List1 is used, and so on.

[0170] At this time, the list signaling and mode signaling can follow the embodiments described above with reference to FIG.

[0171] Next, the method for generating the multiple lists described in Figures 16 and 17 will be further described. The multiple lists may include List 1 and List 2. The multiple lists may be lists used in a multi-hypothesis prediction process.

[0172] According to an embodiment of the present disclosure, multiple lists may be generated by referring to modes around a current block. Also, intra prediction may be performed using a mode selected from the multiple lists, and the intra prediction may be combined with inter prediction (multi-hypothesis prediction) to be used as a predicted block. As an embodiment, modes (candidate modes) that may be included in the multiple lists may be planar mode, DC mode, vertical mode, and horizontal mode of an intra prediction method. Also, the vertical mode may be the mode of index 50 in FIG. 6, and the horizontal mode may be the mode of index 18 in FIG. 6. Also, the planar mode and DC mode may have indexes 0 and 1, respectively.

[0173] According to an embodiment of the present disclosure, candModeList may be generated by referring to the mode around the current block. Also, candModeList may be List1 described in the previous embodiment. As an example, there may be candIntraPredModeX, which is a mode around the current block or a mode based on the mode around the current block. Here, X may be a letter such as A or B to indicate that the mode corresponds to a specific position around the current block.

[0174] As an example, candModeList can be generated based on whether multiple candIntraPredModeX are consistent. For example, candIntraPredModeX may exist for two positions, which can be represented as candIntraPredModeA and candIntraPredModeB. If candIntraPredModeA and candIntraPredModeB are identical, candModeList can include planar mode and DC mode.

[0175] If candIntraPredModeA and candIntraPredModeB are the same and the value indicates a planar mode or a DC mode, the mode indicated by candIntraPredModeA and candIntraPredModeB can be added to candModeList. In this case, among the planar mode and the DC mode, a mode not indicated by candIntraPredModeA and candIntraPredModeB can be added to candModeList. In this case, a mode that has already been set, rather than a planar mode or a DC mode, can be added to candModeList. As an example, in this case, the order of the planar mode, the DC mode, and the already set mode in candModeList may already be set. For example, the order may be planar, DC, and the already set mode. That is, candModeList[0]=planar mode, candModeList[1]=DC mode, candModeList[2]=the already set mode. In addition, the already set mode may be a vertical mode. As yet another embodiment, in this case, among the planar mode, DC mode, and the already set modes, the modes indicated by candIntraPredModeA and candIntraPredModeB are positioned at the top of the candModeList, and among the planar mode and DC mode, the modes not indicated by candIntraPredModeA and candIntraPredModeB are positioned next in the candModeList, and the already set modes are positioned next after that.

[0176] Also, if candIntraPredModeA and candIntraPredModeB are the same and the values ​​do not indicate the planar mode and the DC mode, the mode indicated by candIntraPredModeA and candIntraPredModeB can be added to candModeList. Also, the planar mode and the DC mode may be added to candModeList. Also, in this case, the order of the mode indicated by candIntraPredModeA and candIntraPredModeB, the planar mode, and the DC mode in candModeList may already be set. Also, the already set order may be the mode indicated by candIntraPredModeA and candIntraPredModeB, the planar mode, and the DC mode. That is, candModeList[0]=candIntraPredModeA, candModeList[1]=planar mode, and candModeList[2]=DC mode.

[0177] Also, if candIntraPredModeA and candIntraPredModeB are different, both candIntraPredModeA and candIntraPredModeB can be added to candModeList. Also, candIntraPredModeA and candIntraPredModeB may be included in candModeList according to a specific order. For example, candIntraPredModeA, candIntraPredModeB may be included in candModeList in that order. Also, there may be a pre-set order between the candidate modes, and among the modes according to the pre-set order, modes other than candIntraPredModeA and candIntraPredModeB may be added to candModeList. Also, the modes other than candIntraPredModeA and candIntraPredModeB may be positioned after candIntraPredModeA and candIntraPredModeB in candModeList. Also, the pre-set order may be planar mode, DC mode, and vertical mode. Alternatively, the previously set order may be planar mode, DC mode, vertical mode, and horizontal mode, i.e., candModeList[0]=candIntraPredModeA, candModeList[1]=candIntraPredModeB, and candModeList[2] may be the top mode among planar mode, DC mode, and vertical mode that is neither candIntraPredModeA nor candIntraPredModeB.

[0178] Also, among the candidate modes, a mode not included in candModeList may become candIntraPredModeC. Also, candIntraPredModeC may be included in List2. Also, this can be determined when the signaling indicating whether or not List1 is used indicates that it is not used.

[0179] Also, when List1 is used, it is possible to determine the mode by the index in candModeList, and when List1 is not used, the mode in List2 is used.

[0180] Also, as described above, after generating candModeList, a process of modifying candModeList may be added. For example, the process of modifying candModeList may be further performed or not depending on the size condition of the current block. For example, the size condition of the current block may be based on the width and height of the current block. For example, if the larger of the width and height of the current block is greater than n times the smaller of the width and height of the current block, the process of modifying can be further performed. n may be 2.

[0181] Also, the modifying process may be a process of replacing a mode with another mode when a mode is included in candModeList. For example, when a vertical mode is included in candModeList, a horizontal mode may be included in candModeList instead of the vertical mode. Or, when a vertical mode is included in candModeList, the candIntraPredModeC may be included in candModeList instead of the vertical mode. However, when the above-mentioned candModeList is generated, the planar mode and the DC mode may always be included in candModeList, and in this case, candIntraPredModeC may be a horizontal mode. Also, such a modifying process may be used when the height of the current block is greater than n times the width. For example, n may be 2. This is because when the height is greater than the width, the accuracy of the vertical mode may be low because the lower part of the block is far from the reference sample for intra prediction. Or, such a modifying process may be used when it is inferred that List1 is used.

[0182] As another example of the modification process, when the horizontal mode is included in the candModeList, the vertical mode can be included in the candModeList instead of the horizontal mode. Or, when the horizontal mode is included in the candModeList, the candIntraPredModeC can be included in the candModeList instead of the horizontal mode. However, when the above-mentioned candModeList is generated, the planar mode and the DC mode may always be included in the candModeList, and in this case, the candIntraPredModeC may be the vertical mode. Also, such a modification process may be used when the width of the current block is greater than n times the height. For example, n may be 2. This is because, when the width is greater than the height, the right side of the block is far from the reference sample for intra prediction, and therefore the accuracy of the horizontal mode may be low. Or, such a modification process may be used when it is inferred that List1 is used.

[0183] An example of the above-mentioned list setting method is described again below. In the following, IntraPredModeY may be a mode used for intra prediction in multi-hypothesis prediction. It may also be a mode of a luma component. As an example, the intra prediction mode of a chroma component in multi-hypothesis prediction may follow the mode of the luma component. Also, mh_intra_luma_mpm_flag may be signaling indicating which list to use. That is, for example, it may be mh_intra_luma_mpm_flag in Figs. 13 to 15, or List1_flag in Figs. 16 to 17. Also, mh_intra_luma_mpm_idx may be an index indicating which candidate in the list to use. That is, for example, it may be mh_intra_luma_mpm_idx in Figs. 13 to 15, or List1_index in Fig. 16. Additionally, xCb, yCb may be the x and y coordinates of the top-left corner of the current block, and cbWidth and cbHeight may be the width and height of the current block.

[0184] The candModeList[ x ] with x = 0..2 is derived as follows:

[0185] A. If candIntraPredModeB is equal to candIntraPredModeA, the following applies:

[0186] a. If candIntraPredModeA is less than 2 (i.e., equal to INTRA_PLANAR or INTRA_DC), candModeList[ x ] with x = 0..2 is derived as follows:

[0187] candModeList

[0000] = INTRA_PLANAR

[0188] candModeList

[0001] = INTRA_DC

[0189] candModeList

[0002] = INTRA_ANGULAR50

[0190] b. Otherwise, candModeList[ x ] with x = 0..2 is derived as follows:

[0191] candModeList

[0000] = candIntraPredModeA

[0192] candModeList

[0001] = INTRA_PLANAR

[0193] candModeList

[0002] = INTRA_DC

[0194] B. Otherwise (candIntraPredModeB is not equal to candIntraPredModeA), the following applies:

[0195] a. candModeList

[0000] and candModeList

[0001] are derived as follows:

[0196] candModeList

[0000] = candIntraPredModeA

[0197] candModeList

[0001] = candIntraPredModeB

[0198] b. If neither of candModeList

[0000] and candModeList

[0001] is equal to INTRA_PLANAR, candModeList

[0002] is set equal to INTRA_PLANAR,

[0199] c. Otherwise, if neither of candModeList

[0000] and candModeList

[0001] is equal to INTRA_DC, candModeList

[0002] is set equal to INTRA_DC,

[0200] d. Otherwise, candModeList

[0002] is set equal to INTRA_ANGULAR50.

[0201] IntraPredModeY[ xCb ][ yCb ] is derived by applying the following procedure:

[0202] A. If mh_intra_luma_mpm_flag[ xCb ][ yCb ] is equal to 1, the IntraPredModeY[ xCb ][ yCb ] is set equal to candModeList[ intra_luma_mpm_idx[ xCb ][ yCb ] ].

[0203] B. Otherwise, IntraPredModeY[ xCb ][ yCb ] is set to equal to candIntraPredModeC, derived by applying the following steps:

[0204] a. If neither of candModeList[ x ], x = 0..2 is equal to INTRA_PLANAR, candIntraPredModeC is set equal to INTRA_PLANAR,

[0205] b. Otherwise, if neither of candModeList[ x ], x = 0..2 is equal to INTRA_DC, candIntraPredModeC is set equal to INTRA_DC,

[0206] c. Otherwise, if neither of candModeList[ x ], x = 0..2 is equal to INTRA_ANGULAR50, candIntraPredModeC is set equal to INTRA_ANGULAR50,

[0207] d. Otherwise, if neither of candModeList[ x ], x = 0..2 is equal to INTRA_ANGULAR18. candIntraPredModeC is set equal to INTRA_ANGULAR18,

[0208] The variable IntraPredModeY[ x ][ y ] with x = xCb..xCb + cbWidth - 1 and y = yCb..yCb + cbHeight - 1 is set to be equal to IntraPredModeY[ xCb ][ yCb ]. One additional setting is when cbHeight is larger than double of cbWidth, mh_intra_luma_mpm_flag[ xCb ][ yCb ] is inferred to be 1 and if candModeList[ x ], x = 0..2 is equal to INTRA_ANGULAR50, candModeList[ x ] is replaced with candIntraPredModeC. Another additional setting is when cbWidth is larger than double of cbHeight, mh_intra_luma_mpm_flag[ xCb ][ yCb ] is inferred to be 1 and if candModeList[ x ], x = 0..2 is equal to INTRA_ANGULAR18, candModeList[ x ] is replaced with candIntraPredModeC.

[0209] FIG. 18 is a diagram illustrating surrounding positions referenced in multi-hypothesis prediction according to an embodiment of the present disclosure.

[0210] As described above, the surrounding positions can be referenced in the process of creating a candidate list for multi-hypothesis prediction. For example, the above-mentioned candIntraPredModeX may be necessary. At this time, the positions A and B around the current block to be referenced may be NbA and NbB shown in FIG. 18. That is, they may be immediately to the left and immediately above the upper left end of the current block. If the position of the upper left end of the current block is Cb as shown in FIG. 18 and its coordinates are (xCb, yCb), NbA may be (xNbA, yNbA)=(xCb-1, yCb), and NbB may be (xNbB, yNbB)=(xCb, yCb-1).

[0211] FIG. 19 is a diagram illustrating a method for referring to a surrounding mode according to one embodiment of the present disclosure.

[0212] As described above, the neighboring positions can be referred to in the process of generating a candidate list for multi-hypothesis prediction. In addition, the list can be generated using the neighboring mode as is or using a mode based on the neighboring mode. The mode obtained by referring to the neighboring positions may be candIntraPredModeX.

[0213] In one embodiment, if the surrounding position is unavailable, candIntraPredModeX may be the previously set mode. Unavailable cases may include cases where the surrounding position uses inter prediction, where the mode decision has not been made in the predetermined decoding or encoding order, etc.

[0214] Alternatively, candIntraPredModeX may be the already set mode if the surrounding positions do not use multi-hypothesis prediction.

[0215] Alternatively, if the peripheral position is above the CTU to which the current block belongs, candIntraPredModeX may become the already set mode. As yet another example, if the peripheral position is outside the CTU to which the current block belongs, candIntraPredModeX may become the already set mode.

[0216] In addition, as one embodiment, the preset mode may be a DC mode, and as yet another embodiment, the preset mode may be a planar mode.

[0217] Also, candIntraPredModeX can be set depending on whether the mode at the peripheral position exceeds a threshold angle or whether the index of the mode at the peripheral position exceeds a threshold. For example, if the index of the mode at the peripheral position is greater than a diagonal mode index, candIntraPredModeX can be set to a vertical mode index. Also, if the index of the mode at the peripheral position is less than or equal to the diagonal mode index and is a directional mode, candIntraPredModeX can be set to a horizontal mode index. The diagonal mode index may be mode 34 in FIG. 6.

[0218] Also, if the mode at the peripheral position is the planar mode or the DC mode, candIntraPredModeX may be set to the planar mode or the DC mode as it is.

[0219] Referring to FIG. 19, mh_intra_flag may be signaling indicating whether or not multi-hypothesis prediction is used. Also, the intra prediction mode used in the neighboring block may be X. Also, since the current block can use multi-hypothesis prediction and a candidate list can be generated using candIntraPredMode based on the mode of the neighboring block, but the neighboring block does not use multi-hypothesis prediction, candIntraPredMode can be set to the DC mode, which is the already set mode, regardless of the intra prediction mode of the neighboring block and whether or not the neighboring block used intra prediction.

[0220] An example of the peripheral mode referencing method described above is restated below.

[0221] For X being replaced by either A or B, the variables candIntraPredModeX are derived as follows:

[0222] 1. The availability derivation process for a block as specified in Neighbouring blocks availability checking process is invoked with the location ( xCurr, yCurr ) set equal to ( xCb, yCb ) and the neighbouring location ( xNbY, yNbY ) set equal to ( xNbX, yNbX ) as inputs, and the output is assigned to availableX.

[0223] 2. The candidate intra prediction mode candIntraPredModeX is derived as follows:

[0224] A. If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_DC.

[0225] a. The variable availableX is equal to FALSE.

[0226] b. mh_intra_flag[ xNbX ][ yNbX ] is not equal to 1.

[0227] c. X is equal to B and yCb - 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

[0228] B. Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] > INTRA_ANGULAR34, candIntraPredModeX is set equal to INTRA_ANGULAR50.

[0229] C. Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] <= INTRA_ANGULAR34 and IntraPredModeY[ xNbX ][ yNbX ] > INTRA_DC, candIntraPredModeX is set equal to INTRA_ANGULAR18.

[0230] D. Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

[0231] In the above-mentioned list setting method, candIntraPredModeX may be determined by the surrounding mode reference method.

[0232] FIG. 20 is a diagram illustrating a candidate list generation method according to an embodiment of the present disclosure.

[0233] According to the List1 and List2 generation method described in FIG. 13 to FIG. 17, List1 is generated by referring to the modes around the current block, and the candidate modes not included in List1 can be put into List2. Since there is spatial similarity in the picture, the modes referring to the surrounding modes may have a higher priority. That is, List1 may have a higher priority than List2. However, according to the List1 and List2 signaling method described in FIG. 13 to FIG. 17, when the signaling for determining the list is not inferred, the mode of List1 is used, so that the flag and index are used for signaling, and the mode of List2 is used, so that only the flag can be used. That is, the signaling of List2 can use relatively few bits. However, it may be bad in terms of coding efficiency if relatively many bits are used for signaling for the modes of the high priority list. Therefore, there may be a method of using relatively few bits for signaling for the high priority list and modes as disclosed herein.

[0234] According to an embodiment of the present disclosure, the method of generating the candidate list may differ depending on whether only List1 can be used. Whether only List1 can be used may indicate whether signaling indicating the list to be used is inferred. For example, if there is List3 that has a candidate mode and is generated by a method that has already been set, List3 may be divided into List1 and List2. For example, List3 generated by a method that has already been set and the generation method may be the above-mentioned candModeList and its generation method. If signaling indicating the list to be used is inferred, only List1 may be used, and in that case, List1 may be filled from the beginning of List3. Also, if signaling indicating the list to be used is not inferred, List1 or List2 may be used, and in that case, List2 may be filled from the beginning of List3, and the remainder may be filled in List1. Also, in this case, when filling List1, it is possible to fill it in the order of List3. That is, candIntraPredModeX can be put into the candidate list by referring to the mode around the current block, but candIntraPredModeX can be put into List1 when signaling indicating the list is inferred, and can be put into List2 when it is not inferred. The size of List2 may be 1, in which case candIntraPredModeA can be put into List1 when signaling indicating the list is inferred, and can be put into List2 when it is not inferred. candIntraPredModeA may be List3[0], which is the top mode of List3. Therefore, in the present disclosure, candIntraPredModeA, which is a mode based on the surrounding mode, may be put into both List1 and List2 as it is depending on the case. On the other hand, in the method described in FIG. 13 to FIG. 17, candIntraPredModeA can be put into only List1. Also, in the present disclosure, the method of generating List1 differs depending on whether signaling indicating the list to be used is inferred or not.

[0235] Referring to FIG. 20, the candidate mode may be a candidate that can be used to generate intra-prediction of multi-hypothesis prediction. The method of generating the candidate list may differ depending on whether List1_flag, which is signaling indicating the list to be used, is inferred. If it is inferred, it may be inferred when List1 is used, and since only List1 can be used, the top of List3 can be added to List1. When generating List3, a mode based on a surrounding mode can be added to the top. Also, if it is not inferred, both List1 and List2 are not used, so the top of List3 can be added to List2, which has less signaling. Then, when List1 is necessary, for example, when it is signaled that List1 is used, List3 other than those included in List2 can be added to List1.

[0236] In this disclosure, there are some parts that have been described using List3, but this is merely a conceptual explanation, and it is possible to generate List1 and List2 without actually saving List3.

[0237] According to the embodiment below, the method of creating a candidate list and the method of creating a candidate list described in FIG. 16 to FIG. 17 can be used depending on the situation. For example, one of two methods of creating a candidate list can be selected depending on whether signaling indicating which list to use is inferred. This may be the case of multi-hypothesis prediction. Also, List1 below may contain three modes, and List2 may contain one mode. Also, as for the mode signaling method, as described in FIG. 13 to FIG. 16, the mode of List1 can be signaled by a flag and an index, and the mode of List2 can be signaled by a flag.

[0238] As an example, candIntraPredModeA and candIntraPredModeB are the same, and when candIntraPredModeA is planar mode or DC mode, List2[0]=planar mode, List1[0]=DC mode, List1[1]=vertical mode, and List1[2]=horizontal mode.

[0239] As yet another example, candIntraPredModeA and candIntraPredModeB may be the same, and when candIntraPredModeA is planar mode or DC mode, List2[0]=candIntraPredModeA, List1[0]=!candIntraPredModeA, List1[1]=vertical mode, and List1[2]=horizontal mode.

[0240] As an example, candIntraPredModeA and candIntraPredModeB may be the same, and when candIntraPredModeA is a directional mode, List2[0]=candIntraPredModeA, List1[0]=planar mode, List1[1]=DC mode, and List1[2]=vertical mode.

[0241] As an example, if candIntraPredModeA and candIntraPredModeB are different, then List2[0]=candIntraPredModeA and List1[0]=candIntraPredModeB. List1[1] and List1[2] can be filled from planar modes, DC modes, vertical modes, and horizontal modes that are not candIntraPredModeA and candIntraPredModeB.

[0242] FIG. 21 is a diagram illustrating a candidate list generation method according to an embodiment of the present disclosure.

[0243] In the previous embodiment, a method for determining a mode based on multiple lists was described. In the invention of Fig. 21, a mode can be determined based on one list instead of multiple lists.

[0244] Referring to Fig. 21, as shown in Fig. 21(a), one candidate list including all candidate modes of multi-hypothesis prediction can be generated. Since there is one candidate list, referring to Fig. 21(b), there is no signaling to select the list, and there may be index signaling indicating which mode of the candidate list to use. Therefore, when mh_intra_flag indicating whether or not multi-hypothesis prediction is used is 1, mh_intra_luma_idx, which is a candidate index, can be parsed.

[0245] According to one embodiment, a candidate list generation method for multi-hypothesis prediction may be based on an MPM list generation method in existing intra prediction.

[0246] According to one embodiment, the method of generating a candidate list for multi-hypothesis prediction may be a method of generating List1 and List2 described above with reference to FIG. 17, in which List1 and List2 are joined in that order.

[0247] That is, if the candidate list of multi-hypothesis prediction is candModeList, the size of candModeList may be 4 in this embodiment. If candIntraPredModeA and candIntraPredModeB are the same and are planar mode or DC mode, candModeList may be determined according to a previously set order. For example, candModeList[0]=planar mode, candModeList[1]=DC mode, candModeList[2]=vertical mode, candModeList[3]=horizontal mode.

[0248] As yet another example, if candIntraPredModeA and candIntraPredModeB are the same and are planar mode or DC mode, candModeList[0]=candIntraPredModeA, candModeList[1]=!candIntraPredModeA, candModeList[2]=vertical mode, and candModeList[3]=horizontal mode.

[0249] If candIntraPredModeA and candIntraPredModeB are the same and are directional modes, candModeList[0] = candIntraPredModeA, candModeList[1] = planar mode, candModeList[2] = DC mode, candModeList[3] = candIntraPredModeA, or a mode other than planar mode and DC mode.

[0250] If candIntraPredModeA and candIntraPredModeB are different, candModeList[0]=candIntraPredModeA and candModeList[1]=candIntraPredModeB. Also, candModeList[2] and candModeList[3] may sequentially include modes other than candIntraPredModeA and candIntraPredModeB according to a preset order of the candidate modes. The preset order may be planar mode, DC mode, vertical mode, and horizontal mode.

[0251] According to an embodiment of the present disclosure, the candidate list may differ depending on the block size condition. If the relatively larger of the block width and height is greater than n times the remaining one, the candidate list may be shorter. For example, if the width is greater than n times the height, the horizontal mode may be removed from the candidate list described in FIG. 21, and the next mode may be moved up to fill the candidate list. Also, if the height is greater than n times the width, the vertical mode may be removed from the candidate list described in FIG. 21, and the next mode may be moved up to fill the candidate list. Thus, if the width is greater than n times the height, the candidate list size may be 3. Also, if the width is greater than n times the height, the size of the candidate list may be smaller or the same as when the width is not greater than n times the height.

[0252] According to one embodiment of the present disclosure, the candidate index in the embodiment of Figure 21 may be variable length coded to improve signaling efficiency by populating the top of the list with modes that are more likely to be used.

[0253] According to yet another embodiment, the candidate indexes in the embodiment of FIG. 21 may be fixed length coded. The number of modes used in multi-hypothesis prediction may be an exponential number of 2. For example, as described above, one of four intra prediction modes may be used. In this case, since no unassigned values ​​are generated even when fixed length coding is performed, no unnecessary parts are generated in signaling. In addition, when fixed length coding is performed, the number of list constructions may be one. This is because the number of bits is the same regardless of which index is signaled.

[0254] According to an embodiment, the candidate index may be variable length coded or fixed length coded depending on the case. For example, as in the previous embodiment, the candidate list size may be changed depending on the case. As an example, the candidate index may be variable length coded or fixed length coded depending on the candidate list size. For example, if the candidate list size is a power of 2, the candidate index may be fixed length coded, and if the candidate list size is not a power of 2, the candidate index may be variable length coded. That is, according to the previous embodiment, the coding method may be changed depending on the block size condition.

[0255] According to an embodiment of the present disclosure, when using multi-hypothesis prediction, if DC mode is used, the weights between multiple predictions are the same for the entire block, so it may be the same as adjusting the weights of the predicted block. Therefore, DC mode can be excluded in multi-hypothesis prediction.

[0256] As an example, it is possible to use only one of planar mode, vertical mode, and horizontal mode in multi-hypothesis prediction. In this case, as shown in FIG. 21, it is possible to signal multi-hypothesis prediction using one list. Also, variable length coding can be used for index signaling. As an example, the list can be generated in a fixed order. For example, the order may be planar mode, vertical mode, and horizontal mode. As another example, the list can be generated with reference to the modes around the current block. For example, if candIntraPredModeA and candIntraPredModeB are the same, candModeList[0]=candIntraPredModeA. If candIntraPredModeA and candIntraPredModeB are the same and candIntraPredModeA is a planar mode, candModeList[1] and candModeList[2] can be set according to the previously set order. If candIntraPredModeA and candIntraPredModeB are the same and candIntraPredModeA is not the planar mode, candModeList[1] = planar mode, candModeList[2] = candIntraPredModeA, and a mode other than the planar mode may be used. If candIntraPredModeA and candIntraPredModeB are different, candModeList[0] = candIntraPredModeA, candModeList[1] = candIntraPredModeB, and candModeList[2] = candIntraPredModeA, and a mode other than candIntraPredModeB may be used.

[0257] As yet another embodiment, it is possible to use only one of three modes in multi-hypothesis prediction. The three modes may include a planar mode and a DC mode. The three modes may also include one of a vertical mode and a horizontal mode depending on a condition. The condition may be a condition related to a block size. For example, it is possible to determine whether to include a horizontal mode or a vertical mode depending on which of a width and a height of a block is larger. For example, a vertical mode may be included when a width of a block is larger than a height. A horizontal mode may be included when a height of a block is larger than a width. When a height and a width of a block are the same, a promised mode of the vertical mode or the horizontal mode may be included.

[0258] As an example, the list may be generated in a fixed order. For example, the order may be planar mode, DC mode, and vertical or horizontal mode. As another example, the list may be generated with reference to the modes around the current block. For example, if candIntraPredModeA and candIntraPredModeB are the same, candModeList[0]=candIntraPredModeA. If candIntraPredModeA and candIntraPredModeB are the same and candIntraPredModeA is not a directional mode, candModeList[1] and candModeList[2] may be set according to the previously set order. If candIntraPredModeA and candIntraPredModeB are the same and candIntraPredModeA is a directional mode, candModeList[1]=planar mode and candModeList[2]=DC mode. If candIntraPredModeA and candIntraPredModeB are different, candModeList[0] may be a mode other than candIntraPredModeB, rather than candIntraPredModeA, candModeList[1] = candIntraPredModeB, and candModeList[2] = candIntraPredModeA.

[0259] According to yet another embodiment of the present disclosure, it is possible to use only one of two modes in the multi-hypothesis prediction. The two modes may include a planar mode. The two modes may include one of a vertical mode and a horizontal mode depending on a condition. The condition may be a condition related to a block size. For example, it is possible to determine whether to include a horizontal mode or a vertical mode depending on which of a width and a height of a block is larger. For example, if the width of a block is larger than the height, it may include a vertical mode. If the height of a block is larger than the width, it may include a horizontal mode. If the height and width of a block are the same, it may include a promised mode of the vertical mode or the horizontal mode. In this case, a flag may be signaled to indicate which mode is to be used in the multi-hypothesis prediction. According to an embodiment, a specific mode may be excluded depending on a block size. For example, if the block size is small, a specific mode may be excluded. For example, if the block size is small, it is possible to use only a planar mode in the multi-hypothesis prediction. If a specific mode is excluded, it is possible to signal by omitting or reducing mode signaling.

[0260] According to yet another embodiment of the present disclosure, it is possible to use only one mode in multi-hypothesis prediction. The one mode may be a planar mode. As yet another embodiment, the one mode may be determined from among a vertical mode and a horizontal mode based on a block size. For example, the one mode may be determined from among a vertical mode and a horizontal mode depending on which of the width and height of the block is larger. For example, if the width of the block is larger than the height, the vertical mode may be determined, and if the height of the block is larger than the width, the horizontal mode may be determined. If the width and height of the block are the same, the mode may be determined to be a mode that has already been set. If the width and height of the block are the same, the mode may be determined to be a mode that has already been set among the horizontal mode or the vertical mode. If the width and height of the block are the same, the mode may be determined to be a mode that has already been set among the planar mode or the DC mode.

[0261] Also, according to an embodiment of the present disclosure, there may be a flipping signaling for flipping a prediction generated in multi-hypothesis prediction. This can have the effect of eliminating the opposite residual by flipping even if one mode is selected in multi-hypothesis prediction. This can also have the effect of reducing the number of candidate modes available in multi-hypothesis prediction. More specifically, for example, in the above embodiment, flipping can be used when only one mode is used. This can improve prediction performance. The flipping can mean flipping about the x-axis, flipping about the y-axis, or flipping about both the x-axis and the y-axis. As an example, the flipping direction can be determined based on the mode selected in multi-hypothesis prediction. For example, when the mode selected in multi-hypothesis prediction is a planar mode, it can be determined that the flipping is about both the x-axis and the y-axis. Also, flipping about both the x-axis and the y-axis can be determined according to the block shape. For example, when the block is not a square, it can be determined that the flipping is not about both the x-axis and the y-axis. For example, when the mode selected in multi-hypothesis prediction is a horizontal mode, it can be determined that the flipping is about the x-axis. For example, if the mode selected in the multi-hypothesis prediction is a vertical mode, it can be determined that there is flipping on the y-axis, and if the mode selected in the multi-hypothesis prediction is a DC mode, it can be determined that there is no flipping and no explicit signaling is required.

[0262] In addition, the DC mode may have a similar effect to illumination compensation in multi-hypothesis prediction. Therefore, according to an embodiment of the present disclosure, when either the DC mode or the illumination compensation method is used in multi-hypothesis prediction, the other may not be used.

[0263] Also, multi-hypothesis prediction may have a similar effect to generalized bi-prediction (GBi). For example, DC mode may have a similar effect to GBi in multi-hypothesis prediction. GBi may be a method of adjusting a weight between two reference blocks in bi-prediction in block units or CU units. Therefore, according to an embodiment of the present disclosure, when one of multi-hypothesis prediction (or DC mode in multi-hypothesis prediction) and the GBi method is used, the other may not be used. This may also be the case where predictions of multi-hypothesis prediction include those that are bi-predictive. For example, when a selected merge candidate of multi-hypothesis prediction is bi-predictive, GBi may not be used. In these embodiments, the relationship between multi-hypothesis prediction and GBi may be limited to when a specific mode of multi-hypothesis prediction, for example, DC mode, is used. Or, when GBi-related signaling exists before multi-hypothesis prediction-related signaling, when GBi is used, multi-hypothesis prediction or a specific mode of multi-hypothesis prediction may not be used.

[0264] Not using a method can mean not signaling for that method and not parsing the associated syntax.

[0265] FIG. 22 is a diagram illustrating surrounding positions referenced in multi-hypothesis prediction according to an embodiment of the present disclosure.

[0266] As described above, the surrounding positions can be referenced in the process of creating a candidate list for multi-hypothesis prediction. For example, the above-mentioned candIntraPredModeX may be necessary. At this time, the positions A and B around the current block to be referenced may be NbA and NbB shown in FIG. 22. If the top left end position of the current block is Cb as shown in FIG. 18 and its coordinates are (xCb, yCb), NbA may be (xNbA, yNbA)=(xCb-1, yCb+cbHeight-1), and NbB may be (xNbB, yNbB)=(xCb+cbWidth-1, yCb-1). Here, cbWidth and cbHeight may be the width and height of the current block, respectively. In addition, in the process of creating a candidate list for multi-hypothesis prediction, the surrounding positions may be the same as the surrounding positions referenced in generating the MPM list for intra prediction.

[0267] In yet another embodiment, the neighboring positions referred to in the process of creating a candidate list for multi-hypothesis prediction may be near the left center and the top center of the current block. For example, NbA and NbB may be (xCb-1, yCb+cbHeight / 2-1) and (xCb+cbWidth / 2-1, yCb-1). Or, NbA and NbB may be (xCb-1, yCb+cbHeight / 2) and (xCb+cbWidth / 2, yCb-1).

[0268] FIG. 23 is a diagram illustrating a method for referring to a surrounding mode according to one embodiment of the present disclosure.

[0269] As described above, the surrounding positions can be referenced in the process of creating a candidate list for multi-hypothesis prediction, but in the embodiment of Fig. 19, when the surrounding positions do not use multi-hypothesis prediction, candIntraPredModeX is set to the already set mode. This is because when candIntraPredModeX is set, the mode of the surrounding positions may not be directly set to candIntraPredModeX.

[0270] Therefore, according to an embodiment of the present disclosure, even if the surrounding positions do not use multi-hypothesis prediction, if the mode used by the surrounding positions is the mode used for multi-hypothesis prediction, candIntraPredModeX can be set to the mode used by the surrounding positions. The mode used for multi-hypothesis prediction may be a planar mode, a DC mode, a vertical mode, or a horizontal mode.

[0271] Alternatively, even if the surrounding positions do not use multi-hypothesis prediction, if the mode used by the surrounding positions is a specific mode, candIntraPredModeX can be set to the mode used by the surrounding positions.

[0272] Alternatively, even if the surrounding position does not use multi-hypothesis prediction, if the mode used by the surrounding position is a vertical mode or a height mode, candIntraPredModeX may be set to the mode used by the surrounding position. Alternatively, even if the surrounding position is on the upper side of the current block and the mode used by the surrounding position is a vertical mode, candIntraPredModeX may be set to the mode used by the surrounding position. Also, even if the surrounding position is on the left side of the current block and the mode used by the surrounding position is a horizontal mode, candIntraPredModeX may be set to the mode used by the surrounding position.

[0273] Referring to FIG. 23, mh_intra_flag may be signaling indicating whether or not multi-hypothesis prediction is used. In addition, the intra prediction mode used in the neighboring block may be horizontal mode. In addition, the current block may use multi-hypothesis prediction, and a candidate list may be generated using candIntraPredMode based on the mode of the neighboring block. Even if the neighboring block does not use multi-hypothesis prediction, the intra prediction mode of the neighboring block is a specific mode, for example, horizontal mode, so candIntraPredMode may be set to horizontal mode.

[0274] The above-mentioned example of the peripheral mode referencing method is restated below in conjunction with another embodiment of FIG.

[0275] For X being replaced by either A or B, the variables candIntraPredModeX are derived as follows:

[0276] 1. The availability derivation process for a block as specified in Neighboring blocks availability checking process is invoked with the location ( xCurr, yCurr ) set equal to ( xCb, yCb ) and the neighboring location ( xNbY, yNbY ) set equal to ( xNbX, yNbX ) as inputs, and the output is assigned to availableX.

[0277] 2. The candidate intra prediction mode candIntraPredModeX is derived as follows:

[0278] A. If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_DC.

[0279] a. The variable availableX is equal to FALSE.

[0280] b. mh_intra_flag[ xNbX ][ yNbX ] is not equal to 1, and IntraPredModeY[ xNbX ][ yNbX ] is neither INTRA_ANGULAR50 nor INTRA_ANGULAR18.

[0281] c. X is equal to B and yCb - 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

[0282] B. Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] > INTRA_ANGULAR34, candIntraPredModeX is set equal to INTRA_ANGULAR50.

[0283] C. Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] <= INTRA_ANGULAR34 and IntraPredModeY[ xNbX ][ yNbX ] > INTRA_DC, candIntraPredModeX is set equal to INTRA_ANGULAR18.

[0284] D. Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

[0285] Next is another example.

[0286] For X being replaced by either A or B, the variables candIntraPredModeX are derived as follows:

[0287] 1. The availability derivation process for a block as specified in Neighbouring blocks availability checking process is invoked with the location ( xCurr, yCurr ) set equal to ( xCb, yCb ) and the neighbouring location ( xNbY, yNbY ) set equal to ( xNbX, yNbX ) as inputs, and the output is assigned to availableX.

[0288] 2. The candidate intra prediction mode candIntraPredModeX is derived as follows:

[0289] A. If one or more of the following conditions are true, candIntraPredModeX is set equal to INTRA_DC.

[0290] a. The variable availableX is equal to FALSE.

[0291] b. mh_intra_flag[ xNbX ][ yNbX ] is not equal to 1, and IntraPredModeY[ xNbX ][ yNbX ] is neither INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR50 nor INTRA_ANGULAR18.

[0292] c. X is equal to B and yCb - 1 is less than ( ( yCb >> CtbLog2SizeY ) << CtbLog2SizeY ).

[0293] B. Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] > INTRA_ANGULAR34, candIntraPredModeX is set equal to INTRA_ANGULAR50.

[0294] C. Otherwise, if IntraPredModeY[ xNbX ][ yNbX ] <= INTRA_ANGULAR34 and IntraPredModeY[ xNbX ][ yNbX ] > INTRA_DC, candIntraPredModeX is set equal to INTRA_ANGULAR18.

[0295] D. Otherwise, candIntraPredModeX is set equal to IntraPredModeY[ xNbX ][ yNbX ].

[0296] In the above-mentioned list setting method, candIntraPredModeX may be determined by the surrounding mode reference method.

[0297] FIG. 24 is a diagram illustrating peripheral sample usage according to one embodiment of the present disclosure.

[0298] As described above, when multi-hypothesis prediction is used, intra prediction can be used in combination with other predictions. Therefore, when multi-hypothesis prediction is used, intra prediction can be generated using samples around the current block as reference samples.

[0299] According to an embodiment of the present disclosure, when multi-hypothesis prediction is used, a mode using reconstructed samples may be used. Also, when multi-hypothesis prediction is not used, a mode using reconstructed samples may not be used. The reconstructed samples may be reconstructed samples around a current block.

[0300] An example of a mode using the reconstructed sample may be template matching. A reconstructed sample at a position already set based on a certain block may be defined as a template. Template matching may be an operation of searching for a block with a smaller cost by comparing the cost of a template of a block to be compared with that of a current block. In this case, the cost may be defined as the sum of absolute difference values ​​of templates, the sum of squared difference values, etc. For example, a block expected to be similar to the current block may be searched for using template matching between the current block and a block of a reference picture, and a motion vector may be set or a motion vector may be refined based on the search.

[0301] Examples of modes using the reconstructed samples may include motion compensation using the reconstructed samples, motion vector refinement, and the like.

[0302] In order to use reconstructed samples around the current block, it is necessary to wait for the completion of decoding of the surrounding blocks when decoding the current block. In this case, it may be difficult to process the current block and the surrounding blocks in parallel. Therefore, when multi-hypothesis prediction is not used, it is not necessary to use a mode that uses reconstructed samples around the current block to enable parallel processing. In addition, when multi-hypothesis prediction is used, since intra prediction can be generated using reconstructed samples around the current block, other modes that use reconstructed samples around the current block can also be used.

[0303] Also, according to an embodiment of the present disclosure, even if multi-hypothesis prediction is used, reconstructed samples around the current block may or may not be used depending on the candidate index. As an example, when the candidate index is smaller than a threshold, reconstructed samples around the current block may be used. If the candidate index is small, the number of candidate index signaling bits is small and the candidate accuracy may be high, but it is possible to further increase the accuracy by using reconstructed samples for candidates with high coding efficiency. As another example, when the candidate index is larger than a threshold, reconstructed samples around the current block may be used. If the candidate index is large, the number of candidate index signaling bits is large and the candidate accuracy may be low, but the accuracy can be supplemented by using reconstructed samples around the current block for candidates with low accuracy.

[0304] According to one embodiment of the present disclosure, when using multi-hypothesis prediction, it is possible to generate an inter-prediction using reconstructed samples around the current block, and combine the inter-prediction with the intra-prediction of the multi-hypothesis prediction to generate a predicted block.

[0305] 24, the value of mh_intra_flag, which is signaling indicating whether the current block uses multi-hypothesis prediction, is 1. Since the current block uses multi-hypothesis prediction, a mode that uses reconstructed samples around the current block can be used.

[0306] FIG. 25 is a diagram illustrating a transform mode according to one embodiment of the present disclosure.

[0307] According to an embodiment of the present disclosure, there may be a transform mode that transforms only a sub-part of a block. Such a transform mode may be called a sub-block transform (SBT) or a spatially varying transform (SVT). For example, a CU or a PU may be divided into a number of TUs, and only a part of the number of TUs may be transformed. For example, only one of the number of TUs may be transformed. The TUs that are not transformed among the number of TUs may have a residual of 0.

[0308] Referring to FIG. 25, there are two types of types for dividing one CU or PU into multiple TUs: SVT-V and SVT-H. SVT-V may be a type in which the height of multiple TUs is the same as the height of the CU or PU, and the width of multiple TUs is different from the width of the CU or PU. SVT-H may be a type in which the height of multiple TUs is different from the height of the CU or PU, and the width of multiple TUs is the same as the width of the CU or PU. As an example, the width and position of the TUs converted by the SVT-V may be signaled. Also, the height and position of the TUs converted by the SVT-H may be signaled.

[0309] According to one embodiment, the transformation kernel according to the SVT type and the position, width or height may already be set.

[0310] In this way, there is a mode in which only a portion of a CU or PU is transformed because the residual may mainly exist in a portion of the CU or PU after prediction.

[0311] That is, the SBT has the same concept as the skip mode for a TU. The existing skip mode may be the skip mode for a CU.

[0312] 25, for each type of SVT-V and SVT-H, the conversion positions indicated by A are defined as two locations, and the width or height is defined as 1 / 2 or 1 / 4 of the width or height of the CU. In addition, the residuals in the areas other than the area indicated by A can be set to 0.

[0313] There may also be conditions under which SBT can be used, including, for example, conditions related to block size, signaling values ​​indicating whether or not it is used in high-level (e.g., sequence, slice, tile, etc.) syntax, etc.

[0314] FIG. 26 is a diagram illustrating a relationship between multi-hypothesis prediction and conversion modes according to one embodiment of the present disclosure.

[0315] According to an embodiment of the present disclosure, there may be a correlation between the multi-hypothesis prediction and the conversion mode. For example, whether one of the modes is used may determine whether the other mode is used. Alternatively, whether one of the modes is used may determine whether the other mode is used. Alternatively, whether one of the modes is used may determine whether the other mode is used.

[0316] As an example, the transform mode may be the SVT described in FIG. 25. That is, whether or not the SVT is used may be determined depending on whether or not the multi-hypothesis prediction is used. Alternatively, whether or not the multi-hypothesis prediction is used may be determined depending on whether or not the SVT is used. This is because the multi-hypothesis prediction improves prediction performance for the entire block, and the phenomenon of residuals gathering only in a part of the block is less likely to occur.

[0317] According to an embodiment of the present disclosure, the position of the TU to be converted by the SVT may be restricted depending on whether or not the multi-hypothesis prediction is used or the mode of the multi-hypothesis prediction. Alternatively, the width (SVT-V) or height (SVT-H) of the TU to be converted by the SVT may be restricted depending on whether or not the multi-hypothesis prediction is used or the mode of the multi-hypothesis prediction. Therefore, it is possible to reduce signaling related to the position, width, or height. For example, the position of the TU to be converted by the SVT may not be on the side where the weight value of the intra prediction may be large in the multi-hypothesis prediction. This is because the residual on the side with the large weight value can be reduced by the multi-hypothesis prediction. Therefore, when using the multi-hypothesis prediction, there may be no mode to convert the side with the large weight value in the SVT. For example, when the horizontal mode or the vertical mode is used in the multi-hypothesis prediction, it is possible to omit position 1 in FIG. 25. As another embodiment, when the planar mode is used in the multi-hypothesis prediction, the position of the TU to be converted by the SVT may be restricted. For example, when the planar mode is used in the multi-hypothesis prediction, it is possible to omit position 0 in FIG. 25. This is because when using the planar mode in multi-hypothesis prediction, the nearby part of the reference sample for intra prediction may have a value similar to the reference sample value, which may result in a small residual near the reference sample.

[0318] As another example, when multi-hypothesis prediction is used, the possible width or height value of the TU to be SVT converted may change. Or, when a specific mode is used in multi-hypothesis prediction, the possible width or height value of the TU to be SVT converted may change. For example, when multi-hypothesis prediction is used, since not many residuals remain in a wide part of a block, a large value of the width or height of the TU to be SVT converted may be excluded. Or, when multi-hypothesis prediction is used, the width or height value of the TU to be SVT converted that is the same as the unit in which the weight value changes in multi-hypothesis prediction may be excluded.

[0319] 26, there may be cu_sbt_flag indicating whether SBT is used or not, and mh_intra_flag indicating whether multi-hypothesis prediction is used or not. Referring to the drawing, cu_sbt_flag can be parsed when mh_intra_flag is 0. Also, when cu_sbt_flag does not exist, it can be inferred to be 0.

[0320] Both combining intra prediction in multi-hypothesis prediction and SBT are intended to solve the problem that a large amount of residuals may remain in only a part of a CU or PU when the corresponding technology is not used. Therefore, since both technologies may be related to each other, it is possible to determine whether one technology is used or whether a specific mode of one technology is used based on the other technology.

[0321] Also, in FIG. 26, sbtblockConditions can indicate conditions under which SBT is possible. The conditions under which SBT is possible can include conditions related to block size, signaling values ​​whether or not to be used in high-level (e.g., sequence, slice, tile, etc.) syntax, etc.

[0322] FIG. 27 is a diagram showing the relationship between color components according to one embodiment of the present disclosure.

[0323] Referring to FIG. 27, the color format may be indicated by chroma format information (chroma_format_idc), Chroma format, separate_colour_plane_flag, etc.

[0324] If it is monochrome, there can be only one sample array, and SubWidthC and SubHeightC can both be 1.

[0325] If the sampling is 4:2:0, there may be two chroma arrays. The chroma array may be half the width and half the height of the luma array. The information about the width (SubWidthC) and the information about the height (SubHeightC) may both be 2.

[0326] The width information (SubWidthC) and height information (SubHeightC) may indicate the size of the chroma array compared to the luma array, where if the width or height of the chroma array is half the size of the luma array, the width information (SubWidthC) or height information (SubHeightC) may be 2, and if the width or height of the chroma array is the same size as the luma array, the width information (SubWidthC) or height information (SubHeightC) may be 1.

[0327] If the sampling is 4:2:2, there may be two chroma arrays. The chroma arrays may be half the width and the same height as the luma array. SubWidthC and SubHeightC may be 2 and 1, respectively.

[0328] If sampling is 4:4:4, the chroma array may have the same width and height as the luma array. SubWidthC and SubHeightC may both be 1. In this case, processing may differ based on separate_colour_plane_flag. If separate_colour_plane_flag is 0, the chroma array may have the same width and height as the luma array. If separate_colour_plane_flag is 1, the three color planes (luma, Cb, Cr) can be processed separately. If sampling is 4:4:4 regardless of separate_colour_plane_flag, SubWidthC and SubHeightC may both be 1.

[0329] If separate_colour_plane_flag is 1, then only one colour component may exist in one slice. If separate_colour_plane_flag is 0, then multiple colour components may exist in one slice.

[0330] 27, SubWidthC and SubHeightC can be different only when the size is 4:2:2. Therefore, when the size is 4:2:2, the luma reference width to height relationship and the chroma reference width to height relationship may be different.

[0331] For example, when the luma sample reference width is width L and the chroma sample reference width is width C, if width L and width C correspond to each other, the relationship between the two may be as follows.

[0332] widthC = widthL / SubWidthC

[0333] That is, widthL = widthC * SubWidthC

[0334] Similarly, when the luma sample reference height is heightL and the chroma sample reference height is heightC, if heightL and heightC correspond to each other, the relationship between them may be as follows.

[0335] heightC = heightL / SubHeightC

[0336] That is, heightL = heightC * SubHeightC

[0337] There may also be a value indicating a color component. For example, cIdx may indicate a color component. For example, cIdx may be a color component index. If cIdx is 0, it may indicate a luma component. If cIdx is not 0, it may indicate a chroma component. If cIdx is 1, it may indicate a chroma Cb component. If cIdx is 2, it may indicate a chroma Cr component.

[0338] FIG. 28 is a diagram illustrating the relationship between color components according to one embodiment of the present disclosure.

[0339] Figures 28(a), (b), and (c) show the cases of 4:2:0, 4:2:2, and 4:4:4, respectively.

[0340] Referring to Figure 28(a), one chroma sample (one Cb, one Cr) may be located for every two luma samples in the horizontal direction, and one chroma sample (one Cb, one Cr) may be located for every two luma samples in the vertical direction.

[0341] Referring to Figure 28(b), one chroma sample (one Cb, one Cr) may be located for every two luma samples in the horizontal direction, and one chroma sample (one Cb, one Cr) may be located for every luma sample in the vertical direction.

[0342] 28(c), one chroma sample (one Cb, one Cr) may be located per luma sample in the horizontal direction, and one chroma sample (one Cb, one Cr) may be located per luma sample in the vertical direction.

[0343] As described above, SubWidthC and SubHeightC described in FIG. 27 may be determined based on such a relationship, and conversion between the luma sample basis and the chroma sample basis may be performed based on SubWidthC and SubHeightC.

[0344] FIG. 29 is a diagram illustrating peripheral reference positions according to one embodiment of the present disclosure.

[0345] According to an embodiment of the present disclosure, the surrounding location may be referenced when performing prediction. For example, as described above, the surrounding location may be referenced when performing CIIP. The CIIP may be the multi-hypothesis prediction described above. The CIIP may be a combined inter-picture merge and intra-picture prediction. That is, the CIIP may be a prediction method that combines inter-prediction (e.g., merge mode inter-prediction) and intra-prediction.

[0346] According to an embodiment of the present disclosure, it is possible to combine inter prediction and intra prediction with reference to the surrounding positions. For example, it is possible to determine the ratio of inter prediction to intra prediction with reference to the surrounding positions. Alternatively, it is possible to determine a weight value when combining inter prediction and intra prediction with reference to the surrounding positions. Alternatively, it is possible to determine a weight value when performing a weighted sum (weighted average) of inter prediction and intra prediction with reference to the surrounding positions.

[0347] According to an embodiment of the present disclosure, the referenced peripheral positions may include NbA and NbB, whose coordinates may be (xNbA, yNbA) and (xNbB, yNbB), respectively.

[0348] Also, NbA may be the left position of the current block. More specifically, if the top left coordinate of the current block is (xCb, yCb) and the width and height of the current block are cbWidth and cbHeight, respectively, NbA may be (xCb-1, yCb+cbHeight-1). The top left coordinate (xCb, yCb) of the current block may be a value based on the luma sample. Alternatively, the top left coordinate (xCb, yCb) of the current block may be the luma position of the top left sample of the current luma coding block with respect to the top left luma sample of the current picture. Also, the cbWidth and cbHeight may be values ​​based on the corresponding color component. The above coordinates may be for the luma component (luma block). For example, cbWidth and cbHeight may be values ​​based on the luma component.

[0349] Also, NbB may be the above position of the current block. More specifically, if the top left coordinate of the current block is (xCb, yCb) and the width and height of the current block are cbWidth and cbHeight, respectively, NbB may be (xCb+cbWidth-1, yCb-1). The top left coordinate (xCb, yCb) of the current block may be a value based on the luma sample. Alternatively, the top left coordinate (xCb, yCb) of the current block may be a luma position of the top left sample of the current luma coding block with respect to the top left luma sample of the current picture. Also, the cbWidth and cbHeight may be values ​​based on the corresponding color component. The above coordinates may be for the luma component (luma block). For example, cbWidth and cbHeight may be values ​​based on the luma component.

[0350] Referring to FIG. 29, for a block labeled "luma block" at the top, the top left corner, coordinates of NbA, coordinates of NbB, etc. are indicated.

[0351] Also, NbA may be the left position of the current block. More specifically, if the top left coordinate of the current block is (xCb, yCb) and the width and height of the current block are cbWidth and cbHeight, respectively, NbA may be (xCb-1, yCb+2*cbHeight-1). The top left coordinate (xCb, yCb) of the current block may be a value based on the luma sample. Or, the top left coordinate (xCb, yCb) of the current block may be the top left sample luma position of the current luma coding block with respect to the top left luma sample of the current picture. Also, the cbWidth and cbHeight may be values ​​based on the corresponding color component. The above coordinates may be for a chroma component (chroma block). For example, cbWidth and cbHeight may be values ​​based on the chroma component. Also, these coordinates may correspond to a 4:2:0 format.

[0352] Also, NbB may be the above position of the current block. More specifically, if the top left coordinate of the current block is (xCb, yCb) and the width and height of the current block are cbWidth and cbHeight, respectively, NbB may be (xCb+2*cbWidth-1, yCb-1). The top left coordinate (xCb, yCb) of the current block may be a value based on the luma sample. Alternatively, the top left coordinate (xCb, yCb) of the current block may be a luma position of the top left sample of the current luma coding block with respect to the top left luma sample of the current picture. Also, the cbWidth and cbHeight may be values ​​based on the corresponding color component. The above coordinates may be for a chroma component (chroma block). For example, cbWidth and cbHeight may be values ​​based on the chroma component. The coordinates may also be in 4:2:0 or 4:2:2 format.

[0353] Referring to FIG. 29, the upper left corner, coordinates of NbA, coordinates of NbB, etc. are indicated for a block labeled chroma block at the bottom.

[0354] FIG. 30 is a diagram illustrating a weighted sample prediction process according to one embodiment of the present disclosure.

[0355] The embodiment of FIG 30 may relate to a method of combining two or more prediction signals. The embodiment of FIG 30 may also be applicable when using CIIP. The embodiment of FIG 30 may also include the peripheral location referencing method described in FIG 29.

[0356] Referring to equation (8-838) in FIG. 30, the scale information (scallFact) can be explained as follows:

[0357] scallFact = (cIdx == 0) ? 0 : 1

[0358] That is, if the information (cIdx) on the color components of the current block is 0, the scale information (scallFact) can be set to 0, and if the information (cIdx) on the color components of the current block is not 0, the scale information (scallFact) can be set to 1. In an embodiment of the present disclosure, x?y:z can indicate a y value if x is true or if x is not 0, and can indicate a z value otherwise (if x is false (or x is 0)).

[0359] Also, the coordinates (xNbA, yNbA) and (xNbB, yNbB) of the surrounding positions NbA and NbB to be referred to can be set. According to the embodiment described in FIG. 29, for the luma component, (xNbA, yNbA) and (xNbB, yNbB) are (xCb - 1, yCb + cbHeight - 1) and (xCb + cbWidth - 1, yCb - 1) respectively, and for the chroma component, (xNbA, yNbA) and (xNbB, yNbB) may be (xCb - 1, yCb + 2 * cbHeight - 1) and (xCb + 2 * cbWidth - 1, yCb - 1) respectively. Also, the operation of multiplying by 2^n may be the same as left-shifting n bits. For example, the operation of multiplying by 2 may be the same as left-shifting 1 bit. Also, left-shifting x by n bits can be expressed as "x << n". Also, the operation of dividing by 2^n may be the same as right-shifting n bits. Also, the operation of dividing by 2^n and discarding the decimal part may be the same as right-shifting n bits. For example, the operation of dividing by 2 may be the same as right-shifting 1 bit. Also, right-shifting x by n bits can be expressed as "x >> n". Therefore, (xCb - 1, yCb + 2 * cbHeight - 1) and (xCb + 2 * cbWidth - 1, yCb - 1) can be expressed as (xCb - 1, yCb + (cbHeight << 1) - 1) and (xCb + (cbWidth << 1) - 1, yCb - 1). Therefore, when representing both the coordinates for the above-described luma component and the coordinates for the chroma component, it may be as follows.

[0360] (xNbA, yNbA) = (xCb - 1, yCb + (cbHeight << scallFact) - 1)

[0361] (xNbB, yNbB) = (xCb + (cbWidth<<scallFact) - 1, yCb - 1)

[0362] Here, the scale information (scalFact) may be (cIdx==0)?0:1 as described above. In this case, cbWidth and cbHeight may be indicated based on each color component. For example, when the width and height based on the luma component are cbWidthL and cbHeightL, respectively, cbWidth and cbHeight may be cbWidthL and cbHeightL, respectively, when performing a weighted sample prediction process for the luma component. In addition, when the width and height based on the luma component are cbWidthL and cbHeightL, respectively, cbWidth and cbHeight may be cbWidthL / SubWidthC and cbHeightL / SubHeightC, respectively, when performing a weighted sample prediction process for the chroma component.

[0363] Also, according to an embodiment, a prediction mode of a corresponding position may be determined by referring to a surrounding position. For example, it may be determined whether the prediction mode is intra prediction. Also, the prediction mode may be indicated by CuPredMode. When CuPredMode is MODE_INTRA, intra prediction may be used. Also, the CuPredMode value may be MODE_INTRA, MODE_INTER, MODE_IBC, or MODE_PLT. When CuPredMode is MODE_INTER, intra prediction may be used. Also, when CuPredMode is MODE_IBC, intra copy (intra block copy, IBC) may be used. Also, when CuPredMode is MODE_PLT, palette mode may be used. Also, CuPredMode may be indicated by a channel type (chType) and a position. For example, it may be denoted as CuPredMode[chType][x][y], where the value may be the CuPredMode value for the channel type chType at the (x, y) position. Also, chType may be based on a tree type. For example, the tree type (treeType) may be set to values ​​such as SINGLE_TREE, DUAL_TREE_LUMA, DUAL_TREE_CHROMA, etc. In the case of SINGLE_TREE, there may be a shared portion of block partitioning of the luma component and the chroma component. For example, in the case of SINGLE_TREE, the block partitioning of the luma component and the chroma component may be the same. Or, in the case of SINGLE_TREE, the block partitioning of the luma component and the chroma component may be the same or partially the same.Alternatively, in the case of SINGLE_TREE, block division into luma components and chroma components may be performed by the same syntax element value. In the case of DUAL TREE, block division into luma components and chroma components may be independent. In the case of DUAL TREE, block division into luma components and chroma components may be performed by individual syntax element values. In the case of DUAL TREE, the treeType value may be DUAL_TREE_LUMA or DUAL_TREE_CHROMA. If treeType is DUAL_TREE_LUMA, DUAL TREE may be used to indicate that the processing is for the luma component. If treeType is DUAL_TREE_CHROMA, DUAL TREE may be used to indicate that the processing is for the chroma component. In addition, chType may be determined based on whether the tree type is DUAL_TREE_CHROMA. For example, chType may be set to 1 if treeType is DUAL_TREE_CHROMA, and may be set to 0 if treeType is not DUAL_TREE_CHROMA. Thus, referring to FIG. 30, a CuPredMode[0][xNbX][yNbY] value may be determined. X may be substituted for A and B. That is, a CuPredMode value for NbA and NbB positions may be determined.

[0364] Also, the isIntraCodedNeighbourX value may be set based on the prediction mode for the surrounding position. For example, the isIntraCodedNeighbourX value may be set depending on whether the CuPredMode for the surrounding position is MODE_INTRA. If the CuPredMode for the surrounding position is MODE_INTRA, the isIntraCodedNeighbourX value may be set to TRUE, and if the CuPredMode for the surrounding position is not MODE_INTRA, the isIntraCodedNeighbourX value may be set to FALSE. In the present disclosure described above and below, X may be replaced with A or B, etc. Also, anything written as X may indicate that it corresponds to the X position.

[0365] Also, according to an embodiment, it may be determined whether the corresponding position is available by referring to the neighboring positions. Depending on whether the corresponding position is available, it may be set to availableX. Also, isIntraCodedNeighbourX may be set based on availableX. For example, if availableX is TRUE, isIntraCodedNeighbourX may be set to TRUE, and for example, if availableX is FALSE, isIntraCodedNeighbourX may be set to FALSE. Referring to FIG. 30, whether the corresponding position is available may be determined by "The derivation process for neighbouring block availability". Also, whether the corresponding position is available may be determined based on whether the corresponding position is inside the current picture. When the corresponding position is (xNbY, yNbY), if xNbY or yNbY is less than 0, it is out of the current picture, and availableX may be set to FALSE. Also, if xNbY is equal to or greater than the picture width, it is out of the current picture, and availableX may be set to FALSE. The picture width may be indicated by pic_width_in_luma_samples. If yNbY is equal to or greater than the picture height, the picture is out of the current picture and availableX may be set to FALSE. The picture height may be indicated by pic_height_in_luma_samples. If the location is in a different brick or slice than the current block, availableX may be set to FALSE. If reconstruction of the location is not complete, availableX may be set to FALSE. Whether reconstruction is complete or not may be indicated by IsAvailable[cIdx][xNbY][yNbY].In short, if any one of the following conditions is met, availableX can be set to FALSE, and if not (if none of the following conditions are met), availableX can be set to TRUE.

[0366] Condition 1: xNbY<0

[0367] Condition 2: yNbY<0

[0368] Condition 3: xNbY>=pic_width_in_luma_samples

[0369] Condition 4:yNbY>=pic_height_in_luma_samples

[0370] Condition 5: IsAvailable[cIdx][xNbY][yNbY]==FALSE

[0371] Condition 6: If the corresponding position (peripheral position (xNbY, yNbY) position) belongs to a brick (or a slice) different from the current block

[0372] In addition, it is possible to determine whether the current position and the corresponding position are in the same CuPredMode by using an option and set availableX.

[0373] The two conditions described above can be combined to set isIntraCodedNeighbourX. For example, if all of the following conditions are met, isIntraCodedNeighbourX can be set to TRUE, and if not (if any one of the following conditions is not met), isIntraCodedNeighbourX can be set to FALSE.

[0374] Condition 1: availableX==TRUE

[0375] Condition 2: CuPredMode[0][xNbX][yNbX]==MODE_INTRA

[0376] Also, according to an embodiment of the present disclosure, the weighting of the CIIP may be determined based on a number of pieces of code information (isIntraCodedNeighbourX). For example, when inter prediction and intra prediction are combined based on a number of pieces of code information (isIntraCodedNeighbourX), the weighting may be determined based on the code information (isIntraCodedNeighbourA) related to the left block and the code information (isIntraCodedNeighbourB) related to the upper block. According to an embodiment, if the code information (isIntraCodedNeighbourA) related to the left block and the code information (isIntraCodedNeighbourB) related to the upper block are both TRUE, the weighting value (w) may be set to 3. For example, the weighting value (w) may be the weighting of the CIIP or a value that determines the weighting. Also, if the code information (isIntraCodedNeighbourA) related to the left block and the code information (isIntraCodedNeighbourB) related to the upper block are both FALSE, w may be set to 1. Also, if any one of the code information (isIntraCodedNeighbourA) related to the left block and the code information (isIntraCodedNeighbourB) related to the upper block is FALSE (similar to the case where one of the two is TRUE), the weight (w) may be set to 2. That is, the weight (w) may be set based on whether the surrounding position is predicted by intra prediction or based on how much the surrounding position is predicted by intra prediction.

[0377] Also, the weighting value (w) may be a weighting value corresponding to intra prediction. Also, a weighting value corresponding to inter prediction may be determined based on the weighting value (w). For example, a weighting value corresponding to inter prediction may be (4-w). Referring to equation (8-840) of FIG. 30, when combining two or more prediction signals, the following may be obtained.

[0378] predSampleComb[x][y] = (w*predSamplesIntra[x][y] + (4-w)*predSamplesInter[x][y] + 2 ) >> 2

[0379] Here, the second sample (predSamplesIntra) and the first sample (predSamplesInter) may be prediction signals. For example, the second sample (predSamplesIntra) and the first sample (predSamplesInter) may be prediction signals predicted by intra prediction and prediction signals predicted by inter prediction (e.g., merge mode, more specifically, regular merge mode), respectively. In addition, the combined prediction sample (predSampleComb) may be a prediction signal used in CIIP.

[0380] Also, a process of updating a prediction signal before combination may be included before performing equation (8-840) of Figure 30. For example, the process may be updated by a process such as equation (8-839) of Figure 30. For example, the process of updating a prediction signal may be a process of updating an inter-prediction signal of CIIP.

[0381] FIG. 31 is a diagram illustrating peripheral reference positions according to one embodiment of the present disclosure.

[0382] Although the peripheral reference positions have been described with reference to FIGS. 29 and 30, a problem may occur if the described positions are used in all cases (for example, all chroma blocks), and this problem will be described with reference to FIG.

[0383] The embodiment of FIG. 31 shows a chroma block. In FIG. 29 to FIG. 30, the NbA and NbB coordinates based on the luma sample for the chroma block are (xCb-1, yCb+2*cbHeight-1) and (xCb+2*cbWidth-1, yCb-1), respectively. However, when SubWidthC or SubHeightC is 1, it may show a position in FIG. 31 different from the position shown in FIG. 29. In the above coordinates, cbWidth and cbHeight are multiplied by 2, which may be for compensating the number of luma samples versus chroma samples in the case of 4:2:0, since cbWidth and cbHeight are shown based on each color component (chroma component in this embodiment) and the coordinates are shown based on the luma. That is, it may be for showing the coordinates based on the luma when there is one chroma sample corresponding to two luma samples based on the x-axis and one chroma sample corresponding to two luma samples based on the y-axis. Therefore, when SubWidthC or SubHeightC is 1, a different position may be indicated. Therefore, always using the (xCb-1, yCb+2*cbHeight-1) or (xCb+2*cbWidth-1, yCb-1) position for the chroma block in this manner may refer to a position far away from the current chroma block. In this case, the relative position used for the luma block of the current block may not match the relative position used for the chroma block. Also, since a different position is referenced for the chroma block, weighting may be set by referring to a position that is less related to the current block, or decoding / reconstruction may not be performed due to the block decoding order.

[0384] 31 shows the positions of the above-mentioned luma reference coordinates in the case of 4:4:4, that is, when SubWidthC and SubHeightC are both 1. NbA and NbB may be located away from the chroma blocks shown by solid lines.

[0385] FIG. 32 is a diagram illustrating a weighted sample prediction process according to an embodiment of the present disclosure.

[0386] The embodiment of Fig. 32 may be an embodiment for solving the problems described with reference to Fig. 29 to Fig. 31. Moreover, the above-mentioned contents may be omitted.

[0387] In FIG. 30, the peripheral positions are set based on the scale information (scallFact), and as described in FIG. 31, the scale information (scallFact) is a value for converting the position when SubWidthC and SubHeightC are 2.

[0388] However, as mentioned above, problems may arise depending on the color format, and the ratio of chroma samples to luma samples may differ horizontally and vertically, so according to an embodiment of the present disclosure, the scale information (scalFact) can be separated into horizontal (width) and vertical (height).

[0389] According to an embodiment of the present disclosure, x-axis scale information (scallFactWidth) and y-axis scale information (scallFactHeight) may exist, and a peripheral position may be set based on the x-axis scale information (scallFactWidth) and the y-axis scale information (scallFactHeight). In addition, the peripheral position may be set based on a luma sample (luma block).

[0390] The video signal processing device may perform a step of acquiring information on width (SubWidthC) and information on height (SubHeightC) based on chroma format information (chroma_format_idc), where the chroma format information (chroma_format_idc) may be signaled in any one of a coding tree unit, a slice, a tile, a tile group, a picture, or a sequence unit.

[0391] The video signal processing device can obtain information regarding the width (SubWidthC) and information regarding the height (SubHeightC) based on the chroma format information (chroma_format_idc) on the basis of the table shown in FIG.

[0392] The video signal processing device may perform a step (8-838) of acquiring scale information (scallFactWidth) of the x-axis based on the information (SubWidthC) on the width or the information (cIdx) on the color component of the current block. More specifically, the scale information (scallFactWidth) of the x-axis may be set based on the information (cIdx) on the color component of the current block and the information (SubWidthC) on the width. For example, if the information (cIdx) on the color component of the current block is 0 or the information (SubWidthC) on the width is 1, the scale information (scallFactWidth) of the x-axis may be set to 0, and otherwise (if the information (cIdx) on the color component of the current block is not 0 and the information (SubWidthC) on the width is not 1 (SubWidthC is 2)), the scale information (scallFactWidth) of the x-axis may be set to 1. Referring to the equation (8-838) of FIG. 32, it can be shown as follows.

[0393] scallFactWidth = ( cIdx == 0 || SubWidthC == 1) ? 0 : 1

[0394] Also, the video signal processing device may perform a step (8-839) of obtaining y-axis scale information (scalFactHeight) based on the height information (SubHeightC) or the color component information (cIdx) of the current block.

[0395] Also, the y-axis scale information (scallFactHeight) can be set based on the color component information (cIdx) and height information (SubHeightC) of the current block. For example, if the color component information (cIdx) of the current block is 0 or the height information (SubHeightC) is 1, the y-axis scale information (scallFactHeight) can be set to 0, and otherwise (if the color component information (cIdx) of the current block is not 0 and the height information (SubHeightC) is not 1 (SubHeightC is 2)), the y-axis scale information (scallFactHeight) can be set to 1. Referring to equation (8-839) in FIG. 32, it can be shown as follows.

[0396] scallFactHeight = ( cIdx == 0 || SubHeightC == 1) ? 0 : 1

[0397] Also, the x-coordinate of the peripheral position can be indicated based on the x-axis scale information (scallFactWidth), and the y-coordinate of the peripheral position can be indicated based on the y-axis scale information (scallFactHeight). The video signal processing device can perform a step of determining a position of the left block (NbA) based on the y-axis scale information (scallFactHeight). Also, the video signal processing device can perform a step of determining a position of the upper block (NbB) based on the x-axis scale information (scallFactWidth). For example, the coordinate of the upper block (NbB) can be set based on the x-axis scale information (scallFactWidth). For example, the coordinate of the left block (NbA) can be set based on the y-axis scale information (scallFactHeight). Also, as described above, here, being based on the x-axis scale information (scallFactWidth) may be based on the information on the width (SubWidthC), and being based on the y-axis scale information (scallFactHeight) may be based on the information on the height (SubHeightC). For example, the perimeter position coordinates may be:

[0398] (xNbA, yNbA) = (xCb - 1, yCb + (cbHeight << scallFactHeight) - 1)

[0399] (xNbB, yNbB) = (xCb + (cbWidth< <scallFactWidth) - 1, yCb - 1)

[0400] (xCb, yCb) may be the luma position of the top left sample of the current luma block (luma coding block) with respect to the top left luma sample of the current picture. cbWidth and cbHeight may be the width and height of the current block, respectively. In this case, xCb and yCb may be coordinates expressed based on the luma sample as described above. Also, cbWidth and cbHeight may be expressed based on each color component.

[0401] Therefore, if it is a chroma block and SubWidthC is 1, then (xNbB, yNbB)=(xCb+cbWidth-1, yCb-1). That is, in this case, the NbB coordinate for the luma block and the NbB coordinate for the chroma block may be the same. Also, if it is a chroma block and SubHeightC is 1, then (xNbA, yNbA)=(xCb-1, yCb+cbHeight-1). That is, in this case, the NbA coordinate for the luma block and the NbA coordinate for the chroma block may be the same.

[0402] Therefore, in the embodiment of Figure 32, when the format is 4:2:0, the same peripheral coordinates are set as in the embodiments of Figures 29 to 30, and when the format is 4:2:2 or 4:4:4, peripheral coordinates different from those in the embodiments of Figures 29 to 30 can be set.

[0403] The video signal processing device may perform a step of determining a weight (w) based on a left block (NbA) and an upper block (NbB). Figure 32 may be similar to that described in Figure 30. That is, a prediction mode or availability may be determined based on the peripheral position coordinates described in Figure 32, and a weight of the CIIP may be determined. Some of the descriptions of Figure 32 that overlap with those of Figure 30 may be omitted.

[0404] 32, the video signal processing device can set the code information (isIntraCodedNeighbourX) by combining two conditions. For example, if all of the following conditions are met, the code information (isIntraCodedNeighbourX) can be set to TRUE, and if not (if at least one of the following conditions is not met), the code information (isIntraCodedNeighbourX) can be set to FALSE.

[0405] Condition 1: availableX==TRUE

[0406] Condition 2: CuPredMode[0][xNbX][yNbX]==MODE_INTRA

[0407] More specifically, referring to line 3210, the video signal processing device may perform a step of setting code information (isIntraCodedNeighbourA) regarding the left block to TRUE if the left block is available (availableA==TRUE) and the prediction mode of the left block is intra prediction (CuPredMode[0][xNbA][yNbA] is equal to MODE_INTRA).

[0408] Also, the video signal processing apparatus may perform the step of setting the code information regarding the left block to FALSE if the left block is not available or the prediction mode of the left block is not intra prediction.

[0409] The video signal processing device may also perform a step of setting code information (isIntraCodedNeighbourB) regarding the upper block to TRUE if the upper block is available (availableB==TRUE) and the prediction mode of the upper block is intra prediction (CuPredMode[0][xNbB][yNbB] is equal to MODE_INTRA).

[0410] Also, the video signal processing apparatus may perform the step of setting the code information regarding the upper block to FALSE if the upper block is not available or the prediction mode of the upper block is not intra prediction.

[0411] Referring to line 3220, if the code information (isIntraCodedNeighbourA) regarding the left block and the code information (isIntraCodedNeighbourB) regarding the upper block are both TRUE, the video signal processing apparatus may perform a step of determining the weighting value (w) as 3. Also, if the code information (isIntraCodedNeighbourA) regarding the left block and the code information (isIntraCodedNeighbourB) regarding the upper block are both FALSE, the video signal processing apparatus may perform a step of determining the weighting value (w) as 1. Also, if only one of the code information (isIntraCodedNeighbourA) regarding the left block and the code information (isIntraCodedNeighbourB) regarding the upper block is TRUE, the video signal processing apparatus may perform a step of determining the weighting value (w) as 2.

[0412] The video signal processing device may perform the step of obtaining first samples (predSamplesInter) predicted as a merge mode for the current block, and the step of obtaining second samples (predSamplesIntra) predicted as an intra mode for the current block.

[0413] The video signal processing device may perform a step (8-841) of obtaining a combined prediction sample (predSampleComb) for a current block based on a weight value (w), a first sample (predSamplesInter), and a second sample (predSamplesIntra). For example, the video signal processing device may obtain the combined prediction sample (predSampleComb) based on the following equation:

[0414] predSampleComb[x][y] = (w*predSamplesIntra[x][y] + (4-w)*predSamplesInter[x][y] + 2 ) >> 2

[0415] Here, predSamplesComb means a combined prediction sample, w means a weighting value, predSamplesIntra means a second sample, predSamplesInter means a first sample, [x] means the x-axis coordinate of a sample included in the current block, and [y] means the y-axis coordinate of a sample included in the current block.

[0416] In this disclosure, peripheral location and peripheral location coordinates may be used interchangeably.

[0417] FIG. 33 is a diagram illustrating a weighted sample prediction process according to an embodiment of the present disclosure.

[0418] The embodiment of Fig. 33 expresses the peripheral position coordinates explained in Fig. 32 in a different manner, so the contents that overlap with those explained above may be omitted.

[0419] As described above, a bit shift can be expressed by a multiplication. Fig. 32 may be illustrated using a bit shift, and Fig. 33 may be illustrated using a multiplication.

[0420] According to an embodiment, the scale information (scallFactWidth) of the x-axis may be set based on the information (cIdx) related to the color components of the current block and the information (SubWidthC) related to the width. For example, if the information (cIdx) related to the color components of the current block is 0 or the information (SubWidthC) related to the width is 1, the scale information (scallFactWidth) of the x-axis may be set to 1, otherwise (if cIdx is not 0 and the information (SubWidthC) related to the width is not 1 (if SubWidthC is 2)), the scale information (scallFactWidth) of the x-axis may be set to 2. Referring to equation (8-838) of FIG. 33, it can be shown as follows.

[0421] scallFactWidth = ( cIdx == 0 || SubWidthC == 1) ? 1 : 2

[0422] Also, the y-axis scale information (scallFactHeight) can be set based on the color component information (cIdx) and height information (SubHeightC) of the current block. For example, if the color component information (cIdx) of the current block is 0 or the height information (SubHeightC) is 1, the y-axis scale information (scallFactHeight) can be set to 1, and otherwise (if cIdx is not 0 and the height information (SubHeightC) is not 1 (SubHeightC is 2)), the y-axis scale information (scallFactHeight) can be set to 2. Referring to equation (8-839) in FIG. 33, it can be shown as follows.

[0423] scallFactHeight = ( cIdx == 0 || SubHeightC == 1) ? 1 : 2

[0424] Also, the x-coordinate of the peripheral position can be indicated based on the x-axis scale information (scallFactWidth), and the y-coordinate of the peripheral position can be indicated based on the y-axis scale information (scallFactHeight). For example, the coordinate of NbB can be set based on the x-axis scale information (scallFactWidth). For example, the coordinate of NbA can be set based on the y-axis scale information (scallFactHeight). Also, as described above, being based on the x-axis scale information (scallFactWidth) may be based on SubWidthC, and being based on the y-axis scale information (scallFactHeight) may be based on SubHeightC. For example, the peripheral position coordinates may be as follows:

[0425] (xNbA, yNbA) = (xCb - 1, yCb + (cbHeight * scallFactHeight) - 1)

[0426] (xNbB, yNbB) = (xCb + (cbWidth * scallFactWidth) - 1, yCb - 1)

[0427] In this case, xCb and yCb may be coordinates based on the luma sample as described above, and cbWidth and cbHeight may be coordinates based on each color component.

[0428] FIG. 34 is a diagram illustrating a weighted sample prediction process according to an embodiment of the present disclosure.

[0429] In the embodiments of FIG. 30, FIG. 32, and FIG. 33, the availability of a corresponding position is determined by referring to neighboring positions. At this time, cIdx, which is an index indicating a color component, is set to 0 (luma component). In addition, when determining whether a corresponding position is available, information (cIdx) on the color component of the current block can be used to determine whether reconstruction of the corresponding position cIdx is completed. That is, when determining whether a corresponding position is available, cIdx can be used to determine the value of IsAvailable[cIdx][xNbY][yNbY]. However, when performing a weighted sample prediction process for a chroma block, referring to the IsAvailable value corresponding to cIdx0 may result in an erroneous determination. For example, when the reconstruction of the luma component of a block including a neighboring position is not completed but the reconstruction of the chroma component is completed, when cIdx is 0, IsAvailable[0][xNbY][yNbY] may be FALSE and IsAvailable[cIdx][xNbY][yNbY] may be TRUE. Therefore, a case occurs where it is determined that the neighbor is not available even though it is actually available. In the embodiment of FIG. 34, in order to solve this problem, when determining whether the corresponding position is available with reference to the neighboring position, the cIdx of the current coding block can be used as an input. That is, when invoking "the derivation process for neighbouring block availability", the cIdx of the current coding block can be input as the input cIdx.

[0430] Moreover, the contents described with reference to FIGS. 32 and 33 may be omitted.

[0431] In addition, when determining the prediction mode of the above-mentioned peripheral position, CuPredMode[0][xNbX][yNbY], which is CuPredMode corresponding to chType0, is referenced, but if the chType for the current block does not match, an incorrect parameter may be referenced. Therefore, according to the embodiment of the present disclosure, when determining the prediction mode of the peripheral position, CuPredMode[chType][xNbX][yNbY] corresponding to the chType value corresponding to the current block can be referenced.

[0432] FIG. 35 is a diagram illustrating a CIIP weight derivation according to one embodiment of the present disclosure.

[0433] In the embodiment described with reference to FIGS. 29 to 34, the contents related to the derivation of the CIIP weight value are described, and in this embodiment, the above contents may be omitted.

[0434] Also, in the embodiment described in FIG. 29 to FIG. 34, the weighting used in the CIIP can be determined based on the same position relative to a number of color components. Therefore, this can mean that the weighting used in the CIIP can be determined based on the neighboring locations described for the luma component with respect to the chroma component. Therefore, this can mean that the weighting used in the CIIP can be determined based on the neighboring locations described for the luma component with respect to a number of color components. Therefore, this can mean that the weighting used in the CIIP can be determined based on the same neighboring location with respect to a number of color components. Also, this can mean that the weighting used in the CIIP is the same for a number of color components. This is because, as described above, it may refer to the prediction mode of the neighboring location. Or, this is because the prediction mode at the same position relative to the color component field may be the same. More specifically, when the prediction mode corresponding to the already set position for each color component is the same for the color component field, the weighting used in the CIIP for a number of color components may be the same. In addition, when inter prediction can be used, the prediction mode corresponding to the already set position for each color component may be the same for the color component field. Or, when it is a P slice or a B slice, the prediction mode corresponding to the already set position for each color component may be the same for the color component field. This is because it may be SINGLE_TREE when it is a P slice or a B slice.

[0435] Also, as mentioned above, the neighboring positions described for the luma component may be as follows.

[0436] (xNbA, yNbA) = (xCb - 1, yCb + cbHeight - 1)

[0437] (xNbB, yNbB) = (xCb + cbWidth - 1, yCb - 1)

[0438] In short, according to the embodiment of the present disclosure, the weightings used in the CIIP for multiple color components may be the same. More specifically, the weightings used in the CIIP derived for the luma component may be used for the chroma components. This may be to prevent the process of deriving weightings for multiple color components from being performed multiple times. Also, weighting values ​​based on (xNbA, yNbA) and (xNbB, yNbB) may be used for the chroma components. In this case, the cbHeight and cbWidth in (xNbA, yNbA) and (xNbB, yNbB) may be values ​​based on the luma sample. Also, the xCb and yCb values ​​in (xNbA, yNbA) and (xNbB, yNbB) may be values ​​based on the luma sample. That is, for chroma components, a weighting value determined based on the width, height, and adjacent position based on the coordinates of the luma sample can be used in the CIIP.

[0439] 35, the derived weighting values ​​are used for each color component, i.e., Y, Cb, and Cr, i.e., the same weighting value is used in the CIIP for all color components.

[0440] FIG. 36 is a diagram illustrating a CIIP process according to one embodiment of the present disclosure.

[0441] The embodiment of Fig. 36 may be related to the embodiment described in Fig. 35. Also, the embodiment of Fig. 36 may show the structure of the embodiment described in Fig. 35. Also, in the embodiment of Fig. 36, the contents described in Figs. 29 to 35 may be omitted.

[0442] As described in FIG. 35, when using CIIP for each color component, it is possible to use the same weighting value.

[0443] Referring to FIG. 36, ciip_flag may be signaling indicating whether CIIP is used or not. If CIIP is used, the process referred to in FIG. 36 may be performed. If CIIP is used, a weighting value derivation process may be performed. At this time, the weighting value derivation process may be performed regardless of the color component. The weighting value derivation process may include the process described in FIG. 30. At this time, the neighbor position may be determined regardless of the color component. For example, the following neighbor positions may be used.

[0444] (xNbA, yNbA) = (xCb - 1, yCb + cbHeight - 1)

[0445] (xNbB, yNbB) = (xCb + cbWidth - 1, yCb - 1)

[0446] Also, referring to FIG. 36, xCb, yCb, cbWidth, cbHeight, which are inputs of the weighting value derivation process, and xCb, yCb, cbWidth, cbHeight of the neighboring positions may be based on components that have already been set regardless of the currently performed color component. For example, they may be values ​​based on the luma sample. That is, when the weighting value derivation process is performed for the chroma component, it may be based on xCb, yCb, cbWidth, cbHeight based on the luma sample. More specifically, when the weighting value derivation process is performed for the chroma component, the neighboring positions may be set based on xCb, yCb, cbWidth, cbHeight based on the luma sample, and the weighting value may be determined based on the neighboring positions.

[0447] The weight w may be set based on the weight derivation process. The weight derivation process of Fig. 36 may be the process of setting the weight w described in Fig. 30. In this case, as described above, a neighboring position unrelated to a color component may be used.

[0448] Also, referring to FIG. 36, when the CIIP is used, a general intra sample prediction process and a weighted sample prediction process can be performed. Also, this can be performed for each color component field. The weighted sample prediction process can include a process of combining inter prediction and intra prediction using the weighting described in FIG. 30. For example, the weighted sample prediction process can include a process of combining using equation (8-840) of FIG. 30. Also, according to an embodiment of the present disclosure, the weighted sample prediction process can be performed based on a weighting value, and the same weighting value can be used for multiple color component fields. Also, the weighting value used in the weighted sample prediction can be the weighting value w determined based on the above-mentioned weighting value derivation process.

[0449] Also, referring to Figure 36, the width and height of a coding block, which is an input of the weighted sample prediction process, may be values ​​indicated based on each color component. Referring to Figure 36, in the case of a luma component (when cIdx is 0), the width and height may be cbWidth and cbHeight, respectively, and in the case of a chroma component (when cIdx is not 0; when cIdx is 1 or 2), the width and height may be cbWidth / SubWidthC and cbHeight / SubHeightC, respectively.

[0450] That is, a weighted sample prediction process may be performed for a certain color component using a weighting value determined based on other color components, and using a width and height based on the certain color component. Alternatively, a weighted sample prediction process may be performed for a certain color component using a weighting value determined based on a width and height based on other color components, and using a width and height based on the certain color component. In this case, the certain color component may be a chroma component, and the other color component may be a luma component. In yet another embodiment, the certain color component may be a luma component, and the other color component may be a chroma component.

[0451] This may be for the purpose of simplifying implementation. For example, the process shown in Fig. 36 may be performed for each color component, but by performing the weight derivation process regardless of the color component, the implementation of the weight derivation process may be simplified. Alternatively, this may be for the purpose of avoiding a repetitive process.

[0452] FIG. 37 is a diagram showing the ranges of MV and MVD according to one embodiment of the present disclosure.

[0453] According to the embodiment of the present disclosure, there may be multiple MVD generating or determining methods. The MVD may be the motion vector difference as described above. In addition, there may be multiple motion vector (MV) generating or determining methods.

[0454] For example, the MVD determination method may include a method of determining from the value of a syntax element. For example, the MVD may be determined based on the syntax elements as described in FIG. 9. This is further described in FIG. 40.

[0455] In addition, the MVD determination method may include a method for determining when using the MMVD (merge with MVD) mode. For example, there may be an MVD to be used when using the merge mode. This will be further described with reference to Figures 38 to 39.

[0456] In the embodiment of the present disclosure, the MV or MVD may include a control point motion vector (CPMV) for performing affine motion compensation, i.e., the MV and MVD may include CPMV and CPMVD, respectively.

[0457] According to an embodiment of the present disclosure, the range that an MV or MVD can represent may be limited. This allows the MV or MVD to be represented and stored using limited resources and a limited number of bits, and allows calculations to be performed using the MV or MVD. In the embodiment of the present disclosure, the range that an MV or MVD can represent may be referred to as an MV range and an MVD range, respectively.

[0458] According to one embodiment, the MV range or MVD range may be from -2^N to (2^N-1). In this case, the range may be inclusive of -2^N and (2^N-1). According to yet another embodiment, the MV range or MVD range may be from (-2^N+1) to 2^N. In this case, the range may be inclusive of (-2^N+1) and 2^N. In this embodiment, N may be an integer, for example, a positive integer. More specifically, N may be 15 or 17. In this case, the MV range or MVD range can be represented using N+1 bits.

[0459] In addition, in order to implement the above-described restricted MV range or restricted MVD range, clipping or modulus operation can be performed. For example, the MV or MVD at a certain stage (e.g., before the final stage) can be clipped or modulus operated to derive a value within the MV range or MVD range. Alternatively, in order to implement the restricted MV range or restricted MVD range, the binarized expression range of the syntax element indicating the MV or MVD can be restricted.

[0460] According to an embodiment of the present disclosure, the MV range and the MVD range may be different. Also, there are many methods for determining MVD, and MVD1 and MVD2 may be MVDs determined by different methods. According to another embodiment of the present disclosure, the MVD1 range and the MVD2 range may be different. Using different ranges may be for using different resources or different numbers of bits, thereby avoiding the need to indicate an unnecessarily wide range for a certain element.

[0461] According to an embodiment, MVD1 may be the MVD described in FIG. 40 or FIG. 9. Alternatively, MVD1 may be the MVD of AMVP, inter mode, or affine inter mode. Alternatively, MVD1 may be the MVD when the merge mode is not used. Whether or not the merge mode is used may be indicated by merge_flag or general_merge_flag. For example, when merge_flag or general_merge_flag is 1, the merge mode is used, and when it is 0, the merge mode is not used.

[0462] According to an embodiment, MVD2 may be the MVD described in Figures 38 to 39. Alternatively, MVD2 may be the MVD in the MMVD mode. Alternatively, MVD2 may be the MVD when using the merge mode.

[0463] According to an embodiment, the MV may be the MV used for final motion compensation or prediction. Or the MV may be the MV in the candidate list. Or the MV may be a collocated motion vector (temporal motion vector). Or the MV may be the MVP plus the MVD. Or the MV may be the CPMV. Or the MV may be the CPMVP plus the CPMVD. Or the MV may be the MV for each sub-block in the affine MC. The MV for the sub-block may be an MV derived from the CPMV. Or the MV may be the MV of the MMVD.

[0464] According to an embodiment of the present disclosure, the MVD2 range may be narrower than the MVD1 range or the MV range. Also, the MVD1 range and the MV range may be the same. According to an embodiment, the MVD1 range and the MV range may be from -2^17 to (2^17-1) (inclusive). Also, the MVD2 range may be from -2^15 to (2^15-1) (inclusive).

[0465] Referring to FIG. 37, the dotted rectangle indicates the MV range or MVD range. The inner rectangle may indicate the MVD2 range. The outer rectangle may indicate the MVD1 range or MV range. That is, the MVD2 range may be different from the MVD1 range or MV range. The range in the drawing may indicate a vector range that can be indicated from the point of the drawing. For example, as described above, the MVD2 range may be from -2^15 to (2^15-1) (inclusive). The MVD1 range or MV range may be from -2^17 to (2^17-1) (inclusive). If the MV or MVD is a value in x-pel units, the value indicated by the MV or MVD may actually be (MV*x) or (MVD*x) pixels. For example, if the value is in 1 / 16-pel units, it may indicate (MV / 16) or (MVD / 16) pixels. In this embodiment, in the case of the MVD2 range, the maximum absolute value is 32768, and if this is a value in 1 / 16-pel units, the value of (MVD2 value) / 16 is a maximum of 2048 pixels. Therefore, the 8K picture cannot be completely covered. In the case of the MVD1 range or the MV range, the maximum absolute value is 131072, and when using 1 / 16-pel units, a maximum of 8192 pixels can be indicated. Therefore, the 8K picture can be completely covered. The 8K resolution can indicate a resolution whose horizontal length (or the longer of the horizontal or vertical length) is 7680 pixels or 8192 pixels. For example, a picture such as 7680×4320 may be an 8K resolution.

[0466] FIG. 38 is a diagram illustrating an MMVD according to one embodiment of the present disclosure.

[0467] According to an embodiment of the present disclosure, the MMVD may be a merge mode with MVD or a merge with MVD, which is a method of using the MVD in the merge mode. For example, an MV can be generated based on a merge candidate and an MVD. In addition, the MVD of the MMVD may have a limited range of possible indications compared to the MVD of FIG. 9 or FIG. 40, or the MVD1 described above. For example, the MVD of the MMVD may have only one of a horizontal component and a vertical component. In addition, the absolute values ​​of the values ​​that the MVD of the MMVD can indicate may not be equally spaced from each other. FIG. 38(a) shows points that the MVD of the MMVD can indicate from the central dotted line point.

[0468] FIG. 38(b) illustrates MMVD-related syntax. According to an embodiment, there may be higher level signaling indicating whether MMVD is used. Here, signaling may mean parsing from a bitstream. The higher level may be a unit including a current block and a current coding block, for example, a slice, a sequence, a tile, a tile group, a CTU, etc. The higher level signaling indicating whether MMVD is used may be higher level MMVD activation information (sps_mmvd_enabled_flag). When the higher level MMVD activation information (sps_mmvd_enabled_flag) is 1, it may indicate that MMVD is activated, and when the higher level MMVD activation information (sps_mmvd_enabled_flag) is 0, it may indicate that MMVD is not activated. However, this is not limited to the above, and when the higher level MMVD activity information (sps_mmvd_enabled_flag) is 0, it indicates that the MMVD is activated, and when the higher level MMVD activity information (sps_mmvd_enabled_flag) is 1, it indicates that the MMVD is not activated.

[0469] If the upper level MMVD activeness information (sps_mmvd_enabled_flag), which is the upper level signaling indicating whether the MMVD is used, is 1, the video signal processing device can indicate whether the MMVD is used for the current block by additional signaling. That is, if the upper level MMVD activeness information indicates the activeness of the MMVD, the video signal processing device can perform a step of parsing the MMVD merge information (mmvd_merge_flag), which indicates whether the MMVD is used for the current block, from the bitstream. The signaling indicating whether the MMVD is used may be the MMVD merge information (mmvd_merge_flag). If the MMVD merge information (mmvd_merge_flag) is 1, it may mean that the MMVD is used for the current block. If the MMVD merge information (mmvd_merge_flag) is 0, it may mean that the MMVD is not used for the current block. However, this is not limited to the above, and when the MMVD merge information (mmvd_merge_flag) is 0, it may mean that the MMVD is used for the current block, and when the MMVD merge information (mmvd_merge_flag) is 1, it may mean that the MMVD is not used for the current block.

[0470] If the MMVD merge information (mmvd_merge_flag) is 1 (a value indicating use), the MMVD-related syntax elements can be parsed. The MMVD-related syntax elements can include at least one of mmvd_cand_flag, information related to the distance of the MMVD (mmvd_distance_idx), and information related to the direction of the MMVD (mmvd_direction_idx).

[0471] As an example, mmvd_cand_flag may indicate an MVP to be used in the MMVD mode. Alternatively, mmvd_cand_flag may indicate a merge candidate to be used in the MMVD mode. Also, the MVD of the MMVD may be determined based on information related to the distance of the MMVD (mmvd_distance_idx) and information related to the direction of the MMVD (mmvd_direction_idx). Here, the MVD of the MMVD may indicate information related to the MVD (mMvdLX). For example, the information related to the distance of the MMVD (mmvd_distance_idx) may indicate a value related to the absolute value of the MVD of the MMVD, and the information related to the direction of the MMVD (mmvd_direction_idx) may indicate a value related to the direction of the MVD of the MMVD.

[0472] FIG. 39 is a diagram illustrating MVD derivation of an MMVD according to one embodiment of the present disclosure.

[0473] Referring to the left side of FIG. 39, the video signal processing device may determine whether or not an MMVD is used for the current block based on the MMVD merge information (mmvd_merge_flag). Also, when the MMVD is used for the current block, an MVD derivation process may be performed. The MVD derivation process may be 8.5.2.7 shown on the right side of FIG. 39. Also, the MVD, which is the output of 8.5.2.7, may be the MVD of the MMVD or may be information (mMvdLX) related to the MMVD. As described above, the information (mMvdLX) related to the MMVD may be obtained based on information related to the distance (mmvd_distance_idx) of the MMVD and information related to the direction (mmvd_direction_idx) of the MMVD. Here, X may be replaced with 0 or 1, and may correspond to the reference list L0 and the reference list L1, respectively.

[0474] Also, as shown in FIG. 39, information (mMvdLX) regarding the MVD of MMVD or MMVD can be added to mvLX, which is the motion vector (MV) derived in the previous process ((8-281), (8-282)). Here, X may be replaced with 0, 1, etc., and may correspond to the first reference list (reference list L0) and the second reference list (reference list L1), respectively. Also, here, the MVD of MMVD can indicate information (mMvdLX) regarding MMVD. Also, the video signal processing apparatus can perform a process of restricting the range of the modified motion vector (mvLX) obtained by adding the MVD. For example, clipping can be performed ((8-283), (8-284)). For example, it can be restricted to the above-described MV range. For example, it can be restricted to from -2^17 to (2^17-1) (inclusive).

[0475] In the present disclosure, Clip3(x, y, z) may indicate clipping. For example, the result of Clip3(x, y, z) can be 1) x when z < x, 2) y when z > y, and 3) z in other cases. Therefore, the range of the result of Clip3(x, y, z), the result, may be x <= result <= y.

[0476] In the present embodiment, in mvLX[0][0][comp] and mMvdLX[comp], comp can indicate the x-component or the y-component. For example, they may indicate the horizontal component and the vertical component, respectively.

[0477] Also, the video signal processing apparatus can predict the current block based on the modified motion vector (mvLX). Also, the video signal processing apparatus can restore the current block based on the modified motion vector (mvLX). As already described, the modified motion vector (mvLX) may be a value obtained by adding information (mMvdLX) regarding MMVD to mvLX, which is the motion vector (MV).

[0478] Also, the process of deriving the MVD of the MMVD may be the same as that shown on the right side of FIG. 39. The MMVD offset (MmvdOffset) in the drawing may be a value based on the above-mentioned MMVD-related syntax elements. For example, the MMVD offset (MmvdOffset) may be a value based on the distance-related information (mmvd_distance_idx) of the MMVD and the direction-related information (mmvd_direction_idx) of the MMVD. In short, the MMVD offset (MmvdOffset) may be obtained based on at least one of the distance-related information (mmvd_distance_idx) and the direction-related information (mmvd_direction_idx) of the MMVD. Also, information about the MMVD (mMvdLX) may be derived based on the MMVD offset (MmvdOffset).

[0479] According to one embodiment, in the case of bi-prediction (predFlagLX may indicate which reference list to use, X may be replaced with 0 or 1, and L0 and L1 may correspond to the first reference list (L0) and the second reference list (L1), respectively), it can be determined whether to use the MMVD offset (MmvdOffset) as is as the MVD of the MMVD based on the picture order count (POC), whether to use a value calculated based on the MMVD offset (MmvdOffset), whether to use the MMVD offset (MmvdOffset) as is as a value for a certain reference list, or whether to use a value calculated based on the MMVD offset (MmvdOffset) as a value for a certain reference list.

[0480] According to an embodiment, the MMVD offset (MmvdOffset) may be used to obtain MVD for both the first reference list (reference list L0) and the second reference list (reference list L1) ((8-350) to (8-353)). For example, the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) may be the same. The POC difference (currPocDiffLX) may be the difference between the POC of the current picture and the POC of the reference picture in the reference list LX, where X may be replaced with 0 and 1. The first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) may be determined according to (8-348) and (8-349) of FIG. 39, respectively. DiffPicOrderCnt may be calculated as follows:

[0481] DiffPicOrderCnt( picA, picB ) = PicOrderCnt( picA ) - PicOrderCnt( picB )

[0482] PicOrderCnt(picX) may indicate the POC value of picture picX.

[0483] 39, currPic may indicate a current picture, and RefPicList[X][refIdxLX] may indicate a reference picture when refIdxLX is used in reference list LX.

[0484] According to one embodiment, a value calculated based on MmvdOffset may be used as the MVD for the second reference list (reference list L1) ((8-354) to (8-363)). In this case, the MMVD offset (MmvdOffset) value may be used as the MVD for the first reference list (reference list L0) as is. In this case, Abs(currPocDiffL0) may be the same as or larger than Abs(currPocDiffL1). Alternatively, Abs(currPocDiffL0) may be larger than Abs(currPocDiffL1).

[0485] According to one embodiment, a value calculated based on the MMVD offset (MmvdOffset) may be used as the MVD for the first reference list (reference list L0) ((8-364) to (8-373)). In this case, the MMVD offset (MmvdOffset) value may be used as the MVD for the second reference list (reference list L1) as is. In this case, Abs(currPocDiffL0) may be smaller than Abs(currPocDiffL1). Alternatively, Abs(currPocDiffL0) may be smaller than or equal to Abs(currPocDiffL1).

[0486] According to an embodiment, the calculation based on the MMVD offset (MmvdOffset) may indicate MV scaling. The MV scaling may be (8-356) to (3-361) or (8-366) to (3-371) in FIG. 39. The former may be scaling based on the information (mMvdL0) related to the first MMVD or the MMVD offset (MmvdOffset) and may be a process of generating information (mMvdL1) related to the second MMVD which is a scaled MV, and the latter may be scaling based on the information (mMvdL1) related to the second MMVD or the MMVD offset (MmvdOffset) and may be a process of generating information (mMvdL0) related to the first MMVD which is a scaled motion vector (MV). The MV scaling may be calculated based on a scale factor (distScaleFactor ((8-359), (8-369))) which is a value based on the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1). The MV scaling may also be calculated based on a value obtained by multiplying the scale factor (distScaleFactor) by the scaled MV ((8-360), (8-361), (8-370), (8-371)). The MV scaling may also include a process of limiting the MV range. For example, a range limiting process may be included in (8-360), (8-361), (8-370), and (8-371) in FIG. 39. For example, the MV scaling may include a clipping process. According to an embodiment of the present disclosure, the above-mentioned MVD2 range may be used at this time. For example, the range may be limited to -2^15 to (2^15-1) (inclusive). That is, the MVD range of the MMVD can be limited to -2^15 to (2^15-1) (inclusive). For example, the MVD derivation process of the MMVD can include a process of limiting a value based on the product of distScaleFactor and the MV to be scaled to -2^15 to (2^15-1) (inclusive). For example, the Clip3(-2^15,2^15-1,x) process may be included in the MVD derivation process of the MMVD.More specifically, mMvdLX may be determined as follows:

[0487] mmvdLX = Clip3(-2^15, 2^15 - 1, (distScaleFactor*mMvdLY + 128 - (distScaleFactor*mMvdLY >=0 )) >> 8 )

[0488] In this case, Y may be 0 or 1, or may be !X. mMvdLY may be MmvdOffset. distScaleFactor may be a value shown in (8-359) or (8-369), and may be a value based on currPocDiffL0 and currPocDiffL1. However, the range of the MVD of MMVD, information on MMVD (mMvdLX), or scaled MV is not limited to -2^15 to (2^15-1). The range of the MVD of MMVD, information on MMVD (mMvdLX), or scaled MV may be -2^17 to (2^17-1). This will be described in FIG. 43.

[0489] This process can be performed when neither the reference pictures for L0 nor L1 are long-term reference pictures.

[0490] According to an embodiment, the calculation based on the MMVD offset (MmvdOffset) may indicate an MMVD offset (MmvdOffset) or a negative MMVD offset (-MmvdOffset) based on the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) ((8-362), (8-363), (8-372), (8-373)). This process can be performed when at least one of the reference pictures for L0 and L1 is a long-term reference picture.

[0491] Also, if uni-prediction is used, it is possible to use MmvdOffset as is in mMvdLX ((8-374), (8-375)).

[0492] FIG. 40 is a diagram illustrating MVD and MV derivation according to one embodiment of the present disclosure.

[0493] The embodiment of Figure 40 may be an example using the MV range or the MVD1 range of Figure 37. The embodiment of Figure 40 may be an example using the range from -2^17 to (2^17-1) (inclusive).

[0494] Referring to FIG. 40, lMvd can be derived based on the syntax elements described in FIG. 9 and the like (7-161). lMvd may be in the range of -2^17 to (2^17-1) (inclusive). Values ​​such as MvdLX and MvdCpLX may be set to lMvd. lMvd, MvdLX, MvdCpLX, and the like may be the above-mentioned MVD1.

[0495] Also, when using AMVP, inter mode, etc., the MV range can be limited to -2^17 to (2^17-1) (inclusive). In this case, merge mode may not be used. Referring to FIG. 40, (8-264) to (8-267) may include an MV range limiting process. Referring to the drawing, uLX may be a value based on the sum of MVP (mvpLX) and MVD (mvdLX). Also, mvLX, which is the final MV, can be calculated based on ULX. FIG. 40 includes a modulus operation, but the value indicated by this operation may be limited so that it can be expressed in a certain number of bits or less. For example, it can be expressed in 18 bits by calculating (%2^18). Therefore, the MV range can be limited to -2^17 to (2^17-1) (inclusive). (8-265) and (8-267) can play a role in resolving the overflow that may occur when adding MVP and MVD.

[0496] FIG. 41 is a diagram showing MV and CPMV derivation according to one embodiment of the present disclosure.

[0497] The embodiment of Figure 41 may be an example using the MV range or the MVD1 range of Figure 37. The embodiment of Figure 40 may be an example using the range from -2^17 to (2^17-1) (inclusive).

[0498] FIG. 41(a) may show a method for deriving a collocated motion vector (temporal motion vector). This process may include MV scaling. In this case, it may include an operation of clipping a value based on distScaleFactor and the scaled MV (mvCol) ((8-398)). In this case, the clipping may result in a range (inclusive) of -131072, 131071. -131072 is the same value as -2^17. 131071 is the same value as 2^17-1.

[0499] 41(b) is a diagram showing a part of the process of deriving CPMV (cpMvLX). Again, a clipping process may be included to limit the range. In this case, the range may be from -2^17 to (2^17-1) (inclusive).

[0500] FIG. 41(c) is a diagram showing a part of a process of deriving a subblock-based MV. The subblock-based MV may be an MV when using affine MC, subblock-based temporal MV prediction, etc. In this case, a clipping process for limiting the range may be included. In this case, the range may be from -2^17 to (2^17-1) (inclusive). In the drawing, xSbIdx and ySbIdx may respectively indicate a subblock index for the x axis and a subblock index for the y axis.

[0501] FIG. 42 is a diagram showing the ranges of MV and MVD according to one embodiment of the present disclosure.

[0502] The description of the above-mentioned parts may be omitted in this embodiment.

[0503] As mentioned above, there may be many methods for generating or determining an MVD. The MVD may be a motion vector difference as mentioned above. There may also be many methods for generating or determining an MV.

[0504] At this time, according to an embodiment of the present disclosure, the range that the MV or MVD can indicate may be limited.

[0505] According to an embodiment of the present disclosure, the MV range and the MVD range may be the same. Also, there may be an MVD generation method, and the MVD generated by the MVD generation method 1 may be MVD1, and the MVD generated by the MVD generation method 2 may be MVD2. In this case, the MVD1 range and the MVD2 range may be the same. Also, the MV range, the MVD range, the MVD1 range, and the MVD2 range may be from -2^N to (2^N-1). In this case, the range may include -2^N and (2^N-1). According to still another embodiment, the MV range or the MVD range may be from (-2^N+1) to 2^N. In this case, the range may include (-2^N+1) and 2^N. In these embodiments, N may be an integer, for example, a positive integer. More specifically, N may be 17. Also, in this case, the MV range or the MVD range may be represented using N+1 bits. Therefore, the MV range, MVD range, MVD1 range, and MVD2 range may all be from -2^17 to (2^17-1).

[0506] Also, as described in the previous drawings, MV, MVD1, MVD2, etc. may be defined. That is, MV may mean an MV used for final motion compensation or prediction, etc., and the description of FIG. 37 may be referred to. Also, MVD1 may be the MVD described in FIG. 40 or FIG. 9. Alternatively, MVD1 may be the MVD of AMVP, inter mode, affine inter mode, or the MVD when merge mode is not used, and the description of FIG. 37 may be referred to. Also, MVD2 may be the MVD described in FIG. 38 to FIG. 39, or the MVD of MMVD mode, and the description of FIG. 37 may be referred to. Here, the MVD of the MMVD mode may be information (mMvdLX) related to the MMVD.

[0507] Therefore, according to one embodiment, the MVDs of AMVP, inter mode, affine inter mode, and MMVD may all have the same range of possible indications. Furthermore, the final MV may also have the same range. More specifically, the MVDs of AMVP, inter mode, affine inter mode, and MMVD may all have a range of -2^17 to (2^17-1) (inclusive). Furthermore, the final MV may also have a range of -2^17 to (2^17-1) (inclusive).

[0508] This allows MV and MVD to completely cover a picture of a certain size. Also, when there is a range that a certain MV or MVD1 can indicate, MVD2 can also express the same range. Therefore, for example, a method using MVD2 is not used restrictively compared to other methods depending on the expression range. Also, when MV, MVD1, etc. can indicate any range, making MVD2 indicate the same range does not require additional resources in hardware or software. If MVD2 or MVD of MMVD can express -2^17 to (2^17-1) (inclusive), the maximum absolute value is 131072, and when this is a value in 1 / 16-pel units, it can indicate a maximum of 8192 pixels. Therefore, an 8K picture can be completely covered.

[0509] Referring to FIG. 42, the outer dotted lines indicate the ranges that MV, MVD1, and MVD2 can indicate from the central point. The MV range, MVD1 range, and MVD2 range may be the same. For example, the MV range, MVD1 range, and MVD2 range may be inclusive of -2^17 to (2^17-1). This range may include all 8K pictures. That is, in the worst case, the MV, MVD1, and MVD2 ranges can indicate the picture from end to end. Therefore, better motion vectors can be indicated, and motion compensation can be improved, reducing residuals and increasing coding efficiency.

[0510] FIG. 43 is a diagram illustrating MVD derivation of MMVD according to one embodiment of the present disclosure.

[0511] Fig. 43 may be a partly modified version of Fig. 39. In the embodiment of Fig. 43, the contents described in Fig. 39 may be omitted. Alternatively, the contents described above may be omitted.

[0512] When the MMVD is used, an MVD derivation process can be performed. The MVD derivation process may be 8.5.2.7 shown in FIG. 43. Also, the MVD output from 8.5.2.7 may be the MVD of the MMVD, or may be information about the MMVD (mMvdLX). As already described, the information about the MMVD (mMvdLX) may be obtained based on information related to the distance of the MMVD (mmvd_distance_idx) and information related to the direction of the MMVD (mmvd_direction_idx).

[0513] More specifically, a description will be given together with Fig. 43. Referring to Fig. 43, the video signal processing device can acquire the MMVD offset (MmvdOffset). The video signal processing device can acquire the MMVD offset (MmvdOffset) in order to acquire information (mMvdLX) related to the MMVD. The MMVD offset (MmvdOffset) may be a value based on the above-mentioned MMVD-related syntax elements. For example, the MMVD offset (MmvdOffset) may be a value based on information related to the distance of the MMVD (mmvd_distance_idx) and information related to the direction of the MMVD (mmvd_direction_idx).

[0514] According to an embodiment, a video signal processing apparatus may determine whether a current block is predicted in a bi-predictive manner based on predFlagLX. predFlagLX may indicate which reference list is to be used. Here, X may be replaced with 0 or 1, and L0 and L1 may correspond to reference lists L0 and L1, respectively. When predFlagL0 is 1, it may indicate that a first reference list is to be used. Also, when predFlagL0 is 0, it may indicate that the first reference list is not to be used. When predFlagL1 is 1, it may indicate that a second reference list is to be used. Also, when predFlagL1 is 0, it may indicate that the second reference list is not to be used.

[0515] In FIG. 43, when predFlagL0 and predFlagL1 are both 1, bi-prediction may be indicated. Bi-prediction may indicate that both a first reference list and a second reference list are used. When the first reference list and the second reference list are used, the video signal processing device may perform a step (8-348) of acquiring a difference in POC (Picture Order Count) between a current picture (currPic) including a current block and a first reference picture (RefPicList[0][refIdxL0]) based on the first reference list as a first POC difference (currPocDiffL0). Also, when the first reference list and the second reference list are used, the video signal processing device may perform a step (8-349) of acquiring a difference in POC (Picture Order Count) between a current picture (currPic) and a second reference picture (RefPicList[1][refIdxL1]) based on the first list as a second POC difference (currPocDiffL1).

[0516] The POC difference (currPocDiffLX) may be the difference between the POC of the current picture and the POC of the reference picture (reference list) in the reference list (reference list LX), where X may be replaced with 0 or 1. The first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) may be determined by (8-348) and (8-349) in FIG. 43, respectively. DiffPicOrderCnt can be calculated as follows:

[0517] DiffPicOrderCnt( picA, picB ) = PicOrderCnt( picA ) - PicOrderCnt( picB )

[0518] PicOrderCnt(picX) may indicate the POC value of picture picX.

[0519] The video signal processing device may perform a step of acquiring information (mMvdL0) about a first MMVD associated with a first reference list and information (mMvdL1) about a second MMVD associated with a second reference list based on at least one of the MMVD offset (MmvdOffset), the first POC difference (currPocDiffL0), and the second POC difference (currPocDiffL1). Here, the information (mMvdLX) about the MMVD may include information (mMvdL0) about the first MMVD and information (mMvdL1) about the second MMVD. The process of the video signal processing device acquiring the information (mMvdL0) about the first MMVD and the information (mMvdL1) about the second MMVD will be described in detail below.

[0520] Based on the POC (picture order count), the video signal processing device can determine whether to use the MMVD offset (MmvdOffset) as is as the MVD of the MMVD, or to use a value calculated based on the MMVD offset (MmvdOffset), or to use the MMVD offset (MmvdOffset) as is as a value for a certain reference list, or to use a value calculated based on the MMVD offset (MmvdOffset) as a value for a certain reference list.

[0521] When the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) are the same, the video signal processing device may perform steps (8-350, 8-351) of acquiring the MMVD offset (MmvdOffset) as information (mMvdL0) related to the first MMVD. Also, when the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) are the same, the video signal processing device may perform steps (8-352, 8-353) of acquiring the MMVD offset (MmvdOffset) as information (mMvdL1) related to the second MMVD.

[0522] The video signal processing device may determine whether the absolute value of the first POC difference (Abs(currPocDiffL0)) is greater than the absolute value of the second POC difference (Abs(currPocDiffL1)). In addition, the video signal processing device may perform steps (8-354, 8-355) of acquiring an MMVD offset (MmvdOffset) as information (mMvdL0) related to the first MMVD when the absolute value of the first POC difference (Abs(currPocDiffL0)) is greater than or equal to the absolute value of the second POC difference (Abs(currPocDiffL1)).

[0523] When the first reference picture (RefPicList[0][refIdxL0]) is not a long-term reference picture and the second reference picture (RefPicList[1][refIdxL1]) is not a long-term reference picture, the video signal processing apparatus may perform steps (8-356 to 8-361) of scaling information (mMvdL0) related to the first MMVD to obtain information (mMvdL1) related to the second MMVD. A scale factor (distScaleFactor) may be used for scaling. The scale factor (distScaleFactor) may be obtained based on at least one of the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1).

[0524] When the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture, the video signal processing device may perform steps (8-362, 8-363) of acquiring information (mMvdL1) about the second MMVD without scaling the absolute value of information (mMvdL0) about the first MMVD. Here, not scaling may mean not changing the absolute value of information (mMvdL0) about the first MMVD. That is, the video signal processing device may acquire information (mMvdL1) about the second MMVD with or without changing the sign of information (mMvdL0) about the first MMVD. When the signs of the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) are the same, the video signal processing device may set information (mMvdL1) about the second MMVD to information (mMvdL0) about the first MMVD. In addition, when the signs of the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) are different, the video signal processing device can set information (mMvdL1) regarding the second MMVD by changing the sign of information (mMvdL0) regarding the first MMVD.

[0525] The video signal processing device may determine whether the absolute value of the first POC difference (Abs(currPocDiffL0)) is smaller than the absolute value of the second POC difference (Abs(currPocDiffL1)). If the absolute value of the first POC difference (Abs(currPocDiffL0)) is smaller than the absolute value of the second POC difference (Abs(currPocDiffL1)), the video signal processing device may perform steps (8-364, 8-365) of acquiring an MMVD offset (MmvdOffset) as information (mMvdL1) related to the second MMVD.

[0526] When the first reference picture (RefPicList[0][refIdxL0]) is not a long-term reference picture and the second reference picture (RefPicList[1][refIdxL1]) is not a long-term reference picture, the video signal processing apparatus may perform steps (8-366 to 8-371) of scaling information (mMvdL1) related to the second MMVD and acquiring information (mMvdL0) related to the first MMVD. A scale factor (distScaleFactor) may be used for scaling. The scale factor (distScaleFactor) may be acquired based on at least one of the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1).

[0527] When the first reference picture (RefPicList[0][refIdxL0]) is a long-term reference picture or the second reference picture (RefPicList[1][refIdxL1]) is a long-term reference picture, the video signal processing device may perform steps (8-372, 8-373) of acquiring information (mMvdL0) related to the first MMVD without scaling the absolute value of information (mMvdL1) related to the second MMVD. Here, not scaling may mean not changing the absolute value of information (mMvdL1) related to the second MMVD. That is, the video signal processing device may acquire information (mMvdL0) related to the first MMVD with or without changing the sign of information (mMvdL1) related to the second MMVD. The video signal processing device can set the information (mMvdL0) related to the first MMVD to the information (mMvdL1) related to the second MMVD when the signs of the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) are the same. Also, the video signal processing device can set the information (mMvdL0) related to the first MMVD to the information (mMvdL1) related to the second MMVD by changing the sign of the information (mMvdL1) related to the second MMVD when the signs of the first POC difference (currPocDiffL0) and the second POC difference (currPocDiffL1) are different.

[0528] In Fig. 43, uni-prediction can be indicated when at least one of predFlagL0 and predFlagL1 is not 1. In the case of uni-prediction, the MMVD offset (MmvdOffset) can be used as it is as information (mMvdLX) related to the MMVD ((8-374), (8-375)).

[0529] As already described, the video signal processing device can perform a step of obtaining an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD in order to obtain information related to the MMVD (mMvdLX).

[0530] When only the first reference list is used (predFlagL0==1), the video signal processing device can perform steps (8-374, 8-375) of not scaling the MMVD offset (MmvdOffset) and obtaining it as information (mMvdL0) regarding the first MMVD associated with the first reference list.

[0531] When only the second reference list is used (predFlagL1==1), the video signal processing device can perform steps (8-374, 8-375) of not scaling the MMVD offset (MmvdOffset) and obtaining it as information (mMvdL1) regarding the second MMVD associated with the second reference list.

[0532] As described above, the range of the MVD of the MMVD may be limited. Here, the MVD of the MMVD may be information (mMvdLX) related to the MMVD. According to an embodiment of the present disclosure, the range of the MVD of the MMVD may be the same as the range of the MVD determined by another method (e.g., the MVD based on the syntax element of FIG. 9). Also, the range of the MVD of the MMVD may be the same as the range of the final MV. According to an embodiment of the present disclosure, the MVD range of the MMVD or the information (mMvdLX) related to the MMVD may be from -2^17 to 2^17-1, inclusive.

[0533] As described above, the MVD derivation process of the MMVD may include an MV scaling process, and a clipping operation for limiting the range may be included in the MVD derivation process of the MMVD.

[0534] Referring to FIG. 43, information (mMvdLX) on MMVD may be based on a clipping operation. In this case, X may be replaced with 0 or 1, which may correspond to a first reference list (reference list L0) and a second reference list (reference list L1), respectively. For example, information (mMvdLX) on MMVD may be based on a Clip3 operation. For example, information (mMvdLX) on MMVD may be based on Clip3 (-2^17, 2^17-1, x). In this case, x may be a value based on a first POC difference (currPocDiffL0), a second POC difference (currPocDiffL1), or an MMVD offset (MmvdOffset). For example, x may be a value based on a scale factor (distScaleFactor) and an MMVD offset (MmvdOffset). (8-360), (8-361), (8-370), and (8-371) in FIG. 43 include clipping operations to limit the MVD range.

[0535] As a further example, in Figures 42 to 43, the unified range between MV, MVD, MVD1, and MVD2, the MVD range of MVD and MMVD by multiple MVD generation methods may be from -2^N+1 to 2^N, inclusive. More specifically, N may be 17. That is, the unified range between MV, MVD, MVD1, and MVD2, the MVD range of MVD and MMVD by multiple MVD generation methods may be from -2^17+1 to 2^17, inclusive. In this case, MVD can be expressed in 18 bits.

[0536] According to an embodiment of the present disclosure, the range of a chroma MV may be different from the range of a luma MV. For example, a chroma MV may have a higher resolution than a luma MV. That is, one unit of a chroma MV may represent a smaller pixel than one unit of a luma MV. For example, a luma MV may be 1 / 16-pel units. Also, a chroma MV may be 1 / 32-pel units.

[0537] Also, the chroma MV may be based on the luma MV. For example, the chroma MV may be determined by multiplying the luma MV by any value. The any value may be 2 / SubWidthC for the horizontal component and 2 / SubHeightC for the vertical component. The 2 included in the any value may be a value included because the resolution of the chroma MV is twice as high as the resolution of the luma MV. Also, SubWidthC and SubHeightC may be values ​​determined by a color format. Also, SubWidthC and SubHeightC may be values ​​for chroma sample sampling. For example, SubWidthC and SubHeightC may be values ​​related to how many chroma samples exist for a luma sample. Also, SubWidthC and SubHeightC may be 1 or 2.

[0538] According to an embodiment of the present disclosure, when the resolution of a chroma MV is higher than that of a luma MV, the range of the chroma MV may be wider than that of the luma MV. This may be for the chroma MV to cover the area covered by the luma MV. For example, the range of the chroma MV may be twice the range of the luma MV. For example, the chroma MV range may be from -2^18 to 2^18-1, inclusive of -2^18 and 2^18-1. Also, the luma MV range may be from -2^17 to 2^17-1, inclusive of -2^17 and 2^17-1.

[0539] Also, the range of chroma MV may vary depending on the color format. For example, when SubWidthC or SubHeightC is 1, the range of chroma MV may be different from the range of luma MV. In this case, the range of chroma MV may be twice the range of luma MV. For example, the chroma MV range may be from -2^18 to 2^18-1, and in this case, the range may include -2^18 and 2^18-1, and the luma MV range may be from -2^17 to 2^17-1, and in this case, the range may include -2^17 and 2^17-1. Also, when SubWidthC and SubHeightC are 2, the range of chroma MV may be the same as the range of luma MV. In this case, the range may be from -2^17 to 2^17-1, and in this case, the range may include -2^17 and 2^17-1. This may be for the purpose of unifying the range that chroma MV and luma MV can indicate.

[0540] In an embodiment of the present disclosure, in the case of a 4:2:0 format, SubWidthC and SubHeightC may be 2 and 2, respectively. In the case of a 4:2:2 format, SubWidthC and SubHeightC may be 2 and 1, respectively. In the case of a 4:4:4 format, SubWidthC and SubHeightC may be 1 and 1, respectively.

[0541] The above-described embodiments of the present disclosure may be implemented by various means, for example, by hardware, firmware, software, or a combination thereof.

[0542] In a hardware implementation, the methods according to the embodiments of the present disclosure may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0543] In a firmware or software implementation, the method according to the embodiment of the present disclosure may be implemented in the form of a module, procedure, or function that performs the functions or operations described above. The software code may be stored in a memory and executed by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means known in the art.

[0544] The above description of the present disclosure is for illustrative purposes only, and those skilled in the art will understand that the present disclosure can be easily modified into other specific forms without changing the technical idea or essential features of the present disclosure. Therefore, the above-described embodiments should be interpreted as illustrative and not restrictive in any respect. For example, each component described as a single type may be implemented in a distributed manner, and similarly, each component described as distributed may be implemented in a combined form.

[0545] The scope of the present disclosure is indicated by the claims set forth below rather than by the above detailed description, and any modifications or variations that fall within the meaning and scope of the claims and their equivalent concepts should be construed as being included within the scope of the present disclosure. [Explanation of symbols]

[0546] 110 Conversion unit 115 Quantization section 120 Inverse quantization section 125 Reverse conversion unit 130 Filtering section 150 Prediction Department 152 Intra prediction unit 154 Inter Prediction Unit 154a Motion Estimation Unit 154b Motion compensation section 160 Entropy coding unit 210 Entropy Decoding Unit 220 Inverse quantization section 225 Inverse conversion unit 230 Filtering section 250 Prediction Department 252 Intra prediction unit 254 Inter Prediction Unit 810 Encoder 820 Decoder

Claims

1. Parsing upper level MMVD activation information (sps_mmvd_enabled_flag) from the bitstream, the MMVD activation information indicating whether or not a higher level MMVD (Merge with MVD) including the current block is used; parsing MMVD merge information (mmvd_merge_flag) indicating whether to use MMVD for the current block from a bitstream when the higher level MMVD liveness information indicates liveness of MMVD; parsing distance-related information (mmvd_distance_idx) and direction-related information (mmvd_direction_idx) of the MMVD when the MMVD merge information indicates that the MMVD is to be used for the current block; and obtaining information about the MMVD (mMvdLX) by clipping the obtained value based on information related to the distance of the MMVD and information related to the direction of the MMVD; The clipping limits the output values ​​to the range of -2^17 to 2^17-1; The information (mMvdLX) about the MMVD is the MVD of the MMVD, A method for decoding a video signal, wherein the MVD has a value in the range of -2^17 to 2^17-1.

2. 2. The method of claim 1, wherein the upper level is any one of a coding tree unit, a slice, a tile, a tile group, a picture, or a sequence.

3. generating a merge candidate list for the current block; selecting a motion vector from the merge candidate list based on a merge index parsed from the bitstream; if the MMVD merge information indicates that MMVD is used for the current block, adding information about the MMVD to the motion vector to obtain a modified motion vector; and 2. The method of claim 1, further comprising: reconstructing the current block based on the modified motion vector that is clipped to a range of -2^17 to 2^17-1.

4. The step of acquiring information (mMvdLX) about the MMVD includes: obtaining an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD; obtaining a picture order count (POC) difference between a current picture including the current block and a first reference picture based on the first reference list as a first POC difference, when a first reference list and a second reference list are used, and obtaining a picture order count (POC) difference between the current picture and a second reference picture based on the second reference list as a second POC difference; obtaining information about a first MMVD associated with the first reference list and information about a second MMVD associated with the second reference list based on at least one of the MMVD offset, the first POC difference, and the second POC difference; 2. The method of claim 1, wherein the information about the MMVD includes information about the first MMVD and information about the second MMVD.

5. 5. The method of claim 4, further comprising the step of obtaining the MMVD offset as information about a first MMVD and obtaining the MMVD offset as information about a second MMVD when the first POC difference and the second POC difference are the same.

6. acquiring the MMVD offset as information about the first MMVD when the absolute value of the first POC difference is greater than or equal to the absolute value of the second POC difference; if the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, scaling information about the first MMVD to obtain information about the second MMVD; and 5. The method for decoding a video signal of claim 4, further comprising a step of obtaining information about the second MMVD when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture, and wherein scaling of the absolute value of the information about the first MMVD is not performed to obtain the information about the second MMVD.

7. acquiring the MMVD offset as information about the second MMVD when the absolute value of the first POC difference is smaller than the absolute value of the second POC difference; if the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, scaling information about the second MMVD to obtain information about the first MMVD; and 5. The method for decoding a video signal of claim 4, further comprising a step of obtaining information about the first MMVD when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture, and wherein scaling of the absolute value of the information about the first MMVD is not performed to obtain information about the second MMVD.

8. The step of acquiring information (mMvdLX) about the MMVD includes: obtaining an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD; if only a first reference list is used, not scaling the MMVD offset and obtaining it as information about a first MMVD associated with the first reference list; and 2. The method of claim 1, further comprising the step of: when only a second reference list is used, not scaling the MMVD offset and obtaining it as information about a second MMVD associated with the second reference list.

9. An apparatus for decoding a video signal includes a processor and a memory, The processor parses, based on an instruction stored in the memory, upper level MMVD activation information (sps_mmvd_enabled_flag) from the bitstream, which indicates whether or not a higher level MMVD (Merge with MVD) including the current block is used; If the higher level MMVD liveness information indicates liveness of the MMVD, parse MMVD merge information (mmvd_merge_flag) indicating whether to use MMVD for the current block from the bitstream; If the MMVD merge information indicates that the MMVD is to be used for the current block, parse information related to the distance of the MMVD (mmvd_distance_idx) and information related to the direction of the MMVD (mmvd_direction_idx); obtaining information about the MMVD (mMvdLX) by clipping the obtained values ​​based on information related to the distance of the MMVD and information related to the direction of the MMVD; The clipping limits the output values ​​to the range of -2^17 to 2^17-1; The information (mMvdLX) about the MMVD is the MVD of the MMVD, An apparatus for decoding a video signal, wherein the MVD has a value in the range of -2^17 to 2^17-1.

10. 10. The apparatus of claim 9, wherein the upper level is any one of a coding tree unit, a slice, a tile, a tile group, a picture, or a sequence.

11. The processor, based on the instructions stored in the memory, Generate a merge candidate list for the current block; selecting a motion vector from the merge candidate list based on a merge index parsed from the bitstream; If the MMVD merge information indicates that MMVD is used for the current block, add information about the MMVD to the motion vector to obtain a modified motion vector; 10. The apparatus of claim 9, further comprising:reconstructing the current block based on the modified motion vector that is clipped to a range of -2^17 to 2^17-1.

12. The processor, based on the instructions stored in the memory, obtaining an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD; When a first reference list and a second reference list are used, a difference in Picture Order Count (POC) between a current picture including the current block and a first reference picture based on the first reference list is obtained as a first POC difference; obtaining a picture order count (POC) difference between the current picture and a second reference picture based on the second reference list as a second POC difference; obtaining information about a first MMVD associated with the first reference list and information about a second MMVD associated with the second reference list based on at least one of the MMVD offset, the first POC difference, and the second POC difference; 10. The apparatus for decoding a video signal according to claim 9, wherein the information on the MMVD includes information on the first MMVD and information on the second MMVD.

13. The processor, based on the instructions stored in the memory, 13. The apparatus for decoding a video signal according to claim 12, characterized in that, when the first POC difference and the second POC difference are the same, the MMVD offset is obtained as information about a first MMVD, and the MMVD offset is obtained as information about a second MMVD.

14. The processor, based on the instructions stored in the memory, When the absolute value of the first POC difference is greater than or equal to the absolute value of the second POC difference, the MMVD offset is obtained as information related to the first MMVD; If the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, scaling information about the first MMVD to obtain information about the second MMVD; The apparatus for decoding a video signal as described in claim 12, characterized in that when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture, information about the second MMVD is obtained, and scaling of the absolute value of the information about the first MMVD is not performed to obtain the information about the second MMVD.

15. The processor, based on the instructions stored in the memory, When the absolute value of the first POC difference is smaller than the absolute value of the second POC difference, the MMVD offset is acquired as information related to the second MMVD; if the first reference picture is not a long-term reference picture and the second reference picture is not a long-term reference picture, scaling information about the second MMVD to obtain information about the first MMVD; 13. The apparatus for decoding a video signal as described in claim 12, characterized in that when the first reference picture is a long-term reference picture or the second reference picture is a long-term reference picture, information about the first MMVD is obtained, and scaling of the absolute value of the information about the first MMVD is not performed to obtain information about the second MMVD.

16. The processor, based on the instructions stored in the memory, obtaining an MMVD offset based on information related to the distance of the MMVD and information related to the direction of the MMVD; When only a first reference list is used, the MMVD offset is not scaled and is obtained as information about a first MMVD associated with the first reference list; 10. The apparatus for decoding a video signal according to claim 9, characterized in that, when only a second reference list is used, the MMVD offset is not scaled and is obtained as information about a second MMVD associated with the second reference list.

17. 1. A method for encoding a video signal, comprising the steps of: generating information (mMvdLX) about the MMVD of the current block; generating information related to the distance of the MMVD and information related to the direction of the MMVD based on the information about the MMVD (mMvdLX); generating MMVD merge information (mmvd_merge_flag) indicating whether or not MMVD is to be used for the current block; generating upper level MMVD activation information (sps_mmvd_enabled_flag) indicating whether or not a higher level MMVD (Merge with MVD) including the current block is used; and generating a bitstream based on distance-related information of the MMVD, direction-related information of the MMVD, the MMVD merge information (mmvd_merge_flag), and the upper level MMVD activation information (sps_mmvd_enabled_flag); The information about the MMVD is obtained by a device for decoding the video signal using clipping, the clipping limiting output values ​​to a range from -2^17 to 2^17-1; The information (mMvdLX) about the MMVD is the MVD of the MMVD, A method of encoding a video signal, wherein the MVD has a value ranging from -2^17 to 2^17-1.

18. 1. An apparatus for encoding a video signal, comprising: a processor and a memory; The processor, based on the instructions stored in the memory, Generate information (mMvdLX) about the MMVD of the current block, generating information related to the distance of the MMVD and information related to the direction of the MMVD based on the information (mMvdLX) on the MMVD; Generate MMVD merge information (mmvd_merge_flag) indicating whether or not to use MMVD for the current block; Generate upper level MMVD activation information (sps_mmvd_enabled_flag) indicating whether or not a higher level MMVD (Merge with MVD) including the current block is used; generating a bitstream based on distance-related information of the MMVD, direction-related information of the MMVD, the MMVD merge information (mmvd_merge_flag), and the upper level MMVD activation information (sps_mmvd_enabled_flag); The information about the MMVD is obtained by a device for decoding the video signal using clipping, the clipping limiting output values ​​to a range from -2^17 to 2^17-1; The information (mMvdLX) about the MMVD is the MVD of the MMVD, An apparatus for encoding a video signal, wherein the MVD has a value ranging from -2^17 to 2^17-1.

Images