Video encoding / decoding method and apparatus, and recording medium storing bitstreams

Abstract

A video decoding / encoding method and apparatus according to the present disclosure may determine a filter length of a deblocking filter to be applied to a block boundary between a first block and a second block in a current picture, determine a boundary strength of the deblocking filter, determine a deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and apply the deblocking filter to the block boundary between the first block and the second block. Here, the filter length of the deblocking filter may be determined based on whether multiple reference prediction is applied to at least one of the first block or the second block.

Description

[Technical Field]

[0001] The present invention relates to a video encoding / decoding method and apparatus, and a recording medium storing a bitstream. [Background technology]

[0002] 2. Description of the Related Art In recent years, the demand for high-resolution, high-quality images such as high-definition (HD) images and ultra-high-definition (UHD) images has increased in various application fields, and as a result, highly efficient image compression techniques have been discussed.

[0003] There are various video compression techniques, such as inter-prediction techniques that predict pixel values ​​contained in a current picture from pictures before or after the current picture, intra-prediction techniques that predict pixel values ​​contained in a current picture using pixel information within the current picture, and entropy coding techniques that assign short codes to values ​​that occur frequently and long codes to values ​​that occur less frequently. Using these video compression techniques, video data can be effectively compressed and transmitted or stored. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure seeks to provide a method and apparatus for determining the filter length of a deblocking filter.

[0005] The present disclosure seeks to provide a method and apparatus for determining the boundary strength of a deblocking filter.

[0006] The present disclosure seeks to provide a method and apparatus for deriving variables for determining a deblocking filter. [Means for solving the problem]

[0007] The video decoding method and apparatus according to the present disclosure may determine a filter length of a deblocking filter to be applied to a block boundary between a first block and a second block in a current picture, determine a boundary strength of the deblocking filter, determine the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and apply the deblocking filter to the block boundary between the first block and the second block. Here, the filter length of the deblocking filter may be determined based on whether multiple reference prediction is applied to at least one of the first block or the second block.

[0008] In the video decoding method and apparatus according to the present disclosure, the filter length of the deblocking filter may be determined based on at least one of the size of the second block or whether the second block is a block coded in a sub-block-based inter prediction mode.

[0009] In the video decoding method and apparatus according to the present disclosure, the boundary strength may be determined based on whether the number of multiple reference blocks for the first block is the same as the number of multiple reference blocks for the second block.

[0010] In the video decoding method and apparatus according to the present disclosure, the boundary strength may be determined based on weights for the multi-reference prediction.

[0011] In the video decoding method and apparatus according to the present disclosure, when the multiple reference prediction is applied to the second block, the boundary strength may be determined based on whether a picture to which a normal reference block of the second block belongs is the same as a picture to which an additional reference block of the second block belongs.

[0012] In the video decoding method and apparatus according to the present disclosure, when the multiple reference prediction is applied to the second block, the boundary strength may be determined based on whether a picture to which an additional reference block of the second block belongs is the same as a picture to which a normal reference block of the first block belongs.

[0013] In the video decoding method and apparatus according to the present disclosure, when the multiple reference prediction is applied to the second block, the boundary strength may be determined based on a difference between a motion vector of the second block with respect to an additional reference block and a motion vector of the first block with respect to a normal reference block.

[0014] In the video decoding method and apparatus according to the present disclosure, when the multiple reference prediction is applied to the second block, the boundary strength may be determined based on a difference between a motion vector of the second block with respect to a normal reference block and a motion vector of the first block with respect to an additional reference block.

[0015] In the video decoding method and apparatus according to the present disclosure, the predefined maximum value for the boundary strength may be three or four.

[0016] The video encoding method and apparatus according to the present disclosure may determine a filter length of a deblocking filter to be applied to a block boundary between a first block and a second block in a current picture, determine a boundary strength of the deblocking filter, determine the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and apply the deblocking filter to the block boundary between the first block and the second block. Here, the filter length of the deblocking filter may be determined based on whether multiple reference prediction is applied to at least one of the first block or the second block.

[0017] A computer-readable digital storage medium is provided having encoded video / image information stored thereon that enables a video decoding method to be performed by a decoding device according to the present disclosure.

[0018] A computer-readable digital storage medium is provided having stored thereon video / image information generated by the video encoding method of the present disclosure.

[0019] A method and apparatus for transmitting video / image information generated by a video encoding method according to the present disclosure is provided. [Effects of the Invention]

[0020] According to the present disclosure, the filter length of the deblocking filter can be adaptively determined by taking into consideration whether or not multiple reference prediction is applied and / or whether or not overlapped block motion compensation is applied.

[0021] According to the present disclosure, by taking into consideration whether or not multiple reference prediction is applied and / or whether or not overlapping block motion compensation is applied, the boundary strength of the deblocking filter can be determined more accurately, and based on this, the smoothing effect at block boundaries can be improved.

[0022] According to the present disclosure, by defining an extended range for the boundary strength of the deblocking filter, the deblocking filter can be applied more finely.

[0023] According to the present disclosure, by selectively utilizing a variable derivation method with an extended range, variables for determining a deblocking filter can be effectively derived. [Brief explanation of the drawings]

[0024] [Figure 1] 1 illustrates a video / image coding system according to the present disclosure. [Figure 2]1 is a schematic block diagram of an encoding device to which an embodiment of the present disclosure can be applied, in which video / image signals are encoded. [Figure 3] 1 is a schematic block diagram of a decoding device to which an embodiment of the present disclosure can be applied, in which video / image signals are decoded. [Figure 4] 1 is a diagram illustrating a video decoding method performed by a decoding device (300) according to an embodiment of the present disclosure. [Figure 5] 1 is a diagram showing a schematic configuration of a decoding device (300) that performs a video decoding method according to the present disclosure. [Figure 6] 1 is a diagram illustrating a video encoding method performed by an encoding device (200) according to an embodiment of the present disclosure. [Figure 7] 1 is a diagram showing a schematic configuration of an encoding device (200) that performs a video encoding method according to the present disclosure. [Figure 8] FIG. 1 illustrates an example of a content streaming system to which embodiments of the present disclosure can be applied. DETAILED DESCRIPTION OF THE INVENTION

[0025] While the present disclosure may be modified in various ways and may have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present disclosure to the specific embodiments, and it should be understood that the present disclosure includes all modifications, equivalents, and alternatives within the spirit and technical scope of the present disclosure. In the description of each figure, similar reference numerals are used to refer to similar components.

[0026] Terms such as "first," "second," etc. may be used to describe various components, but these components should not be limited by such terms. These terms are used merely to distinguish one component from another. For example, a first component could be termed a second component, and similarly, a second component could be termed a first component, without departing from the scope of this disclosure. The term "and / or" includes a combination of multiple associated listed items or any item of multiple associated listed items.

[0027] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, and that there may be additional components in between. On the other hand, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no additional components in between.

[0028] The terms used in this application are merely for the purpose of describing particular embodiments and are not intended to limit the present disclosure. The singular terms also include the plural terms unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0029] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in the versatile video coding (VVC) standard. The methods / embodiments disclosed herein may also be applied to methods disclosed in the essential video coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio video coding standard (AVS2), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0030] This specification presents various embodiments relating to video / image coding, and unless otherwise stated, the above embodiments may be performed in combination with each other.

[0031] In this specification, video may refer to a collection of a series of images over time. A picture generally refers to a unit representing an image at a specific time period, and a slice / tile is a unit constituting part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may consist of one or more slices / tiles. A tile is a rectangular area consisting of multiple CTUs in a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​CTUs with a height equal to the height of the picture and a width specified by a syntax requirement of the picture parameter set. A tile row is a rectangular area of ​​CTUs with a height specified by the picture parameter set and a width equal to the width of the picture. CTUs within a tile are arranged consecutively by CTU raster scanning, while tiles within a picture may be arranged consecutively by tile raster scanning. A slice may contain an integer number of complete tiles or an integer number of consecutive complete CTU rows within the tiles of a picture that may be contained exclusively in a single NAL unit. A picture may, on the other hand, be partitioned into two or more sub-pictures. A sub-picture may be a rectangular region of one or more slices in a picture.

[0032] A picture element, pixel, or pel can refer to the smallest unit that makes up a picture (or an image). A "sample" can also be used as a term corresponding to a pixel. A sample can generally indicate a pixel or a pixel value, and may indicate only a pixel / pixel value of a luminance (luma) component, or may indicate only a pixel / pixel value of a chrominance (chroma) component.

[0033] A unit may refer to a basic unit of video processing. A unit may include at least one of a specific region of a picture and information related to that region. One unit may include one luma block and two chroma (e.g., cb, cr) blocks. The term unit may sometimes be used interchangeably with terms such as block or area. In general, an MxN block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0034] As used herein, "A or B" can mean "A only," "B only," or "both A and B." In other words, as used herein, "A or B" can be interpreted as "A and / or B." For example, as used herein, "A, B, or C" can mean "A only," "B only," "C only," or "any combination of A, B, and C."

[0035] As used herein, a slash ( / ) or a comma can mean "and / or." For example, "A / B" can mean "A and / or B." Thus, "A / B" can mean "A only," "B only," or "both A and B." For example, "A, B, C" can mean "A, B, or C."

[0036] As used herein, "at least one of A and B" can mean "A only," "B only," or "both A and B." Furthermore, as used herein, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted as being the same as "at least one of A and B."

[0037] Furthermore, in this specification, "at least one of A, B and C" can mean "A only," "B only," "C only," or "any combination of A, B and C." Furthermore, "at least one of A, B or C" or "at least one of A, B and / or C" can mean "at least one of A, B and C."

[0038] Furthermore, parentheses used in this specification may mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" may be suggested as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra prediction," and "intra prediction" may be suggested as an example of "prediction." Furthermore, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" may be suggested as an example of "prediction."

[0039] In this specification, technical features individually described in the same drawing may be embodied individually or simultaneously.

[0040] FIG. 1 is a diagram illustrating a video / image coding system according to this disclosure.

[0041] Referring to FIG. 1, a video / image coding system may include a first device (a source device) and a second device (a receiving device).

[0042] A source device can transmit encoded video / image information or data to a receiving device via a digital storage medium or a network in the form of a file or streaming. The source device may include a video source, an encoding device, and a transmitting unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, which may be a separate device or an external component.

[0043] A video source can acquire video / images through a video / image capture, synthesis, or generation process. A video source can include a video / image capture device and / or a video / image generation device. A video / image capture device can include one or more cameras, a video / image archive containing previously captured video / images, etc. A video / image generation device can include a computer, tablet, smartphone, etc., and can (electronically) generate video / images. For example, a virtual video / image can be generated through a computer, etc., in which case the video / image capture process can be replaced by a process in which the associated data is generated.

[0044] An encoding device may encode input video / images. The encoding device may perform a series of steps such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.

[0045] The transmitting unit can transmit the encoded video / image information or data output in the form of a bitstream to a receiving unit of a receiving device via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating a media file according to a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0046] The decoding device can decode the video / image by performing a series of steps such as inverse quantization, inverse transform, and prediction, which correspond to the operations of the encoding device.

[0047] The renderer can render the decoded video / image, and the rendered video / image can be displayed on a display unit.

[0048] FIG. 2 is a schematic block diagram of an encoding device to which the embodiments of the present disclosure can be applied, in which video / image signals are encoded.

[0049] Referring to FIG. 2, the encoding apparatus 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-prediction unit 221 and an intra-prediction unit 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. The above-described image divider 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filterer 260 may be configured by one or more hardware components (e.g., an encoding device chipset or processor) depending on the embodiment. In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0050] The image division unit 210 may divide an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, the coding units may be recursively divided into coding tree units (CTUs) or largest coding units (LCUs) according to a QTBTTT (Quad-tree, Binary-tree, Ternary-tree) structure.

[0051] For example, one coding unit may be divided into multiple coding units having deeper depths based on a quadtree structure, a binary tree structure, and / or a tertiary structure. In this case, for example, the quadtree structure may be applied first, and then the binary tree structure and / or the tertiary structure may be applied later. Alternatively, the binary tree structure may be applied before the quadtree structure. The coding procedure according to the present specification may be performed based on a final coding unit that is not further divided. In this case, based on coding efficiency according to video characteristics, the largest coding unit may be immediately used as the final coding unit, or, if necessary, the coding unit may be recursively divided into coding units of lower depths, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later.

[0052] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the final coding unit. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0053] The term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luma component, or may represent only a pixel / pixel value of a chroma component. A sample may be used in terms corresponding to one picture (or image), pixel, or pel.

[0054] The encoding apparatus 200 may subtract a prediction signal (prediction block, prediction sample array) output from the inter prediction unit 221 or the intra prediction unit 222 from an input video signal (original block, original sample array) to generate a residual signal (residual block, residual sample array), and the generated residual signal is transmitted to the conversion unit 232. In this case, a unit in the encoding apparatus 200 that subtracts the prediction signal (prediction block, prediction sample array) from the input video signal (original block, original sample array) may be referred to as a subtraction unit 231.

[0055] The prediction unit 220 may perform prediction on a current block (hereinafter referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit 220 may determine whether intra prediction or inter prediction is applied to the current block or CU. The prediction unit 220 may generate various information related to prediction, such as prediction mode information, as will be described later in the description of each prediction mode, and transmit the information related to prediction to the entropy encoding unit 240. The entropy encoding unit 240 may encode the information related to prediction and output it in the form of a bitstream.

[0056] The intra prediction unit 222 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or at a certain distance from the current block depending on the prediction mode. In intra prediction, prediction modes may include one or more non-directional modes and multiple directional modes. The non-directional modes may include at least one of DC mode and planar mode. The directional modes may include 33 directional modes or 65 directional modes depending on the granularity of the prediction direction. However, this is merely an example, and more or less directional modes may be used depending on the setting. The intra prediction unit 222 may also determine the prediction mode to be applied to the current block using the prediction modes applied to neighboring blocks.

[0057] The inter prediction unit 221 may derive a prediction block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. To reduce the amount of motion information transmitted in inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction information (e.g., L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, the inter prediction unit 221 may construct a motion information candidate list based on neighboring blocks and generate information indicating which candidates are used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter prediction unit 221 may use motion information of neighboring blocks as motion information of the current block. In the case of skip mode, unlike in the merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the current block may be indicated by using the motion vector of a neighboring block as a motion vector predictor and signaling a motion vector difference.

[0058] The prediction unit 220 may generate a prediction signal based on various prediction methods, which will be described later. For example, the prediction unit may apply intra prediction or inter prediction for predicting a block, or may simultaneously apply intra prediction and inter prediction. This may be referred to as a combined inter and intra prediction (CIIP) mode. The prediction unit may also use an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for coding content images / videos, such as games, such as screen content coding (SCC). IBC basically performs prediction within a current picture, but may be similar to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described herein. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, intra-picture sample values ​​may be signaled based on information about a palette table and a palette index. The predicted signal generated by the prediction unit 220 may be used to generate a reconstructed signal or a residual signal.

[0059] The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, GBT refers to a transform obtained from a graph representing inter-pixel relationship information. CNT refers to a transform obtained based on a predicted signal generated using all previously reconstructed pixels. The transform process may be applied to square pixel blocks of the same size, or to non-square blocks of variable sizes.

[0060] The quantization unit 233 quantizes the transform coefficients and transmits the quantized signal to the entropy encoding unit 240. The entropy encoding unit 240 encodes the quantized signal (information about the quantized transform coefficients) and outputs it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantization unit 233 rearranges the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and generates information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0061] The entropy encoding unit 240 can perform various encoding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoding unit 240 can encode information necessary for video / image restoration (e.g., values ​​of syntax elements) together with or separately from the quantized transform coefficients.

[0062] Encoded information (e.g., encoded video / video information) may be transmitted or stored in the form of a bitstream in network abstraction layer (NAL) units. The video / video information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / video information may also include general constraint information. In this specification, information and / or syntax elements transmitted / signaled from an encoding device to a decoding device may be included in the video / video information. The video / video information may be encoded using the encoding procedure described above and included in the bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media, such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 may be transmitted to a transmitting unit (not shown) and / or stored to a storing unit (not shown) configured as an internal / external element of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0063] The quantized transform coefficients output from the quantization unit 233 may be used to generate a prediction signal. For example, the inverse quantization unit 234 and the inverse transform unit 235 may apply inverse quantization and inverse transform to the quantized transform coefficients to reconstruct a residual signal (residual block or residual sample). The adder 250 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to a prediction signal output from the inter prediction unit 221 or the intra prediction unit 222. When there is no residual for the current block, such as when a skip mode is applied, a predicted block may be used as a reconstructed block. The adder 250 may be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, or may be used for inter prediction of the next picture after filtering, as described below. Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during picture encoding and / or reconstruction.

[0064] The filtering unit 260 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 260 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture and store the modified reconstructed picture in the memory 270, specifically, in the DPB of the memory 270. The various filtering methods may include deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 260 may generate various information related to filtering and transmit it to the entropy encoding unit 240. The information related to filtering may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[0065] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. This allows the encoding apparatus to avoid prediction mismatch between the encoding apparatus 200 and the decoding apparatus when inter prediction is applied, and also improves coding efficiency.

[0066] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter predictor 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter predictor 221 to be used as motion information of a spatially neighboring block or a temporally neighboring block. The memory 270 may store reconstructed samples of reconstructed blocks in the current picture and transmit these to the intra predictor 222.

[0067] FIG. 3 is a schematic block diagram of a decoding device to which the embodiments of the present disclosure can be applied, in which video / image signals are decoded.

[0068] 3, the decoding device 300 may include an entropy decoding unit (entropy decoder 310), a residual processor (residual processor 320), a predictor (predictor 330), an adder (adder 340), a filter (filter 350), and a memory (memory 360). The predictor 330 may include an inter predictor 331 and an intra predictor 332. The residual processor 320 may include a dequantizer (dequantizer 321) and an inverse transformer (inverse transformer 322).

[0069] The entropy decoding unit 310, residual processing unit 320, prediction unit 330, addition unit 340, and filtering unit 350 may be configured as a single hardware component (e.g., a decoding device chipset or processor) depending on the embodiment. Also, the memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0070] When a bitstream including video / image information is input, the decoding apparatus 300 can reconstruct an image corresponding to the process by which the video / image information was processed by the encoding apparatus of FIG. 2. For example, the decoding apparatus 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding apparatus 300 can perform decoding using a processing unit applied by the encoding apparatus. Accordingly, the processing unit for decoding may be a coding unit, which may be divided from a coding tree unit or a maximum coding unit according to a quad tree structure, a binary tree structure, and / or a tertiary tree structure. One or more transform units may be derived from the coding unit. The reconstructed image signal decoded and output by the decoding apparatus 300 may be played back by a playback device.

[0071] The decoding apparatus 300 may receive a signal output from the encoding apparatus of FIG. 2 in the form of a bitstream, and the received signal may be decoded by the entropy decoding unit 310. For example, the entropy decoding unit 310 may parse the bitstream and derive information (e.g., video / video information) necessary for video restoration (or picture restoration). The video / video information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / video information may also include general constraint information. The decoding apparatus may decode pictures further based on the information on the parameter sets and / or the general constraint information. Signal / received information and / or syntax elements described later in this specification may be decoded by the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 may decode information in a bitstream based on a coding method such as Exponential-Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using information on the syntax element to be decoded, decoding information on neighboring and current blocks, or information on symbols / bins decoded in previous steps, predicts the occurrence probability of the bins based on the determined context model, and generates symbols corresponding to the values ​​of each syntax element by performing arithmetic decoding of the bins. In this case, after determining the context model, the CABAC entropy decoding method may update the context model using information on the decoded symbols / bins for the context model of the next symbol / bin.Information related to prediction among the information decoded by the entropy decoding unit 310 may be provided to a prediction unit (inter prediction unit 332 and intra prediction unit 331), and residual values ​​entropy decoded by the entropy decoding unit 310, i.e., quantized transform coefficients and related parameter information, may be input to a residual processing unit 320. The residual processing unit 320 may derive a residual signal (residual block, residual sample, residual sample array). In addition, information related to filtering among the information decoded by the entropy decoding unit 310 may be provided to a filtering unit 350. Meanwhile, a receiving unit (not shown) that receives a signal output from the encoding apparatus may be further configured as an internal / external element of the decoding apparatus 300, or the receiving unit may be a component of the entropy decoding unit 310.

[0072] Meanwhile, the decoding apparatus according to the present specification may be referred to as a video / image / picture decoding apparatus, and the decoding apparatus may be divided into an information decoding apparatus (video / image / picture information decoding apparatus) and a sample decoding apparatus (video / image / picture sample decoding apparatus). The information decoding apparatus may include the entropy decoding unit 310, and the sample decoding apparatus may include at least one of the inverse quantization unit 321, the inverse transform unit 322, the adder 340, the filtering unit 350, the memory 360, the inter prediction unit 332, and the intra prediction unit 331.

[0073] The inverse quantization unit 321 may inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 321 may rearrange the quantized transform coefficients in a two-dimensional block format. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding apparatus. The inverse quantization unit 321 may inverse quantize the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0074] The inverse transform unit 322 performs inverse transform on the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0075] The prediction unit 320 may perform prediction on a current block and generate a predicted block including prediction samples for the current block. The prediction unit 320 may determine whether intra prediction or inter prediction is applied to the current block based on the prediction information output from the entropy decoding unit 310, and may determine a specific intra / inter prediction mode.

[0076] The prediction unit 320 may generate a prediction signal based on various prediction methods, which will be described later. For example, the prediction unit 320 may apply intra prediction or inter prediction for predicting a block, or may simultaneously apply intra prediction and inter prediction. This may be referred to as a combined inter and intra prediction (CIIP) mode. The prediction unit may also use an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / movie coding, such as screen content coding (SCC), for games. IBC basically performs prediction within a current picture, but may be similar to inter prediction in that a reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction methods described herein. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, information about a palette table and a palette index may be included in the video / picture information and signaled.

[0077] The intra prediction unit 331 may predict a current block by referring to samples in a current picture. The referenced samples may be located in the neighborhood of the current block or at a certain distance from the current block depending on the prediction mode. In intra prediction, prediction modes may include one or more non-directional modes and multiple directional modes. The intra prediction unit 331 may determine a prediction mode to be applied to the current block using prediction modes applied to neighboring blocks.

[0078] The inter predictor 332 may derive a prediction block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. To reduce the amount of motion information transmitted in inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction information (e.g., L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter predictor 332 may construct a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on received candidate selection information. Inter prediction may be performed based on various prediction modes, and the prediction information may include information indicating the inter prediction mode for the current block.

[0079] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to a prediction signal (prediction block, prediction sample array) output from a prediction unit (including the inter prediction unit 332 and / or the intra prediction unit 331). When there is no residual for the current block, such as when the skip mode is applied, the prediction block may be used as the reconstructed block.

[0080] The adder 340 may be referred to as a reconstruction unit or a reconstruction block generator. The generated reconstruction signal may be used for intra prediction of a next block to be processed in a current picture, or may be output after filtering as described below, or may be used for inter prediction of a next picture. Meanwhile, luma mapping with chroma scaling (LMCS) may be applied during picture decoding.

[0081] The filtering unit 350 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 350 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture and transmit the modified reconstructed picture to the memory 360, specifically, to the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.

[0082] The (modified) reconstructed picture stored in the DPB of the memory 360 may be used as a reference picture in the inter predictor 332. The memory 360 may store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information may be transmitted to the inter predictor 260 to be used as motion information of a spatially neighboring block or a temporally neighboring block. The memory 360 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra predictor 331.

[0083] In this specification, the embodiments described for the filtering unit 260, inter prediction unit 221, and intra prediction unit 222 of the encoding device 200 may also be applied identically or correspondingly to the filtering unit 350, inter prediction unit 332, and intra prediction unit 331 of the decoding device 300, respectively.

[0084] FIG. 4 illustrates a video decoding method performed by a decoding device according to the present disclosure.

[0085] Referring to FIG. 4, the filter length of the deblocking filter can be determined (S400).

[0086] The deblocking filter may be applied to a boundary between adjacent blocks (i.e., a block boundary) based on a vertical or horizontal boundary within the current picture. Here, a block may refer to a coding unit (CU) or a transform unit (TU). Furthermore, one coding unit or transform unit may be composed of one or more sub-blocks, and the block may refer to sub-blocks.

[0087] Hereinafter, for convenience of explanation, the blocks will be referred to as a first block and a second block. Also, in this disclosure, the first block and the second block may be referred to as a P block and a Q block, respectively. When the boundary between the blocks is a vertical boundary, the first block may refer to the block at the top of the vertical boundary, and the second block may refer to the block at the bottom of the vertical boundary. When the boundary between the blocks is a horizontal boundary, the first block may refer to the block to the left of the horizontal boundary, and the second block may refer to the block to the right of the horizontal boundary.

[0088] The process of determining the filter length of the deblocking filter may determine at least one of whether the boundary corresponds to a coding unit boundary (CU boundary), whether the boundary corresponds to a transform unit boundary (TU boundary), or whether the boundary corresponds to a subblock boundary, and then determine the filter length according to the block. The filter length may be divided into a filter length of a deblocking filter applied to a first block (hereinafter referred to as P length) and a filter length of a deblocking filter applied to a second block (hereinafter referred to as Q length), and these may be determined separately.

[0089] Specifically, whether a block corresponds to a CU boundary may be determined by checking whether the block corresponds to a picture boundary, a subpicture boundary, a slice boundary, a tile boundary, or a virtual boundary within a picture. Whether a block corresponds to a TU boundary may be determined by moving horizontally and vertically in N-pixel increments within a coding unit. For example, for a luma component, whether a block corresponds to a boundary between transform units may be determined by moving in 4-pixel increments, and for a chroma component, whether a block corresponds to a boundary between transform units may be determined by moving in 8-pixel increments. Furthermore, whether a block corresponds to a subblock boundary may be determined for the luma component. When a subblock-based inter prediction mode (e.g., subblock merge mode, inter affine mode, etc.) is applied to a block, the block may include one or more subblocks. In this case, whether a block corresponds to a subblock boundary may be checked / determined by moving in 8-pixel increments within the block. In other words, a deblocking filter may not be applied to 4-pixel subblock boundaries. A method for determining the filter length of a deblocking filter will now be described.

[0090] Example 1-A

[0091] A filter length at a block boundary may be determined based on the size of the block, and the filter length may be determined for each of a first block and a second block adjacent to each other on the block boundary.

[0092] Specifically, when the size of the second block is greater than or equal to the first threshold size, the filter length (Q length) of the deblocking filter applied to the second block may be determined to be 7. When the size of the second block is smaller than the first threshold size and greater than or equal to the second threshold size, Q length may be determined to be 3. When the size of the second block is smaller than the second threshold size, Q length may be determined to be 1. As an example, the first threshold size may be 32, and the second threshold size may be 4.

[0093] The predetermined filter length may be updated based on at least one of whether the block is a block coded in a sub-block-based inter-prediction mode, the position of the sub-block, or the distance from the block boundary (CU / TU boundary).

[0094] Specifically, if the second block is a block coded in a sub-block-based inter prediction mode (or if the second block includes multiple sub-blocks), the Q length for the second block may be updated to the minimum value of a predetermined Q length and 5. Otherwise, the Q length for the second block may not be updated.

[0095] While moving within the second block in 8-pixel increments, it may be checked whether the corresponding position corresponds to a subblock boundary. In this case, it may be checked whether the corresponding position (pos) corresponds to a TU boundary. If the pos corresponds to a TU boundary, the Q length for the second block may be updated to the minimum of a predetermined Q length or 5. If the pos does not correspond to a TU boundary, it may be checked whether a position moved 4 pixels from the pos corresponds to a TU boundary. If the position moved 4 pixels corresponds to a TU boundary, the Q length for the second block may be updated to 1. If the position moved 4 pixels does not correspond to a TU boundary, the Q length for the second block may be updated based on at least one of whether a position moved 8 pixels from the pos corresponds to a TU boundary or the position of the subblock to which the pos belongs. If a position moved by 8 pixels from pos corresponds to a TU boundary or the sub-block to which pos belongs corresponds to the first sub-block (particularly, the right boundary of the first sub-block) or the last sub-block (particularly, the left boundary of the last sub-block) in the block, the Q length for the second block may be updated to 2. Otherwise (i.e., if a position moved by 8 pixels from pos does not correspond to a TU boundary or the sub-block to which pos belongs does not correspond to the first or last sub-block in the block), the Q length for the second block may be updated to 3.

[0096] The filter length for the first block may be determined or updated in the same manner as for the second block described above, and a duplicated description will be omitted.

[0097] Example 1-B

[0098] The filter length at the block boundary may be determined based on whether the block is a block to which multiple reference prediction is applied.

[0099] The multiple reference prediction according to the present disclosure may derive a final predicted block using a weighted sum of multiple reference blocks. Here, the multiple reference blocks may include a reference block for unidirectional prediction or bidirectional prediction in inter mode and one or more additional reference blocks for multiple reference prediction. For example, the additional reference block may be an inter-predicted block derived based on the inter mode or an intra-predicted block derived based on the intra mode. The multiple reference prediction according to the present disclosure is not limited to being applied only to blocks coded in inter mode. Multiple reference prediction may also be applied to blocks coded in intra mode or intra block copy (IBC) mode, in which case an inter-predicted block and / or an intra-predicted block may be used as the additional reference block.

[0100] A deblocking filter can remove blocking artifacts caused by discontinuities between blocks by filtering block boundaries. As described above, the filter length may be adjusted based on at least one of the block size, whether the block is coded in a sub-block-based inter prediction mode, the position of the sub-block, or the distance from the block boundary. However, since a block to which multiple reference prediction is applied has an additional reference block in addition to a maximum of two reference blocks for unidirectional or bidirectional prediction, samples within the block may be considered to be sufficiently smoothed. Therefore, the filter length of the deblocking filter may differ depending on whether multiple reference prediction is applied.

[0101] Specifically, it may be possible to check whether the second block is a block to which multiple reference prediction is applied. If the second block is a block to which multiple reference prediction is applied, it may be possible to check whether the size of the second block is smaller than a first threshold size. Here, regardless of whether the first block is a block to which multiple reference prediction is applied, if the second block is a block to which multiple reference prediction is applied, it may be possible to check whether the size of the second block is greater than or equal to the first threshold size. Alternatively, if both the first block and the second block are blocks to which multiple reference prediction is applied, it may be possible to limit the check to whether the size of the second block is greater than or equal to the first threshold size. If the second block is not a block to which multiple reference prediction is applied, the filter length may be determined according to the above-described embodiment 1-A.

[0102] If the size of the second block is greater than or equal to the first threshold size, the Q length for the second block may be determined to be 3. If the size of the second block is smaller than the first threshold size, the Q length for the second block may be determined to be 1. As an example, the first threshold size may be 32.

[0103] The predetermined filter length may be updated based on whether the second block is a block coded in a sub-block-based inter prediction mode. If the second block is a block coded in a sub-block-based inter prediction mode (or if the second block includes multiple sub-blocks), the Q length for the second block may be updated to 1. Otherwise, the Q length for the second block may not be updated.

[0104] In the same manner as for the second block described above, P length for the first block may be determined or updated depending on whether the first block is a block to which multiple reference prediction is applied.

[0105] In this way, a block to which multiple reference prediction is applied may have a shorter filter length than a block to which multiple reference prediction is not applied. If the first or second block has a sub-block-based reference block and is a block to which multiple reference prediction is applied, the filter length may be set to be shorter since discontinuity at the sub-block boundary is mitigated. Alternatively, if the first or second block has a sub-block-based reference block and is a block to which multiple reference prediction is applied, a deblocking filter may not be applied to the sub-block boundary.

[0106] Example 1-C

[0107] The filter length at the block boundary may be determined based on whether or not the block is a block to which overlapped block motion compensation (OBMC) is applied.

[0108] Specifically, it may be possible to check whether the second block is a block to which OBMC is applied. If the second block is a block to which OBMC is applied, it may be possible to check whether the size of the second block is smaller than a first threshold size. In this case, regardless of whether the first block is a block to which OBMC is applied, if the second block is a block to which OBMC is applied, it may be possible to check whether the size of the second block is greater than or equal to the first threshold size. Alternatively, if both the first block and the second block are blocks to which OBMC is applied, it may be possible to limit the check to whether the size of the second block is greater than or equal to the first threshold size. If the second block is not a block to which OBMC is applied, the filter length may be determined according to the above-described Example 1-A.

[0109] If the size of the second block is greater than or equal to the first threshold size, the Q length for the second block may be determined to be 3. If the size of the second block is smaller than the first threshold size, the Q length for the second block may be determined to be 1. As an example, the first threshold size may be 32.

[0110] The predetermined filter length may be updated based on whether the second block is a block coded in a sub-block-based inter prediction mode. If the second block is a block coded in a sub-block-based inter prediction mode (or if the second block includes multiple sub-blocks), the Q length for the second block may be updated to 1. Otherwise, the Q length for the second block may not be updated.

[0111] In the same manner as for the second block described above, the P length for the first block may be determined or updated based on whether the first block is a block to which OBMC is applied.

[0112] In this way, a block to which OBMC is applied may have a shorter filter length than a block to which OBMC is not applied. If the first or second block has a subblock-based reference block and is a block to which OBMC is applied, the filter length may be set to be shorter since discontinuity at the subblock boundary is mitigated. Alternatively, if the first or second block has a subblock-based reference block and is a block to which OBMC is applied, a deblocking filter may not be applied to the subblock boundary. The filter length for the deblocking filter may be determined / updated based on a comparison between the filter length for OBMC and the filter length for the deblocking filter. If the filter length for OBMC is smaller than the filter length for the deblocking filter, the deblocking filter may be applied.

[0113] Referring to FIG. 4, the boundary strength (BS) of the deblocking filter can be determined (S410).

[0114] The BS may be determined by taking into consideration the characteristics of the first and second blocks adjacent to each other based on the block boundary (CU / TU / subblock boundary). The block boundary may be searched for by moving in N-pixel units. For example, the block boundary may be searched for by moving in 4-pixel units for the luma component, and by moving in 8-pixel units for the chroma component.

[0115] The BS may be determined based on at least one of the prediction mode of the first block and / or the second block and the presence or absence of non-zero transform coefficients. Furthermore, if the prediction mode of the first block and / or the second block is the inter mode or the IBC mode, the BS may be determined based on motion information of the corresponding block. In this case, a block vector (BV) may be used as the motion information for the IBC mode, and at least one of the number of motion vectors, a reference picture, or a motion vector may be used as the motion information for the inter mode. A method for determining the BS of the deblocking filter will be described in detail below.

[0116] Example 2-A

[0117] It can be checked whether the P block and / or Q block are blocks to which BDPCM (block difference pulse code modulation) is applied. If both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined to be 0. If not (i.e., if at least one of the P block or the Q block is a block to which BDPCM is not applied), it can be checked whether the P block and / or Q block are blocks coded in intra mode or combined inter-intra prediction mode (CIIP mode).

[0118] If the P block and / or the Q block is a block coded in intra mode or CIIP mode, BS may be determined to be 2. Otherwise (i.e., if neither the P block nor the Q block is a block coded in intra mode or CIIP mode), at least one of whether the block edge corresponds to a transform unit edge or whether at least one of the P block or the Q block has non-zero transform coefficients can be checked.

[0119] If the block edge corresponds to a transform unit edge and at least one of the P block and the Q block has non-zero transform coefficients, BS may be determined to be 1. Otherwise (i.e., if the block edge does not correspond to a transform unit edge or if neither the P block nor the Q block has non-zero transform coefficients), it can be checked whether at least one of the following conditions 1 to 4 is met.

[0120] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block, where each of the P and Q blocks has one of a plurality of predefined prediction modes, and the plurality of prediction modes may include at least one of intra mode, inter mode, IBC mode, or palette mode.

[0121] (Condition 2) Both the P block and the Q block are blocks coded in IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0122] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of motion vectors for predicting the P block and the number of motion vectors for predicting the Q block are different from each other.

[0123] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0124] If at least one of the conditions 1 to 4 is satisfied, BS may be determined to be 1. If none of the conditions 1 to 4 is satisfied, BS may be determined to be 0.

[0125] The method for determining BS depending on whether conditions 2 to 4 are satisfied will be described in more detail below. When the right sample is q and the left sample is p relative to the vertical edge (or when the bottom sample is q and the top sample is p relative to the horizontal edge), the block containing sample q is called the Q block, and the block containing sample p is called the P block. The motion vectors of the Q block and the P block can be expressed as (mvQX_x, mvQX_y) and (mvPX_x, mvPX_y), respectively, where X represents List and may be replaced with 0 or 1.

[0126] Conditions 2 to 4 may be applied when both the Q block and the P block are blocks coded in IBC mode or blocks coded in inter mode. When both the Q block and the P block are blocks coded in IBC mode or a slice including the Q block and / or the P block is a P slice, BS may be determined as follows. That is, when a reference picture for predicting the Q block and a reference picture for predicting the P block are different from each other, BS may be determined to be 1. When a reference picture for predicting the Q block and a reference picture for predicting the P block are the same, BS may be determined based on the difference between the block vectors or motion vectors of the Q block and the P block. When the difference between the block vectors or motion vectors is greater than or equal to a threshold (nTh), BS may be determined to be 1; otherwise, BS may be determined to be 0. A method of determining BS based on the difference between the motion vectors of both blocks may be expressed as in Equation 1 below.

[0127] [Formula 1]

number

[0128] In Equation 1, mvQ0_x and mvQ0_y may represent the x and y components of the L0 motion vector of the Q block, respectively. mvP0_x and mvP0_y may represent the x and y components of the L0 motion vector of the P block, respectively. nTh may be a threshold for determining BS. If either the difference between mvQ0_x and mvP0_x or the difference between mvQ0_y and mvP0_y is greater than or equal to nTh, BS may be determined to be 1; otherwise, BS may be determined to be 0. That is, if the difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold, BS may be determined to be 1; otherwise, BS may be determined to be 0. Here, the difference may refer to an absolute difference value, which may be calculated for the x and y components of the motion vector.

[0129] When neither the Q block nor the P block is coded in IBC mode and the slice including the Q block and / or the P block is a B slice, the BS may be determined as follows: That is, when the reference picture for predicting the Q block and the reference picture for predicting the P block are different from each other, the BS may be determined to be 1. When the reference picture for predicting the Q block and the reference picture for predicting the P block are the same from each other, the BS may be determined based on the difference between the motion vectors of the Q block and the P block. A method for determining the BS when the reference picture for the Q block and the reference picture for the P block are different from each other will be described below.

[0130] First, it may be determined whether the reference picture of the P block and the reference picture of the Q block are the same. For example, if the L0 reference picture of the P block and the L0 reference picture of the Q block are the same and the L1 reference picture of the P block and the L1 reference picture of the Q block are the same, the reference picture of the P block and the reference picture of the Q block may be determined to be the same. Alternatively, if the L1 reference picture of the P block and the L0 reference picture of the Q block are the same and the L0 reference picture of the P block and the L1 reference picture of the Q block are the same, the reference picture of the P block and the reference picture of the Q block may be determined to be the same.

[0131] If it is determined that the reference picture of the P block and the reference picture of the Q block are the same, it can be checked whether the L0 reference picture of the P block and the L1 reference picture are the same. If the L0 reference picture of the P block and the L1 reference picture are different from each other, the BS may be determined based on whether the L0 reference picture of the P block is the same as the L0 reference picture of the Q block. Specifically, if the L0 reference picture of the P block is the same as the L0 reference picture of the Q block, the BS may be determined as shown in Equation 2 below.

[0132] [Formula 2]

number

[0133] In Equation 2, mvQ0_x and mvQ0_y may represent the x and y components of the L0 motion vector of the Q block, respectively. mvP0_x and mvP0_y may represent the x and y components of the L0 motion vector of the P block, respectively. mvQ1_x and mvQ1_y may represent the x and y components of the L1 motion vector of the Q block, respectively. mvP1_x and mvP1_y may represent the x and y components of the L1 motion vector of the P block, respectively. nTh may be a threshold for determining BS. These terms may be interpreted interchangeably hereinafter.

[0134] According to Equation 2, if any one of the difference between mvQ0_x and mvP0_x, the difference between mvQ0_y and mvP0_y, the difference between mvQ1_x and mvP1_x, or the difference between mvQ1_y and mvP1_y is greater than or equal to nTh, BS may be determined to be 1. Otherwise (i.e., if all of these differences are less than nTh), BS may be determined to be 0.

[0135] That is, according to Equation 2, if the difference between the motion vectors in the same direction (L0 or L1) between the P block and the Q block is greater than or equal to the threshold, BS may be determined to be 1, and if not, BS may be determined to be 0. Here, the difference may mean the absolute difference value, and may be calculated for each of the x component and y component of the motion vector.

[0136] If the L0 reference picture of the P block is different from the L0 reference picture of the Q block, the BS may be determined as shown in Equation 3 below.

[0137] [Formula 3]

number

[0138] According to Equation 3, if any one of the difference between mvQ1_x and mvP0_x, the difference between mvQ1_y and mvP0_y, the difference between mvQ0_x and mvP1_x, or the difference between mvQ0_y and mvP1_y is greater than or equal to nTh, BS may be determined to be 1. Otherwise (i.e., if all of these differences are less than nTh), BS may be determined to be 0.

[0139] That is, according to Equation 3, if the difference between the motion vectors in different directions between the P block and the Q block is greater than or equal to the threshold, BS may be determined to be 1, and if not, BS may be determined to be 0. Here, the difference may mean the absolute difference value, and may be calculated for each of the x component and y component of the motion vector.

[0140] When the L0 reference picture and the L1 reference picture of a P block are the same, the BS may be determined as shown in Equation 4 below.

[0141] [Formula 4]

number

[0142] According to Equation 4, if the difference between the motion vectors in the same direction between the P block and the Q block is greater than or equal to a threshold, and the difference between the motion vectors in different directions between the P block and the Q block is greater than or equal to a threshold, BS may be determined to be 1. Otherwise, BS may be determined to be 0. This is as described with reference to Equations 2 and 3, and a duplicated description will be omitted.

[0143] Example 2-B

[0144] A block to which multi-reference prediction is applied may have an additional reference block in addition to a maximum of two reference blocks for unidirectional or bidirectional prediction. Here, to distinguish the maximum two reference blocks from the additional reference blocks, the maximum two reference blocks are referred to as normal reference blocks. If the motion information of a block differs from that of a neighboring block (or if the difference in motion information between the block and the neighboring block is greater than a specific threshold), discontinuity between blocks may occur. Therefore, when determining whether to filter block boundaries for a block to which multi-reference prediction is applied, discontinuity between block boundaries may be determined by taking into account motion information identifying the additional reference block in addition to motion information identifying the normal reference block of the block.

[0145] As an example of multi-reference prediction, if a current block has motion information for bi-directional prediction, two reference blocks (P0, P1) for bi-directional prediction may be derived. Also, additional reference blocks (P2, P3) for multi-reference prediction may be derived. This is merely an example, and the number of reference blocks for multi-reference prediction may be determined within a predefined maximum number, N.

[0146] In this way, the current block has four reference blocks (P0 to P3), and the final predicted block of the current block may be generated by a weighted sum of the four reference blocks. In this case, a weight (W2) applied to the P2 reference block and a weight (W3) applied to the P3 reference block may be used. For example, the final predicted block (P) of the current block may be generated as shown in Equation 5 below.

[0147] [Formula 5]

number

[0148] Since additional reference blocks for multiple reference prediction are used in generating the final predicted block of the current block, at least one of information about additional reference blocks P2 and P3 or information about multiple reference prediction may be further considered in determining the BS of the deblocking filter in addition to information about reference blocks P0 and P1. Here, the information about the reference blocks may include at least one of a motion vector for identifying the reference block, a reference picture to which the reference block belongs, or a prediction mode for guiding the reference block. The information about multiple reference prediction may include at least one of the number of reference blocks used for multiple reference prediction, a motion vector for multiple reference prediction, a reference picture for multiple reference prediction, an index of the reference picture, or a reference picture output order (POC).

[0149] For example, the BS may be determined based on the number of reference blocks used for predicting the Q block and / or the P block. This may be applied when multiple reference prediction is applied to at least one of the Q block and the P block. However, without being limited thereto, it may also be applied when multiple reference prediction is not applied to the Q block and the P block.

[0150] Specifically, the number of reference blocks used for predicting the Q block is denoted by numPred_Q, and the number of reference blocks used for predicting the P block is denoted by numPred_P. If numPred_Q and numPred_P are not the same, BS may be determined to be 1. Otherwise, BS may be determined based on the difference between the motion vectors of the Q block and the P block. That is, if numPred_Q and numPred_P are the same, BS may be determined based on whether the difference between the motion vectors of the Q block and the P block is smaller than a threshold. This is as described in detail in Condition 4 of Example 2-A.

[0151] For example, assume that a Q block performs bidirectional prediction and has two additional reference blocks for multi-reference prediction, and a P block performs bidirectional prediction but does not perform multi-reference prediction. In this case, the number of reference blocks used for predicting the Q block (numPred_Q) is 4, and the number of reference blocks used for predicting the P block (numPred_P) is 2. Thus, if numPred_Q and numPred_P are not the same, BS may be determined to be 1.

[0152] For example, it can be checked whether the P block and / or the Q block are blocks to which BDPCM is applied. If both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined to be 0. Otherwise, it can be checked whether the P block and / or the Q block are blocks coded in intra mode or combined inter-intra prediction mode (CIIP mode).

[0153] If the P block and / or the Q block is a block coded in intra mode or CIIP mode, BS may be determined to be 2. Otherwise, it can check at least one of whether the block edge corresponds to a transform unit edge or whether at least one of the P block and the Q block has non-zero transform coefficients.

[0154] If the block edge corresponds to a transform unit edge and at least one of the P block and the Q block has a non-zero transform coefficient, BS may be determined to be 1. Otherwise, it can be checked whether at least one of the following conditions 1 to 4 is met.

[0155] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0156] (Condition 2) Both the P block and the Q block are blocks coded in IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0157] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of reference blocks for predicting the P block and the number of reference blocks for predicting the Q block are different from each other.

[0158] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0159] If at least one of the conditions 1 to 4 is satisfied, BS may be determined to be 1. If none of the conditions 1 to 4 is satisfied, BS may be determined to be 0.

[0160] For example, when multiple reference prediction is applied to a Q block and / or a P block, an additional reference block for the multiple reference prediction may exist. In this case, BS may be determined taking into consideration a prediction mode used to derive the additional reference block. Assume that multiple reference prediction is applied to a Q block coded in inter mode. In this case, if the Q block uses a prediction block derived based on an intra mode or an IBC mode as an additional reference block for the multiple reference prediction, BS may be determined to be 2. Alternatively, assume that multiple reference prediction is applied to a P block coded in intra mode. In this case, if the P block uses a prediction block derived based on an inter mode as an additional reference block for the multiple reference prediction, BS may be determined to be 2.

[0161] For example, if both the Q block and the P block are blocks to which multi-reference prediction is applied, BS may be determined to be 0. BS may be determined based on whether information regarding multi-reference prediction between the Q block and the P block is the same. If information regarding multi-reference prediction between the Q block and the P block is the same, BS may be determined to be 0. Otherwise, BS does not need to be determined to be 0. In this way, complexity can be reduced by determining BS to be 0.

[0162] When multiple reference prediction is applied to the Q block and / or the P block, the BS may be determined without comparing the motion information of the Q block and the P block (e.g., whether the motion information is the same or not). Here, the motion information refers to a motion vector, a reference picture, the number of reference blocks, etc., as described above. This is because, in the case of multiple reference prediction, additional weights (W2, W3) may be used for weighted sum with additional reference blocks, and even if the current block has the same motion vector as a neighboring block, the brightness of the predicted block relative to the current block may change depending on these weights.

[0163] Alternatively, when the additional weights (W2, W3) are smaller than a certain threshold, the difference between the predicted block before and after multi-reference prediction may not be large. In such a case, the application of multi-reference prediction does not affect the determination of the BS. Therefore, either Example 2-A or Example 2-B described above may be selected depending on the additional weights, and the BS may be determined based on the selected method. For example, when the additional weights (W2, W3) are smaller than a certain threshold, the BS may be determined based on Example 2-A. When the additional weights (W2, W3) are greater than or equal to the certain threshold, the BS may be determined based on Example 2-B. However, this is merely an example, and at least one of information related to multi-reference prediction may be further considered in addition to the additional weights. Depending on the additional weights and information related to multi-reference prediction, either Example 2-A or Example 2-B described above may be selected, and the BS may be determined based on the selected method.

[0164] The BS may be determined as any one integer ranging from 0 to k, where k may represent the maximum value of available BSs. When the additional weight is greater than or equal to a specific threshold, k may have a larger value than when multiple reference prediction is not applied, thereby expanding the range of available BSs.

[0165] Alternatively, multiple weight intervals for multi-reference prediction may be defined, and a range of available BSs (i.e., k values) may be defined for each interval. One of the multiple weight intervals may have a different k value from the other intervals. A weight interval to which the additional weight belongs may be determined, and a BS may be determined within the range of BSs corresponding to the weight interval.

[0166] Example 2-C

[0167] When the Q block and / or the P block is a block to which multiple reference prediction is applied, the block may have an additional reference block in addition to a reference block for unidirectional or bidirectional prediction (hereinafter referred to as a normal reference block). In this case, the BS can be determined by comparing motion information of the normal reference block and the additional reference block.

[0168] In this embodiment, when multiple reference prediction is applied to a Q block and / or a P block, the process of determining the boundary strength of the deblocking filter may change. The BS may be determined based on whether the reference picture to which the normal reference block belongs and the reference picture to which the additional reference block belongs are the same. The comparison between reference pictures may be applied between the normal reference block and the additional reference block of the Q block. The comparison between reference pictures may be applied between the normal reference block and the additional reference block of the P block. The comparison between reference pictures may be applied between the additional reference block of the Q block and the normal reference block of the P block. The comparison between reference pictures may be applied between the additional reference block of the P block and the normal reference block of the Q block. The comparison between reference pictures may be applied between the additional reference block of the P block and the normal reference block of the Q block.

[0169] Furthermore, the BS may be determined based on a difference between a motion vector pointing to the normal reference block and a motion vector pointing to the additional reference block. The difference in motion vectors may be calculated between the normal reference block and the additional reference block of a Q block. The difference in motion vectors may be calculated between the normal reference block and the additional reference block of a P block. The difference in motion vectors may be calculated between the additional reference block of a Q block and the normal reference block of a P block. The difference in motion vectors may be calculated between the additional reference block of a Q block and the additional reference block of a P block. The difference in motion vectors may be calculated between the additional reference block of a P block and the normal reference block of a Q block. The difference in motion vectors may be calculated between the additional reference block of a P block and the normal reference block of a Q block. The difference in motion vectors may be calculated between the additional reference block of a P block and the additional reference block of a Q block. The BS may be determined based on whether the difference in motion vectors is smaller than a predefined threshold.

[0170] Assuming that each of the Q block and the P block performs bidirectional prediction and has two additional reference blocks for multi-reference prediction, a method for determining a BS in this case will be described. For convenience of explanation, a picture to which a normal reference block belongs is referred to as a normal reference picture, and a motion vector pointing to the normal reference block is referred to as a normal motion vector. Also, a picture to which an additional reference block belongs is referred to as an additional reference picture, and a motion vector pointing to the additional reference block is referred to as an additional motion vector.

[0171] First, it may be determined whether the normal reference picture of the P block and the normal reference picture of the Q block are the same. For example, if the L0 normal reference picture of the P block and the L0 normal reference picture of the Q block are the same and the L1 normal reference picture of the P block and the L1 normal reference picture of the Q block are the same, the normal reference picture of the P block and the normal reference picture of the Q block may be determined to be the same. Alternatively, if the L1 normal reference picture of the P block and the L0 normal reference picture of the Q block are the same and the L0 normal reference picture of the P block and the L1 normal reference picture of the Q block are the same, the normal reference picture of the P block and the normal reference picture of the Q block may be determined to be the same.

[0172] When it is determined that the normal reference picture of the P block and the normal reference picture of the Q block are the same, it can be checked whether the L0 normal reference picture of the P block and the L1 normal reference picture are the same. When the L0 normal reference picture of the P block and the L1 normal reference picture are different from each other, the BS may be determined based on whether the L0 normal reference picture of the P block is the same as the L0 normal reference picture of the Q block. Specifically, when the L0 normal reference picture of the P block is the same as the L0 normal reference picture of the Q block, the BS may be determined as shown in Equation 6 below.

[0173] [Formula 6]

number

[0174] In Equation 6, mvQ0_x and mvQ0_y may represent the x and y components of the L0 normal motion vector of the Q block, respectively. mvP0_x and mvP0_y may represent the x and y components of the L0 normal motion vector of the P block, respectively. mvQ1_x and mvQ1_y may represent the x and y components of the L1 normal motion vector of the Q block, respectively. mvP1_x and mvP1_y may represent the x and y components of the L1 normal motion vector of the P block, respectively. nTh may be a threshold for determining BS. According to Equation 6, if any one of the difference between mvQ0_x and mvP0_x, the difference between mvQ0_y and mvP0_y, the difference between mvQ1_x and mvP1_x, or the difference between mvQ1_y and mvP1_y is greater than or equal to nTh, BS may be determined to be 1. Otherwise (ie, if these differences are all less than nTh), BS may be determined to be 0.

[0175] That is, according to Equation 6, if the difference between the normal motion vectors in the same direction (L0 or L1) between the P block and the Q block is greater than or equal to the threshold, BS may be determined to be 1, and if not, BS may be determined to be 0. Here, the difference may mean the absolute difference value, and may be calculated for each of the x component and y component of the motion vector.

[0176] If the L0 normal reference picture of the P block is different from the L0 normal reference picture of the Q block, BS may be determined as shown in Equation 7.

[0177] [Formula 7]

number

[0178] According to Equation 7, if any one of the difference between mvQ1_x and mvP0_x, the difference between mvQ1_y and mvP0_y, the difference between mvQ0_x and mvP1_x, or the difference between mvQ0_y and mvP1_y is greater than or equal to nTh, BS may be determined to be 1. Otherwise (i.e., if all of these differences are less than nTh), BS may be determined to be 0.

[0179] That is, according to Equation 7, if the difference between the normal motion vectors in different directions between the P block and the Q block is greater than or equal to the threshold, BS may be determined to be 1, and if not, BS may be determined to be 0. Here, the difference may mean the absolute difference value, and may be calculated for each of the x component and y component of the motion vector.

[0180] When the L0 normal reference picture and the L1 normal reference picture of a P block are the same, the BS may be determined as shown in Equation 8 below.

[0181] [Formula 8]

number

[0182] According to Equation 8, if the difference between the normal motion vectors in the same direction between the P block and the Q block is greater than or equal to a threshold, and the difference between the normal motion vectors in different directions between the P block and the Q block is greater than or equal to a threshold, BS may be determined to be 1. Otherwise, BS may be determined to be 0.

[0183] Then, at least one of a reference picture or a motion vector may be compared between the normal reference block of the Q block and the additional reference block. Also, at least one of a reference picture or a motion vector may be compared between the additional reference block of the Q block and the normal reference block of the P block. This comparison process may be repeated as many times as the number of additional reference blocks of the Q block. Accordingly, the predetermined BS may be updated.

[0184] Specifically, it can be determined whether the normal reference picture of the Q block and the additional reference picture are the same. If the L0 normal reference picture of the Q block is different from the additional reference picture of the Q block, or if the L1 normal reference picture of the Q block is different from the additional reference picture of the Q block, it may be determined that the normal reference picture of the Q block and the additional reference picture are not the same.

[0185] It can be determined whether the normal reference picture of the P block and the additional reference picture of the Q block are identical. If the L0 normal reference picture of the P block is identical to the additional reference picture of the Q block, or if the L1 normal reference picture of the P block is identical to the additional reference picture of the Q block, it may be determined that the normal reference picture of the P block and the additional reference picture of the Q block are identical to each other.

[0186] When the normal reference picture of the P block and the additional reference picture of the Q block are the same (especially when the L0 normal reference picture of the P block is the same as the additional reference picture of the Q block), the BS may be determined as shown in the following equation 9.

[0187] [Formula 9]

number

[0188] In Equation 9, mvQi_x and mvQi_y may represent the x and y components of an additional motion vector of the Q block, respectively. mvP0_x and mvP0_y may represent the x and y components of an L0 normal motion vector of the P block, respectively. nTh1 may represent a threshold for determining BS. If either the difference between mvQi_x and mvP0_x or the difference between mvQi_y and mvP0_y is greater than or equal to nTh1, BS may be determined to be 1. Otherwise (i.e., if both of these differences are less than nTh1), BS may be determined to be 0.

[0189] When the normal reference picture of the P block and the additional reference picture of the Q block are the same (especially when the L1 normal reference picture of the P block is the same as the additional reference picture of the Q block), the BS may be determined as shown in the following equation 10.

[0190] [Formula 10]

number

[0191] In Equation 10, mvQi_x and mvQi_y may represent the x and y components of an additional motion vector of the Q block, respectively. mvP1_x and mvP1_y may represent the x and y components of an L1 normal motion vector of the P block, respectively. nTh1 may represent a threshold for determining BS. If either the difference between mvQi_x and mvP1_x or the difference between mvQi_y and mvP1_y is greater than or equal to nTh1, BS may be determined to be 1. Otherwise (i.e., if both of these differences are less than nTh1), BS may be determined to be 0.

[0192] On the other hand, if it is determined that the normal reference picture of the P block and the additional reference picture of the Q block are different from each other, BS may be determined to be 1.

[0193] Then, at least one of a reference picture or a motion vector may be compared between the normal reference block of the P block and the additional reference block. Also, at least one of a reference picture or a motion vector may be compared between the additional reference block of the P block and the normal reference block of the Q block. This comparison process may be repeated as many times as the number of additional reference blocks of the Q block. Accordingly, the predetermined BS may be updated.

[0194] Specifically, it can be determined whether the normal reference picture and the additional reference picture of the P block are the same. If the L0 normal reference picture of the P block is different from the additional reference picture of the P block, or if the L1 normal reference picture of the P block is different from the additional reference picture of the P block, it may be determined that the normal reference picture and the additional reference picture of the P block are not the same.

[0195] If the normal reference picture and the additional reference picture of the P block are not identical, it can be determined whether the normal reference picture of the Q block and the additional reference picture of the P block are identical. If the L0 normal reference picture of the Q block is identical to the additional reference picture of the P block, or the L1 normal reference picture of the Q block is identical to the additional reference picture of the P block, it may be determined that the normal reference picture of the Q block and the additional reference picture of the P block are identical to each other.

[0196] When the normal reference picture of the Q block and the additional reference picture of the P block are identical to each other (especially when the L0 normal reference picture of the Q block is identical to the additional reference picture of the P block), the BS may be determined as shown in the following equation 11.

[0197] [Formula 11]

number

[0198] In Equation 11, mvQ0_x and mvQ0_y may represent the x and y components of the L0 normal motion vector of the Q block, respectively. mvPi_x and mvPi_y may represent the x and y components of the additional motion vector of the P block, respectively. nTh2 may represent a threshold for determining BS. If either the difference between mvQ0_x and mvPi_x or the difference between mvQ0_y and mvPi_y is greater than or equal to nTh2, BS may be determined to be 1. Otherwise (i.e., if both of these differences are less than nTh2), BS may be determined to be 0.

[0199] When the normal reference picture of the Q block and the additional reference picture of the P block are identical to each other (especially when the L1 normal reference picture of the Q block is identical to the additional reference picture of the P block), the BS may be determined as shown in the following equation 12.

[0200] [Formula 12]

number

[0201] In Equation 12, mvQ1_x and mvQ1_y may represent the x and y components of the L1 normal motion vector of the Q block, respectively. mvPi_x and mvPi_y may represent the x and y components of the supplemental motion vector of the P block, respectively. nTh2 may represent a threshold for determining BS. If either the difference between mvQ1_x and mvPi_x or the difference between mvQ1_y and mvPi_y is greater than or equal to nTh2, BS may be determined to be 1. Otherwise (i.e., if both of these differences are less than nTh2), BS may be determined to be 0.

[0202] On the other hand, if it is determined that the normal reference picture of the Q block and the additional reference picture of the P block are different from each other, BS may be determined to be 1.

[0203] The BS may be determined based on at least one of the above-described formulas 6 to 12, and the determined BS may be determined as the final BS.

[0204] Example 2-D

[0205] The present disclosure defines an expanded range for the BS of a deblocking filter, thereby enabling more precise application of the deblocking filter. The present disclosure can determine the BS to be any one value within the expanded range, and can subdivide the process of determining the deblocking filter for each stage. According to the present disclosure, a range of 0 to 3 can be defined for the BS of the deblocking filter, and the BS can be determined to be any one value within the range.

[0206] Specifically, it can be checked whether the P block and / or the Q block are blocks to which BDPCM is applied. If both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined to be 0. Otherwise, it can be checked whether the P block and / or the Q block are blocks coded in intra mode or combined inter-intra prediction mode (CIIP mode).

[0207] If the P block and / or the Q block is a block coded in intra mode or CIIP mode, BS may be determined to be 3. Otherwise, it can check at least one of whether the block edge corresponds to a transform unit edge or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0208] If the block edge corresponds to a transform unit edge and at least one of the P block and the Q block has a non-zero transform coefficient, the BS may be determined to be 2. Otherwise, it can be checked whether at least one of the following conditions 1 to 3 is met.

[0209] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0210] (Condition 2) Both the P block and the Q block are blocks coded in IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0211] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of motion vectors for predicting the P block and the number of motion vectors for predicting the Q block are different from each other.

[0212] If at least one of the conditions 1 to 3 is satisfied, BS may be determined to be 2. If none of the conditions 1 to 3 is satisfied, BS may be determined to be 1 or 0. If none of the conditions 1 to 3 is satisfied, it is possible to further check whether the following condition 4 is satisfied.

[0213] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0214] If the condition 4 is satisfied, the BS may be determined to be 1. If the condition 4 is not satisfied, the BS may be determined to be 0.

[0215] Example 2-E

[0216] According to the present disclosure, a range of 0 to 4 may be defined for the BS of the deblocking filter, and the BS may be determined to be any one value within the range.

[0217] Specifically, it can be checked whether the P block and / or the Q block are blocks to which BDPCM is applied. If both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined to be 0. Otherwise, it can be checked whether the P block and / or the Q block are blocks coded in intra mode or combined inter-intra prediction mode (CIIP mode).

[0218] If the P block and / or the Q block is a block coded in intra mode or CIIP mode, BS may be determined to be 4. Otherwise, it can check at least one of whether the block edge corresponds to a transform unit edge or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0219] If the block edge corresponds to a transform unit edge and at least one of the P block and the Q block has a non-zero transform coefficient, the BS may be determined to be 3. Otherwise, it can be checked whether the following condition 1 is met.

[0220] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0221] If the condition 1 is met, the BS may be determined to be 3. If the condition 1 is not met, the BS may be determined to be 2, 1, or 0. If the condition 1 is not met, it can be further checked whether the following condition 2 is met.

[0222] (Condition 2) Both the P block and the Q block are blocks coded in IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0223] If the condition 2 is met, the BS may be determined to be 2. If the condition 2 is not met, the BS may be determined to be 2, 1, or 0. If the condition 2 is not met, it can be further checked whether the following condition 3 is met.

[0224] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of motion vectors for predicting the P block and the number of motion vectors for predicting the Q block are different from each other.

[0225] If the condition 3 is met, the BS may be determined to be 2. If the condition 3 is not met, the BS may be determined to be 1 or 0. If the condition 3 is not met, it can be further checked whether the following condition 4 is met.

[0226] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0227] If the condition 4 is satisfied, the BS may be determined to be 1. If the condition 4 is not satisfied, the BS may be determined to be 0.

[0228] Example 2-F

[0229] This disclosure relates to a method for determining the BS of a deblocking filter when OBMC is applied, which can improve subjective image quality. When OBMC is applied, boundaries between blocks (e.g., CU boundaries, subblock boundaries) are smoothed using motion information from neighboring blocks, so discontinuities between blocks can be removed. Therefore, applying a deblocking filter to a block to which OBMC is applied may result in excessive smoothing of the boundaries between blocks.

[0230] Specifically, it can be checked whether the P block and / or the Q block are blocks to which BDPCM or OBMC is applied. As an example, if both the P block and the Q block are blocks to which OBMC is applied, BS may be determined to be 0. If not (i.e., if at least one of the P block or the Q block is a block to which OBMC is not applied), BS may be determined to be 2, 1, or 0 based on whether the P block and / or the Q block is a block to which OBMC is applied. Alternatively, regardless of whether the P block is a block to which OBMC is applied, BS may be determined to be 0 if the Q block is a block to which OBMC is applied. If not (i.e., if the Q block is a block to which OBMC is not applied), BS may be determined to be 2, 1, or 0 based on whether the P block and / or the Q block is a block to which OBMC is applied.

[0231] If the P block and / or the Q block is a block coded in intra mode or CIIP mode, BS may be determined to be 2. Otherwise (i.e., if neither the P block nor the Q block is a block coded in intra mode or CIIP mode), at least one of whether the block edge corresponds to a transform unit edge or whether at least one of the P block or the Q block has non-zero transform coefficients can be checked.

[0232] If the block edge corresponds to a transform unit edge and at least one of the P block and the Q block has non-zero transform coefficients, BS may be determined to be 1. Otherwise (i.e., if the block edge does not correspond to a transform unit edge or if neither the P block nor the Q block has non-zero transform coefficients), it can be checked whether at least one of the following conditions 1 to 4 is met.

[0233] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block, where each of the P and Q blocks has one of a plurality of predefined prediction modes, and the plurality of prediction modes may include at least one of intra mode, inter mode, IBC mode, or palette mode.

[0234] (Condition 2) Both the P block and the Q block are blocks coded in IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0235] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of motion vectors for predicting the P block and the number of motion vectors for predicting the Q block are different from each other.

[0236] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0237] If at least one of the conditions 1 to 4 is satisfied, the BS may be determined to be 1. If none of the conditions 1 to 4 is satisfied, the BS may be determined to be 0. The method of determining the BS depending on whether the conditions 2 to 4 are satisfied is as described in Example 2-A.

[0238] Example 2-G

[0239] This disclosure relates to a method for determining the BS of a deblocking filter when OBMC is applied, which can improve subjective image quality. When OBMC is applied, boundaries between blocks (e.g., CU boundaries, subblock boundaries) are smoothed using motion information from neighboring blocks, so discontinuities between blocks can be removed. Therefore, applying a deblocking filter to a block to which OBMC is applied may result in excessive smoothing of the boundaries between blocks.

[0240] The present disclosure defines an expanded range for the BS of a deblocking filter, thereby enabling more precise application of the deblocking filter. The present disclosure can determine the BS to be any one value within the expanded range, and can subdivide the process of determining the deblocking filter for each stage. According to the present disclosure, a range of 0 to 4 can be defined for the BS of the deblocking filter, and the BS can be determined to be any one value within the range.

[0241] Specifically, it can be checked whether the P block and / or the Q block are blocks to which BDPCM is applied. If both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined to be 0. Otherwise, it can be checked whether the P block and / or the Q block are blocks coded in intra mode or combined inter-intra prediction mode (CIIP mode).

[0242] If the P block and / or the Q block is a block coded in intra mode or CIIP mode, BS may be determined to be 4. Otherwise, it can check at least one of whether the block edge corresponds to a transform unit edge or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0243] If the block edge corresponds to a transform unit edge and at least one of the P block and the Q block has a non-zero transform coefficient, the BS may be determined to be 3. Otherwise, it can be checked whether the following condition 1 is met.

[0244] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0245] If the condition 1 is met, the BS may be determined to be 3. If the condition 1 is not met, the BS may be determined to be 2, 1, or 0. If the condition 1 is not met, it can be further checked whether the following condition 2 is met.

[0246] (Condition 2) Both the P block and the Q block are blocks coded in IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0247] If the condition 2 is met, the BS may be determined to be 2. If the condition 2 is not met, the BS may be determined to be 2, 1, or 0. If the condition 2 is not met, it can be further checked whether the following condition 3 is met.

[0248] (Condition 3) Neither the P block nor the Q block is a block to which OBMC is applied, and the reference picture of the P block is different from the reference picture of the Q block. Or, neither the P block nor the Q block is a block to which OBMC is applied, and the number of motion vectors used to predict the P block is different from the number of motion vectors used to predict the Q block. Or, the Q block is not a block to which OBMC is applied, and the reference picture of the P block is different from the reference picture of the Q block. Or, the Q block is not a block to which OBMC is applied, and the number of motion vectors used to predict the P block is different from the number of motion vectors used to predict the Q block.

[0249] If the condition 3 is met, the BS may be determined to be 2. If the condition 3 is not met, the BS may be determined to be 1 or 0. If the condition 3 is not met, it can be further checked whether the following condition 4 is met.

[0250] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0251] If the condition 4 is satisfied, the BS may be determined to be 1. If the condition 4 is not satisfied, the BS may be determined to be 0.

[0252] Referring to FIG. 4, a deblocking filter having a predetermined type and strength can be determined based on the filter length and BS of the deblocking filter (S420).

[0253] Specifically, based on at least one of the sample variation amount within a block or the sample variation amount at an edge between blocks, it may be determined whether the edge corresponds to an actual edge of the block or whether it is an edge caused by a blocking artifact. Based on the determination, the type and / or strength of a deblocking filter may be determined. Examples of the type of the deblocking filter include a short filter and a long filter. Examples of the strength of the deblocking filter include a strong filter and a weak filter.

[0254] The deblocking filter is a filter that is implemented by a predetermined variable (e.g., β, t C The predetermined variable may be compared to at least one of a sample variation within the Q and P blocks or a sample variation at an edge, thereby determining the type and / or strength of the deblocking filter.

[0255] The offset for each variable (e.g., qpOffset, sh_luma_beta_offset_div2, sh_luma_tc_offset_div2) and the quantization parameter (Qp) of the Q / P block are used to calculate the variables β and t C As an example, the variables β and t C may be calculated as shown in Equation 13 and Table 1 below.

[0256] [Formula 13]

number

[0257] [Table 1]

[0258] In Equation 13, Qp Q and Qp P may represent the quantization parameters of the Q block and the P block, respectively. In Equation 13, qpOffset may be derived as shown in Table 2 below.

[0259] [Table 2]

[0260] The semantics of the syntax elements (sps_ladf_qp_offset, sh_luma_beta_offset_div2, sh_luma_tc_offset_div2) are as shown in Table 3 below.

[0261] [Table 3]

[0262] When the range of BS changes, the variables β, t C may be changed as follows: For example, if the range of 0 to 3 is defined for the BS of the deblocking filter, the variables β, t C may be changed as shown in Equation 14. When the range of 0 to 2 is defined, the amount of change in Q by BS becomes 2, 0, and -2. On the other hand, when the range of 0 to 3 is defined, the amount of change in Q by BS may be changed to 3, 1, -1, and -3 and applied.

[0263] [Formula 14]

number

[0264] Or, if the range of 0 to 4 is defined for the BS of the deblocking filter, the variables β, t C may be changed as shown in Equation 15. When the range of 0 to 4 is defined, the amount of change in Q by BS may be changed to 4, 2, 0, −2, and −4 and applied.

[0265] [Formula 15]

number

[0266] Or, if the range of 0 to 3 or the range of 0 to 4 is defined for the BS of the deblocking filter, the variables β, t C may be changed as shown in Equation 16. When the range of 0 to 3 or the range of 0 to 4 is defined, the amount of change in Q due to BS may be changed to 3, 2, 1, 0 or 4, 3, 2, 1, similarly to the value of BS, and applied.

[0267] [Formula 16]

number

[0268] As mentioned above, the value of Q in Table 1 can be adjusted depending on the value of BS. Alternatively, Q and the variables β and t C The mapping table between them may be finer tuned.

[0269] Referring to FIG. 4, the current picture can be updated by applying a deblocking filter to block boundaries within the current picture (S430).

[0270] FIG. 5 shows a schematic configuration of a decoding device 300 that performs the video decoding method according to the present disclosure.

[0271] Referring to FIG. 5, the decoding apparatus 300 may include a filter length determination unit 500, a boundary strength determination unit 510, a filter determination unit 520, and a filter application unit 530.

[0272] The filter length determination unit 500 may determine the filter length of the deblocking filter based on the filter length determination method described with reference to FIG. 4, and therefore, a redundant description will be omitted here.

[0273] The boundary strength determination unit 510 may determine the boundary strength (BS) of the deblocking filter. The boundary strength determination unit 510 may determine the BS of the deblocking filter based on the BS determination method described with reference to FIG. 4, and redundant description will be omitted here.

[0274] The filter determination unit 520 can determine a deblocking filter having a predetermined type and strength based on the filter length and BS of the deblocking filter. Here, the deblocking filter has predetermined variables (e.g., β, t C ), and the method for deriving said variables is as described with reference to FIG. 4.

[0275] The filter applicator 530 may apply a deblocking filter to block boundaries within the current picture and update the current picture.

[0276] FIG. 6 illustrates a video encoding method performed by the encoding device according to the present disclosure.

[0277] 6, the filter length of the deblocking filter can be determined (S600). The method for determining the filter length of the deblocking filter is the same as that described with reference to FIG.

[0278] 6, the boundary strength (BS) of the deblocking filter can be determined (S610). The method of determining the BS is the same as that described with reference to FIG.

[0279] 6, a deblocking filter having a predetermined type and strength can be determined based on the filter length and BS of the deblocking filter (S620). Here, the deblocking filter is determined based on predetermined variables (e.g., β, t C ), and the method for deriving said variables is as described with reference to FIG. 4.

[0280] 6, a deblocking filter may be applied to block boundaries within the current picture to modify the current picture (S630). The modified current picture may be stored in memory 270 within encoding apparatus 200 and may be referenced for inter-prediction of other pictures.

[0281] FIG. 7 shows a schematic configuration of an encoding device 200 that performs the video encoding method according to the present disclosure.

[0282] Referring to FIG. 7, the encoding apparatus 200 may include a filter length determination unit 700, a boundary strength determination unit 710, a filter determination unit 720, and a filter application unit 730.

[0283] The filter length determination unit 700 may determine the filter length of the deblocking filter based on the filter length determination method described with reference to FIG. 4, and therefore, a redundant description will be omitted here.

[0284] The boundary strength determination unit 710 may determine the boundary strength (BS) of the deblocking filter. The boundary strength determination unit 710 may determine the BS of the deblocking filter based on the BS determination method described with reference to FIG. 4, and redundant description will be omitted here.

[0285] The filter determination unit 720 may determine a deblocking filter having a predetermined type and strength based on the filter length and BS of the deblocking filter. Here, the deblocking filter may be a filter having predetermined variables (e.g., β, t C ), and the method for deriving said variables is as described with reference to FIG. 4.

[0286] The filter applicator 730 may modify the current picture by applying a deblocking filter to block boundaries within the current picture.

[0287] In the above-described embodiments, the method is described based on a flowchart with a series of steps or blocks, but the embodiment is not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included, or one or more steps of the flowchart may be deleted without affecting the scope of the embodiments of this document.

[0288] The methods according to the embodiments of the present document described above may be implemented in the form of software, and the encoding device and / or decoding device according to the present document may be included in a device that performs video processing, such as a TV, a computer, a smartphone, a set-top box, or a display device.

[0289] When embodiments in this document are embodied as software, the methods described above may be embodied as modules (processes, functions, etc.) that perform the functions described above. The modules may be stored in memory and executed by a processor. The memory may be internal or external to the processor and may be coupled to the processor by various known means. The processor may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and / or other storage devices. That is, the embodiments described herein may be embodied and executed on a processor, microprocessor, controller, or chip. For example, the functional units illustrated in the figures may be embodied and executed on a computer, processor, microprocessor, controller, or chip. In this case, information (e.g., information on instructions) or algorithms for the implementation may be stored on a digital storage medium.

[0290] In addition, the decoding device and encoding device to which the embodiments of the present specification are applied may be included in a multimedia broadcast transmitting / receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camcorder, a custom video (VoD) service providing device, an over-the-top (OTT) video (over-the-top) device, an internet streaming service providing device, a three-dimensional (3D) video device, a virtual reality (VR) device, an augmented reality (AR) device, a video telephone video device, a vehicle terminal (e.g., a vehicle terminal (including an autonomous vehicle), an airplane terminal, a ship terminal, etc.), a medical video device, etc., and may be used to process video signals or data signals. For example, over-the-top (OTT) video (over-the-top) video devices may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), etc.

[0291] In addition, a processing method to which the embodiments of the present specification are applied may be produced in the form of a program executed by a computer and stored in a computer-readable recording medium. Multimedia data having a data structure according to the embodiments of the present specification may also be stored in a computer-readable recording medium. The computer-readable recording medium may include any type of storage device or distributed storage device in which computer-readable data is stored. The computer-readable recording medium may include, for example, a Blu-ray Disc (BD), a Universal Serial Bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. The computer-readable recording medium may also include media embodied in the form of a carrier wave (e.g., transmission via the Internet). In addition, a bitstream generated by the encoding method may be stored in a computer-readable recording medium or transmitted via a wired or wireless communication network.

[0292] Furthermore, the embodiments of the present specification may be embodied as a computer program product using program code, which may be executed by a computer according to the embodiments of the present specification. The program code may be stored on a computer-readable carrier.

[0293] FIG. 8 illustrates an example of a content streaming system to which the embodiments of the present disclosure can be applied.

[0294] Referring to FIG. 8, a content streaming system to which the embodiments of the present specification are applied may broadly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0295] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, camcorder, etc. into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0296] The bitstream may be generated by an encoding method or a bitstream generation method to which the embodiments of this specification are applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0297] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server acts as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, which controls commands and responses between devices in the content streaming system.

[0298] The streaming server can receive content from a media storage and / or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.

[0299] Examples of the user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, and head-mounted displays (HMDs)), digital TVs, desktop computers, and digital signage.

[0300] Each server in the content streaming system may be operated as a distributed server, in which case data received by each server may be processed in a distributed manner.

[0301] The claims described herein may be combined in various ways. For example, technical features of method claims herein may be combined and embodied as an apparatus, and technical features of apparatus claims herein may be combined and embodied as a method. Furthermore, technical features of method claims herein and technical features of apparatus claims herein may be combined and embodied as an apparatus, and technical features of method claims herein and technical features of apparatus claims herein may be combined and embodied as a method.

Claims

1. determining a filter length of a deblocking filter to be applied to a block boundary between a first block and a second block in a current picture; determining a boundary strength of the deblocking filter; determining the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength; applying the deblocking filter to a block boundary between the first block and the second block; A video decoding method, wherein a filter length of the deblocking filter is determined based on whether multiple reference prediction is applied to at least one of the first block or the second block.

2. 2. The video decoding method of claim 1, wherein a filter length of the deblocking filter is determined based on at least one of a size of the second block or whether the second block is a block coded in a sub-block-based inter prediction mode.

3. The image decoding method of claim 1 , wherein the boundary strength is determined based on whether a number of multiple reference blocks for the first block is equal to a number of multiple reference blocks for the second block.

4. The video decoding method of claim 1 , wherein the boundary strength is determined based on a weight value for the multi-reference prediction.

5. 2. The video decoding method of claim 1, wherein, when the multiple reference prediction is applied to the second block, the boundary strength is determined based on whether a picture to which a normal reference block of the second block belongs is the same as a picture to which an additional reference block of the second block belongs.

6. 2. The video decoding method of claim 1, wherein, when the multiple reference prediction is applied to the second block, the boundary strength is determined based on whether a picture to which an additional reference block of the second block belongs is the same as a picture to which a normal reference block of the first block belongs.

7. 2. The video decoding method of claim 1, wherein, when the multiple reference prediction is applied to the second block, the boundary strength is determined based on a difference between a motion vector of the second block with respect to an additional reference block and a motion vector of the first block with respect to a normal reference block.

8. 2. The video decoding method of claim 1, wherein, when the multiple reference prediction is applied to the second block, the boundary strength is determined based on a difference between a motion vector of the second block with respect to a normal reference block and a motion vector of the first block with respect to an additional reference block.

9. The video decoding method of claim 1 , wherein the predefined maximum value for the boundary strength is 3 or 4.

10. determining a filter length of a deblocking filter to be applied to a block boundary between a first block and a second block in a current picture; determining a boundary strength of the deblocking filter; determining the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength; applying the deblocking filter to a block boundary between the first block and the second block; A video encoding method, wherein a filter length of the deblocking filter is determined based on whether multiple reference prediction is applied to at least one of the first block or the second block.

11. A computer-readable recording medium storing a bitstream generated by the video encoding method of claim 10.

12. obtaining a bitstream for video information, wherein the bitstream is generated by determining a filter length of a deblocking filter to be applied to a block boundary between a first block and a second block in a current picture, determining a boundary strength of the deblocking filter, determining the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and applying the deblocking filter to the block boundary between the first block and the second block; transmitting data including the bitstream; A data transmission method, wherein a filter length of the deblocking filter is determined based on whether multiple reference prediction is applied to at least one of the first block or the second block.

Images