Image encoding / decoding method and apparatus, and recording medium storing bitstream thereon

Abstract

The image decoding / encoding method and apparatus according to the present disclosure can determine a filter length of a deblocking filter applied to a block boundary between a first block and a second block in a current picture, determine a boundary strength of the deblocking filter, determine a deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and decode / encode the image based on the determined deblocking filter. And applying a deblocking filter to a block boundary between the first block and the second block. Here, the filter length of the deblocking filter can be determined on the basis of whether the multi-reference prediction is applied to the first block and / or the second block.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and a recording medium storing a bitstream. Background Art

[0002] Recently, the demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images has been increasing in various application fields, and thus, efficient image compression techniques are being discussed.

[0003] There are various techniques such as an inter prediction technique that predicts a pixel value included in a current picture from a picture before or after the current picture using a video compression technique, an intra prediction technique that predicts a pixel value included in the current picture by using pixel information in the current picture, an entropy coding technique that assigns a short symbol to a value with a high occurrence frequency and a long symbol to a value with a low occurrence frequency, etc., and these image compression techniques can be used to effectively compress image data and transmit or store it. Summary of the Invention

[0004] Technical Problem

[0005] The present disclosure provides a method and apparatus for determining a filter length of a deblocking filter.

[0006] The present disclosure provides a method and apparatus for determining a boundary strength of a deblocking filter.

[0007] The present disclosure provides a method and apparatus for deriving a variable for determining a deblocking filter.

[0008] Technical Solution

[0009] An image decoding method and apparatus according to the present disclosure may: determine a filter length of a deblocking filter applied to a block boundary between a first block and a second block in a current picture, determine a boundary strength of the deblocking filter, determine a deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and apply the deblocking filter to the block boundary between the first block and the second block. Here, the filter length of the deblocking filter may be determined based on whether multi-reference prediction is applied to at least one of the first block or the second block.

[0010] In the image decoding method and apparatus according to the present disclosure, the filter length of the deblocking filter may be determined based on at least one of the size of the second block or whether the second block is a block encoded in an inter prediction mode based on sub-blocks.

[0011] In the image decoding method and apparatus according to the present disclosure, the boundary strength may be determined based on whether the number of multiple reference blocks for the first block is the same as the number of multiple reference blocks for the second block.

[0012] In the image decoding method and apparatus according to the present disclosure, the boundary strength may be determined based on weights for multi-reference prediction.

[0013] In the image decoding method and apparatus according to the present disclosure, when multi-reference prediction is applied to a second block, the boundary strength may be determined based on whether the picture to which the normal reference block of the second block belongs is the same as the picture to which the additional reference block of the second block belongs.

[0014] In the image decoding method and apparatus according to the present disclosure, when multi-reference prediction is applied to a second block, the boundary strength may be determined based on whether the picture to which the additional reference block of the second block belongs is the same as the picture to which the normal reference block of the first block belongs.

[0015] In the image decoding method and apparatus according to the present disclosure, when multi-reference prediction is applied to a second block, the boundary strength may be determined based on the difference between the motion vector associated with the additional reference block of the second block and the motion vector associated with the normal reference block of the first block.

[0016] In the image decoding method and apparatus according to the present disclosure, when multi-reference prediction is applied to a second block, the boundary strength may be determined based on the difference between the motion vector associated with the normal reference block of the second block and the motion vector associated with the additional reference block of the first block.

[0017] In the image decoding method and apparatus according to the present disclosure, a predefined maximum value for the boundary strength may be 3 or 4.

[0018] The image encoding method and apparatus according to the present disclosure may determine the filter length of a deblocking filter applied to a block boundary between a first block and a second block in a current picture, determine the boundary strength of the deblocking filter, determine a deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and apply the deblocking filter to the block boundary between the first block and the second block. Here, the filter length of the deblocking filter may be determined based on whether multi-reference prediction is applied to at least one of the first block or the second block.

[0019] A computer-readable digital storage medium is provided that stores encoded video / image information, thereby causing an image decoding method to be performed by a decoding apparatus according to the present disclosure.

[0020] A computer-readable digital storage medium according to the present disclosure stores video / image information generated according to an image encoding method.

[0021] A method and apparatus for transmitting video / image information generated according to an image encoding method according to the present disclosure are provided.

[0022] Advantageous Effects

[0023] According to the present disclosure, it is possible to adaptively determine the filter length of a deblocking filter by considering whether to apply multi-reference prediction and / or whether to apply overlapping block motion compensation.

[0024] According to the present disclosure, by considering whether to apply multi-reference prediction and / or whether to apply overlapping block motion compensation, it is possible to more accurately determine the boundary strength of the deblocking filter and improve the smoothing effect at the block boundary based on this.

[0025] According to the present disclosure, by defining an extended range for the boundary strength of the deblocking filter, it is possible to more precisely apply the deblocking filter.

[0026] According to the present disclosure, by selectively using a variable derivation method according to the extended range, it is possible to effectively derive variables for determining the deblocking filter. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Shows a video / image compilation system according to the present disclosure.

[0028] Figure 2 Shows a schematic block diagram of an encoding device to which an embodiment of the present disclosure is applicable and which encodes a video / image signal.

[0029] Figure 3 Shows a schematic block diagram of a decoding device to which an embodiment of the present disclosure is applicable and which decodes a video / image signal.

[0030] Figure 4 Illustrates an image decoding method performed in an image decoding device (300) according to the present disclosure.

[0031] Figure 5 Illustrates a schematic configuration of a decoding device (300) that performs an image decoding method according to the present disclosure.

[0032] Figure 6 Illustrates an image encoding method performed in an encoding device (200) according to the present disclosure.

[0033] Figure 7 Illustrates a schematic configuration of an encoding device (200) that performs an image encoding method according to the present disclosure.

[0034] Figure 8 Shows an example of a content streaming system to which an embodiment of the present disclosure can be applied. DETAILED DESCRIPTION

[0035] Since the present disclosure can be made in various changes and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all changes, equivalents, and alternatives included in the spirit and technical scope of the present disclosure. When describing each drawing, like reference numerals are used for like components.

[0036] Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by these terms. These terms are only used to distinguish one component from other components. For example, without departing from the scope of the rights of the present disclosure, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes any one or more combinations of a plurality of related recited items.

[0037] When a component is referred to as being "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to another component, but there may also be another component in between. On the other hand, when a component is referred to as being "directly connected" or "directly linked" to another component, it should be understood that there is no other component in between.

[0038] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, the singular expression includes the plural expression. In this application, it should be understood that terms such as "including" or "having" are intended to designate the presence of the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but do not preclude the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof in advance.

[0039] The present disclosure relates to video / image compilation. For example, the methods / embodiments disclosed herein can be applied to the methods disclosed in the general video coding (VVC) standard. Additionally, the methods / embodiments disclosed herein can be applied to the methods disclosed in the basic video coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second-generation audio video coding standard (AVS2), or the next-generation video / image coding standards (such as H.267 or H.268, etc.).

[0040] This specification presents various embodiments of video / image compilation, and unless otherwise stated, these embodiments can be combined with each other for implementation.

[0041] Here, a video can refer to a collection of a series of images over time. A picture generally refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms part of a picture during compilation. A slice / tile can include at least one Compilation Tree Unit (CTU). A picture can be composed of at least one slice / tile. A tile is a rectangular area composed of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of CTUs having the same height as the picture and a width assigned by the syntax requirements of the picture parameter set. A tile row is a rectangular area of CTUs having a height assigned by the picture parameter set and the same width as the picture. The CTUs within a tile can be arranged consecutively according to the CTU raster scan, while the tiles within a picture can be arranged consecutively according to the tile raster scan. A slice can include an integer number of complete tiles or an integer number of consecutive complete CTU rows within the tiles of a picture that can be exclusively included in a single NAL unit. At the same time, a picture can be divided into at least two sub-pictures. A sub-picture can be a rectangular area of at least one slice within the picture.

[0042] A pixel, pel, or picture element can refer to the smallest unit that makes up a picture (or image). Additionally, "sample" can be used as a term corresponding to a pixel. A sample generally can represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component.

[0043] A unit can represent the basic unit of image processing. A unit can include at least one of a specific area of a picture and information related to the corresponding area. A unit can include one luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" can be used interchangeably with terms such as "block" or "region". Generally, an MxN block can include a set (or array) of transform coefficients or samples (or sample arrays) composed of M columns and N rows.

[0044] Here, "A or B" can refer to "only A", "only B", or "both A and B". In other words, here, "A or B" can be interpreted as "A and / or B". For example, here, "A, B, or C" can refer to "only A", "only B", "only C", or "any combination of A, B, and C".

[0045] The slashes ( / ) or commas used in this document can refer to "and / or". For example, "A / B" can refer to "A and / or B". Thus, "A / B" can refer to "only A", "only B", or "both A and B". For example, "A, B, C" can refer to "A, B, or C".

[0046] Here, "at least one of A and B" can refer to "only A", "only B", or "both A and B". Additionally, in this document, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same way as "at least one of A and B".

[0047] Furthermore, here, "at least one of A, B, and C" can refer to "only A", "only B", "only C", or any combination of A, B, and C. Additionally, "at least one of A, B, or C" or "at least one of A, B, and / or C" can refer to "at least one of A, B, and C".

[0048] Moreover, the parentheses used in this document can refer to "for example". Specifically, when it is indicated as "prediction (intra prediction)", "intra prediction" can be presented as an example of "prediction". In other words, "prediction" here is not limited to "intra prediction", and "intra prediction" can be presented as an example of "prediction". Additionally, even when it is indicated as "prediction (i.e., intra prediction)", "intra prediction" can be presented as an example of "prediction".

[0049] Here, the technical features described separately in one drawing can be implemented individually or simultaneously.

[0050] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0051] Reference Figure 1 , the video / image compilation system may include a first device (source device) and a second device (receiving device).

[0052] The source device can send the encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or a network. The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. A transmitter may be included in the encoding device. A receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0053] The video source can obtain video / images through processes such as capturing, synthesizing, or generating video / images. The video source can include devices for capturing video / images and devices for generating video / images. Devices for capturing video / images can include at least one camera, video / image archives including previously captured video / images, etc. Devices for generating video / images can include computers, tablets, smartphones, etc., and can (electronically) generate video / images. For example, virtual video / images can be generated by a computer, etc., and in this case, the process of capturing video / images can be replaced by the process of generating relevant data.

[0054] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation, quantization, etc. for compression and compilation efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0055] The sending unit can send the encoded video / image information or data output in the form of a bitstream to the receiving unit of the receiving device in the form of a file or streaming through a digital storage medium or a network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit can include elements for generating a media file in a predetermined file format and can include elements for transmission through a broadcast / communication network. The receiving unit can receive / extract the bitstream and send it to the decoding device.

[0056] The decoding device can decode the video / image by performing a series of processes such as inverse quantization, inverse transformation, prediction, etc. corresponding to the operations of the encoding device.

[0057] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0058] Figure 2 A rough block diagram of an encoding device that can apply embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0059] Reference Figure 2, the encoding device 200 may be composed of an image splitter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, an inverse quantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to an embodiment, the above-mentioned image splitter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or a processor). Additionally, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory 270 as an internal / external component.

[0060] The image splitter 210 may partition an input image (or picture, frame) input to the encoding device 200 into at least one processing unit. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively partitioned from a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure.

[0061] For example, one coding unit may be partitioned into multiple coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quadtree structure may be applied first, and later the binary tree structure and / or the ternary structure may be applied. Alternatively, the binary tree structure may be applied before the quadtree structure. The coding process according to this specification may be performed based on the final coding unit that is no longer partitioned. In this case, based on the coding efficiency according to the image characteristics, etc., the largest coding unit may be directly used as the final coding unit, or if necessary, the coding unit may be recursively partitioned into coding units with a deeper depth, and the coding unit with the optimal size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction described later.

[0062] As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may be respectively divided or partitioned from the above-mentioned final coding unit. The prediction unit may be a unit for sample prediction, and the transformation unit may be a unit for deriving transformation coefficients and / or a unit for deriving a residual signal from the transformation coefficients.

[0063] In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". Generally, an MxN block may represent a set of transform coefficients or samples consisting of M columns and N rows. Samples may generally represent pixels or pixel values, and may represent only the pixels / pixel values of the luminance component, or only the pixels / pixel values of the chrominance component. Samples may be used as a term corresponding to pixels or pels in a picture (or image).

[0064] The encoding device 200 may subtract the prediction signal (prediction block, prediction sample array) output from the inter-frame predictor 221 or the intra-frame predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to the transformer 232. In this case, the unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device 200 may be referred to as the subtractor 231.

[0065] The predictor 220 may perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block including the prediction samples for the current block. The predictor 220 may determine whether to apply intra-frame prediction or inter-frame prediction in units of the current block or CU. The predictor 220 may generate various information about the prediction, such as prediction mode information, etc., and send it to the entropy encoder 240, as will be described later in the description of each prediction mode. The information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0066] The intra-frame predictor 222 may predict the current block by referring to the samples within the current picture. Depending on the prediction mode, the samples referred to may be located near the current block or may be located at a certain distance from the current block. In intra-frame prediction, the prediction mode may include at least one non-directional mode and multiple directional modes. The non-directional mode may include at least one of the DC mode or the planar mode. Depending on the level of detail of the prediction direction, the directional mode may include 33 directional modes or 65 directional modes. However, this is only an example, and more or fewer directional modes may be used depending on the configuration. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0067] The inter-frame predictor 221 may derive a prediction block for a current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or the reference picture index of the current block. The inter-frame prediction may be performed based on various prediction modes, and for example, for the skip mode and the merge mode, the inter-frame predictor 221 may use the motion information of neighboring blocks as the motion information of the current block. For the skip mode, different from the merge mode, the residual signal may not be transmitted. For the motion vector prediction (MVP) mode, the motion vector of a neighboring block is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0068] The predictor 220 may generate a prediction signal based on various prediction methods described later. For example, the predictor may not only apply intra-frame prediction or inter-frame prediction to predict a block, but also apply intra-frame prediction and inter-frame prediction simultaneously. It may be referred to as a combined inter-frame and intra-frame prediction (CIIP) mode. Additionally, the predictor may be based on the intra-block copy (IBC) prediction mode or may be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode may be used for content image / video compilation such as games, such as screen content compilation (SCC), etc. IBC basically performs prediction within the current picture, but it may be performed similar to inter-frame prediction because it derives a reference block within the current picture. In other words, IBC may use at least one of the inter-frame prediction techniques described herein. The palette mode may be considered an example of intra-frame compilation or intra-frame prediction. When the palette mode is applied, the sample values within the picture may be signaled based on the information about the palette table and the palette index. The prediction signal generated by the predictor 220 may be used to generate a reconstructed signal or a residual signal.

[0069] The transformer 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional non-linear transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on a prediction signal generated by using all previously reconstructed pixels. Additionally, the transform process may be applied to square pixel blocks of the same size or may be applied to non-square blocks of variable size.

[0070] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[0071] The entropy encoder 240 may perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may encode information necessary for video / video image reconstruction (e.g., values of syntax elements, etc.) in addition to the transform coefficients quantized together or individually.

[0072] Encoded information (e.g., encoded video / image information) can be sent or stored in the form of a bitstream in units of Network Abstraction Layer (NAL) units. The video / image information may further include information on various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may further include general constraint information. Here, the information and / or syntax elements sent / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information can be encoded through the above encoding process and included in the bitstream. The bitstream can be sent over a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmission and / or a storage unit (not shown) for storing the signal output from the entropy encoder 240 may be configured as internal / external elements of the encoding device 200, or the transmission unit may also be included in the entropy encoder 240.

[0073] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the dequantization and inverse transformation can be applied to the quantized transform coefficients by the dequantizer 234 and the inverse transformer 235 to reconstruct a residual signal (residual block or residual samples). The adder 250 can add the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed, such as when the skip mode is applied, the predicted block can be used as the reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed within the current picture and can also be used for inter-frame prediction of the next picture through filtering described later. Meanwhile, a Luminance Mapping with Chroma Scaling (LMCS) can be applied during the picture encoding and / or reconstruction process.

[0074] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 270, specifically in the DPB of the memory 270. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information about the filtering and send it to the entropy encoder 240. The information about the filtering can be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0075] The modified reconstructed picture sent to the memory 270 can be used as a reference picture in the inter - frame predictor 221. When inter - frame prediction is applied thereto, the encoding device can avoid prediction mismatches in the encoding device 200 and the decoding device, and can also improve the encoding efficiency.

[0076] The DPB of the memory 270 can store the modified reconstructed picture to use it as a reference picture in the inter - frame predictor 221. The memory 270 can store the motion information of the blocks from which the motion information in the current picture is derived (or encoded) and / or the motion information of the blocks in the pre - reconstructed picture. The stored motion information can be sent to the inter - frame predictor 221 to be used as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 270 can store the reconstructed samples of the reconstructed blocks in the current picture and send them to the intra - frame predictor 222.

[0077] Figure 3 A rough block diagram of a decoding device that can apply embodiments of the present disclosure and perform decoding of video / image signals is shown.

[0078] Reference Figure 3 , the decoding device 300 can be configured by including an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 can include an inter - frame predictor 332 and an intra - frame predictor 331. The residual processor 320 can include an inverse quantizer 321 and an inverse transformer 321.

[0079] According to an embodiment, the above - mentioned entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 can be configured by one hardware component (e.g., a decoder chipset or a processor). Additionally, the memory 360 can include a decoded picture buffer (DPB) and can be configured by a digital storage medium. The hardware component can further include the memory 360 as an internal / external component.

[0080] When a bitstream including video / image information is input, the decoding device 300 can reconstruct an image in response to the process of processing the video / image information in the Figure 2 encoding device. For example, the decoding device 300 can derive units / blocks based on the relevant information of block segmentation obtained from the bitstream. The decoding device 300 can perform decoding by using the processing units applied in the encoding device. Therefore, the decoded processing unit can be a compile unit, and the compile unit can be segmented from a compile tree unit or a maximum compile unit according to a quadtree structure, a binary tree structure, and / or a ternary tree structure. At least one transform unit can be derived from the compile unit. And the reconstructed image signal decoded and output by the decoding device 300 can be played by a playback device.

[0081] The decoding device 300 can receive a signal output from the Figure 2 encoding device in the form of a bitstream, and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information can further include information about various parameter sets such as an Adaptive Parameter Set (APS), a Picture Parameter Set (PPS), a Sequence Parameter Set (SPS), or a Video Parameter Set (VPS). Additionally, the video / image information can further include general constraint information. The decoding device can further decode the picture based on the information about the parameter sets and / or the general constraint information. The information sent / received by signal and / or the syntax elements described later in this document can be decoded through the decoding process and obtained from the bitstream. For example, the entropy decoder 310 can decode the information in the bitstream based on coding methods such as Exponential Golomb coding, CAVLC, CABAC, etc., and output the values of the syntax elements necessary for image reconstruction and the quantization values of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive the bins corresponding to each syntax element from the bitstream, determine the context model by using the information of the syntax element to be decoded, the neighboring blocks, and the decoding information of the block to be decoded, or the information of the symbols / bins decoded in the previous step, perform arithmetic decoding on the bins by predicting the occurrence probability of the bins according to the determined context model, and generate symbols corresponding to the values of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins about the context model for the next symbol / bin. Among the information decoded in the entropy decoder 310, the information about prediction is provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values for which entropy decoding is performed in the entropy decoder 310, i.e., the quantized transform coefficients and the related parameter information, can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). Additionally, the information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. Meanwhile, the receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300 or the receiving unit can be a component of the entropy decoder 310.

[0082] Meanwhile, the decoding device according to this specification may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of an inverse quantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0083] The inverse quantizer 321 may inverse-quantize the quantized transform coefficients and output the transform coefficients. The inverse quantizer 321 may rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scan order executed in the encoding device. The inverse quantizer 321 may perform inverse quantization on the quantized transform coefficients by using a quantization parameter (e.g., quantization step information) and obtain the transform coefficients.

[0084] The inverse transformer 322 inverse-transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0085] The predictor 320 may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor 320 may determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the information about prediction output from the entropy decoder 310, and determine a specific intra-frame / inter-frame prediction mode.

[0086] The predictor 320 may generate a prediction signal based on various prediction methods described later. For example, the predictor 320 may not only apply intra-frame prediction or inter-frame prediction to predict a block, but also apply intra-frame prediction and inter-frame prediction simultaneously. It may be referred to as a combined inter-frame and intra-frame prediction (CIIP) mode. Additionally, the predictor may be based on the intra-block copy (IBC) prediction mode or may be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode may be used for content image / video compilation such as games, such as screen content compilation (SCC), etc. IBC basically performs prediction within the current picture, but it may be performed similar to inter-frame prediction because it derives a reference block within the current picture. In other words, IBC may use at least one of the inter-frame prediction techniques described herein. The palette mode may be considered an example of intra-frame compilation or intra-frame prediction. When the palette mode is applied, information about the palette table and the palette index may be included in the video / image information and signaled.

[0087] The intra predictor 331 can predict the current block by referring to samples within the current picture. Depending on the prediction mode, the samples referred to can be located near the current block or can be located at a certain distance away from the current block. In intra prediction, the prediction mode can include at least one non - directional mode and multiple directional modes. The intra predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to neighboring blocks.

[0088] The inter predictor 332 can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter - prediction mode, the motion information can be predicted in units of blocks, sub - blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter - prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter - prediction, neighboring blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter predictor 332 can configure a motion information candidate list based on neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter - prediction can be performed based on various prediction modes, and the information regarding the prediction can include information indicating the inter - prediction mode for the current block.

[0089] The adder 340 can add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter predictor 332 and / or the intra predictor 331) to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed, such as when the skip mode is applied, the prediction block can be used as the reconstructed block.

[0090] The adder 340 can be referred to as a reconstructor or a reconstructed - block generator. The generated reconstructed signal can be used for intra - prediction of the next block to be processed in the current picture, can be output through filtering described later, or can be used for inter - prediction of the next picture. Meanwhile, a luminance mapping with chroma scaling (LMCS) can be applied during the picture decoding process.

[0091] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and send the modified reconstructed picture to the memory 360, specifically to the DPB of the memory 360. Various filtering methods can include de - blocking filtering, sample - adaptive offset, adaptive loop filter, bilateral filter, etc.

[0092] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter - predictor 332. The memory 360 can store the motion information of the blocks from which the motion information in its current picture is derived (or decoded) and / or the motion information of the blocks in the pre - reconstructed picture. The stored motion information can be sent to the inter - predictor 260 to be used as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 360 can store the reconstructed samples of the reconstructed blocks in the current picture and send them to the intra - predictor 331.

[0093] Here, the embodiments described in the filter 260, the inter - predictor 221, and the intra - predictor 222 of the encoding device 200 can also be equally or correspondingly applied to the filter 350, the inter - predictor 332, and the intra - predictor 331 of the decoding device 300, respectively.

[0094] Figure 4 The figure illustrates an image decoding method performed in an image decoding device according to the present disclosure.

[0095] Reference Figure 4 , the filter length of the de - blocking filter can be determined (S400).

[0096] The de - blocking filter can be applied to the boundary (i.e., the block boundary) between adjacent blocks based on the vertical boundary or the horizontal boundary within the current picture. Here, a block can mean a coding unit (CU) or a transform unit (TU). Additionally, one coding unit or transform unit can be composed of one or more sub - blocks, and a block can mean a sub - block.

[0097] Hereinafter, for convenience of explanation, the blocks are referred to as the first block and the second block. Additionally, the first block and the second block in the present disclosure can be referred to as the P - block and the Q - block, respectively. When the boundary between the blocks is a vertical boundary, the first block can mean the top block of the vertical boundary, and the second block can mean the bottom block of the vertical boundary. When the boundary between the blocks is a horizontal boundary, the first block can mean the left block of the horizontal boundary, and the second block can mean the right block of the horizontal boundary.

[0098] The process of determining the filter length of the de - blocking filter can determine whether it corresponds to the boundary between coding units (CU boundary), whether it corresponds to the boundary between transform units (TU boundary), or whether it corresponds to the boundary between sub - blocks (sub - block boundary), and determine the filter length according to the corresponding blocks. The filter length can be divided into the filter length of the de - blocking filter applied to the first block (hereinafter referred to as the P - length) and the filter length of the de - blocking filter applied to the second block (hereinafter referred to as the Q - length), and these lengths can be determined separately.

[0099] Specifically, it can be determined whether it corresponds to a CU boundary by checking whether it corresponds to a picture boundary, a sub-picture boundary, a slice boundary, a tile boundary, or a virtual boundary within a picture. It can be determined whether it corresponds to a TU boundary while moving in units of N pixels in the horizontal and vertical directions within a coding unit. For example, in the case of a luminance component, it can be determined whether it corresponds to a boundary between transform units while moving in units of 4 pixels; and in the case of a chrominance component, it can be determined whether it corresponds to a boundary between transform units while moving in units of 8 pixels. Additionally, it can be determined whether it corresponds to a sub-block boundary of a luminance component. When an inter-frame prediction mode based on sub-blocks (e.g., sub-block merge mode, inter-frame affine mode, etc.) is applied to a block, the block may include one or more sub-blocks. In this case, it can be checked / determined whether it corresponds to a sub-block boundary while moving in units of 8 pixels within the block. That is, the deblocking filter may not be applied to sub-block boundaries in units of 4 pixels. Hereinafter, a method for determining the filter length of the deblocking filter will be described.

[0100] Example 1-A

[0101] The filter length at a block boundary can be determined based on the size of the block. The filter length can be determined for each of a first block and a second block adjacent to each other based on the block boundary.

[0102] Specifically, when the size of the second block is greater than or equal to a first threshold size, the filter length (Q length) of the deblocking filter applied to the second block can be determined to be 7. When the size of the second block is less than the first threshold size and greater than or equal to a second threshold size, the Q length can be determined to be 3. When the size of the second block is less than the second threshold size, the Q length can be determined to be 1. For example, the first threshold size can be 32 and the second threshold size can be 4.

[0103] The predetermined filter length can be updated based on at least one of whether the block is a block compiled in an inter-frame prediction mode based on sub-blocks, the position of the sub-block, or the distance to the block boundary (CU / TU boundary).

[0104] Specifically, when the second block is a block compiled in an inter-frame prediction mode based on sub-blocks (or when the second block includes multiple sub-blocks), the Q length for the second block can be updated to the minimum of the predetermined Q length and 5. Otherwise, the Q length for the second block may not be updated.

[0105] While moving in units of 8 pixels within the second block, it is possible to check whether the position corresponds to the sub-block boundary. In this case, it is possible to check whether the position (pos) corresponds to the TU boundary. When pos corresponds to the TU boundary, the Q length for the second block can be updated to the minimum of a predetermined Q length and 5. When pos does not correspond to the TU boundary, it is possible to check whether the position moved 4 pixels from pos corresponds to the TU boundary. When the position moved 4 pixels corresponds to the TU boundary, the Q length for the second block can be updated to 1. When the position moved 4 pixels does not correspond to the TU boundary, the Q length for the second block can be updated based on at least one of whether the position moved 8 pixels from pos corresponds to the TU boundary or the position of the sub-block to which pos belongs. When the position moved 8 pixels from pos corresponds to the TU boundary or the sub-block to which pos belongs corresponds to the first sub-block (specifically, the right boundary of the first sub-block) or the last sub-block (specifically, the left boundary of the last sub-block) in the block, the Q length for the second block can be updated to 2. Otherwise (that is, the position moved 8 pixels from pos does not correspond to the TU boundary and the sub-block to which pos belongs does not correspond to the first sub-block or the last sub-block in the block), the Q length for the second block can be updated to 3.

[0106] The filter length of the first block can be determined or updated in the same manner as the second block described above, and repeated descriptions are omitted.

[0107] Example 1-B

[0108] The filter length at the block boundary can be determined based on whether it is a block to which multi-reference prediction is applied.

[0109] The multi-reference prediction according to the present disclosure can derive a final prediction block through a weighted sum of multiple reference blocks. Here, the multiple reference blocks can include reference blocks for uni-directional prediction or bi-directional prediction in an inter-frame mode, and one or more additional reference blocks for multi-reference prediction. For example, the additional reference block can be an inter-frame prediction block derived based on the inter-frame mode, or an intra-frame prediction block derived based on the intra-frame mode. The multi-reference prediction according to the present disclosure is not limited to being applied only to blocks compiled in the inter-frame mode. The multi-reference prediction can also be applied to blocks compiled in the intra-frame mode or the IBC (intra-block copy) mode. In this case, the inter-frame prediction block and / or the intra-frame prediction block can be used as the additional reference block.

[0110] The deblocking filter can remove blocking artifacts caused by discontinuities between blocks by filtering the block boundaries. As described above, the filter length can be adjusted based on at least one of the block size, whether the block is coded in a sub-block based inter prediction mode, the position of the sub-block, and the distance from the block boundary. However, since a block to which multi-reference prediction is applied has additional reference blocks in addition to up to two reference blocks for uni-directional or bi-directional prediction, it can be seen that the samples in the block have been sufficiently smoothed. Therefore, the filter length of the deblocking filter may vary depending on whether multi-reference prediction is applied.

[0111] Specifically, it can be checked whether the second block is a block to which multi-reference prediction is applied. When the second block is a block to which multi-reference prediction is applied, it can be checked whether the size of the second block is less than a first threshold size. In this case, regardless of whether the first block is a block to which multi-reference prediction is applied, when the second block is a block to which multi-reference prediction is applied, it can be checked whether the size of the second block is greater than or equal to the first threshold size. Alternatively, when both the first block and the second block are blocks to which multi-reference prediction is applied, it can be limited to checking whether the size of the second block is greater than or equal to the first threshold size. When the second block is not a block to which multi-reference prediction is applied, the filter length can be determined according to the aforementioned Embodiment 1-A.

[0112] When the size of the second block is greater than or equal to the first threshold size, the Q length for the second block can be determined to be 3. When the size of the second block is less than the first threshold size, the Q length for the second block can be determined to be 1. For example, the first threshold size can be 32.

[0113] Based on whether the second block is a block coded in a sub-block based inter prediction mode, a predetermined filter length can be updated. When the second block is a block coded in a sub-block based inter prediction mode (or, when the second block includes multiple sub-blocks), the Q length for the second block can be updated to 1. Otherwise, the Q length for the second block may not be updated.

[0114] In the same manner as the second block described above, the P length of the first block can be determined or updated based on whether the first block is a block to which multi-reference prediction is applied.

[0115] In this way, a block to which multi-reference prediction is applied may have a shorter filter length than a block to which multi-reference prediction is not applied. When the first block or the second block is a block to which multi-reference prediction is applied and has reference blocks in units of sub-blocks, the filter length can be set short because the discontinuity at the sub-block boundary is alleviated. Alternatively, when the first block or the second block is a block to which multi-reference prediction is applied and has reference blocks in units of sub-blocks, the deblocking filter may not be applied to the sub-block boundary.

[0116] Example 1-C

[0117] The filter length at the block boundary can be determined based on whether the block to which overlapping block motion compensation (OBMC) is applied.

[0118] Specifically, it can be checked whether the second block is a block to which OBMC is applied. When the second block is a block to which OBMC is applied, it can be checked whether the size of the second block is smaller than a first threshold size. In this case, regardless of whether the first block is a block to which OBMC is applied, when the second block is a block to which OBMC is applied, it can be checked whether the size of the second block is greater than or equal to the first threshold size. Alternatively, when both the first block and the second block are blocks to which OBMC is applied, it can be limited to checking whether the size of the second block is greater than or equal to the first threshold size. When the second block is not a block to which OBMC is applied, the filter length can be determined according to the foregoing Embodiment 1-A.

[0119] When the size of the second block is greater than or equal to the first threshold size, the Q length for the second block can be determined to be 3. When the size of the second block is smaller than the first threshold size, the Q length for the second block can be determined to be 1. For example, the first threshold size can be 32.

[0120] Based on whether the second block is a block compiled in the sub-block based inter prediction mode, a predetermined filter length can be updated. When the second block is a block compiled in the sub-block based inter prediction mode (alternatively, when the second block includes a plurality of sub-blocks), the Q length for the second block can be updated to 1. Otherwise, the Q length for the second block may not be updated.

[0121] In the same manner as the above second block, the P length of the first block can be determined or updated based on whether the first block is a block to which OBMC is applied.

[0122] In this way, a block to which OBMC is applied can have a shorter filter length than a block to which OBMC is not applied. When the first block or the second block is a block to which OBMC is applied and has a reference block in units of sub-blocks, the filter length can be set short because the discontinuity at the sub-block boundary is mitigated. Alternatively, when the first block or the second block is a block to which OBMC is applied and has a reference block in units of sub-blocks, the deblocking filter may not be applied to the sub-block boundary. The filter length of the deblocking filter can be determined / updated based on a comparison between the filter length for OBMC and the filter length for the deblocking filter. When the filter length for OBMC is less than the filter length for the deblocking filter, the deblocking filter can be applied.

[0123] Reference Figure 4 , the boundary strength (BS) of the deblocking filter can be determined (S410).

[0124] The BS can be determined by considering the characteristics of the first block and the second block adjacent to each other based on the aforementioned block boundaries (CU / TU / sub-block boundaries). The block boundaries can be found while moving in units of N pixels. For example, in the case of the luminance component, the block boundaries can be found while moving in units of 4 pixels, and in the case of the chrominance component, the block boundaries can be found while moving in units of 8 pixels.

[0125] The BS can be determined based on at least one of the prediction modes of the first block and / or the second block or the presence of non-zero transform coefficients. Additionally, when the prediction mode of the first block and / or the second block is an inter-frame mode or an IBC mode, the BS can be determined based on the motion information of the corresponding block. In this case, the block vector (BV) can be used as the motion information for the IBC mode, and at least one of the number of motion vectors, the reference picture, or the motion vectors can be used as the motion information for the inter-frame mode. Hereinafter, the method for determining the BS of the deblocking filter will be described in detail.

[0126] Example 2-A

[0127] It can be checked whether the P block and / or the Q block apply BDPCM (Block Differential Pulse Code Modulation). When both the P block and the Q block are blocks to which BDPCM is applied, the BS can be determined to be 0. Otherwise (i.e., when at least one of the P block or the Q block is a block to which BDPCM is not applied), it can be checked whether the P block and / or the Q block are blocks encoded in the intra-frame prediction mode or the combined inter-frame prediction mode (CIIP mode).

[0128] When the P block and / or the Q block are blocks encoded in the intra-frame mode or the CIIP mode, the BS can be determined to be 2. Otherwise (i.e., when neither the P block nor the Q block is a block encoded in the intra-frame mode and the CIIP mode), it can be checked whether at least one of the block edges corresponds to a transform unit edge, or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0129] When the block edge corresponds to a transform unit edge and at least one of the P block or the Q block has non-zero transform coefficients, the BS can be determined to be 1. Otherwise (i.e., when the block edge does not correspond to a transform unit edge, or neither the P block nor the Q block has non-zero transform coefficients), it can be checked whether at least one of the following conditions 1 to 4 is satisfied.

[0130] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block. Here, each of the P block and the Q block has one of a plurality of predefined prediction modes, and the plurality of prediction modes can include at least one of the intra-frame mode, the inter-frame mode, the IBC mode, or the palette mode.

[0131] (Condition 2) Both the P block and the Q block are blocks compiled in the IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (e.g., 8).

[0132] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of motion vectors used to predict the P block and the number of motion vectors used to predict the Q block are different from each other.

[0133] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (e.g., 8).

[0134] When at least one of the above Conditions 1 to 4 is satisfied, BS can be determined as 1. When none of the above Conditions 1 to 4 is satisfied, BS can be determined as 0.

[0135] A method for determining BS depending on whether Conditions 2 to 4 are satisfied will be described in more detail. When based on a vertical edge, the right sample is q and the left sample is p (or, when based on a horizontal edge, the lower sample is q and the upper sample is p), the block including the sample q is called the Q block, and the block including the sample p is called the P block. The motion vectors of the Q block and the P block can be respectively expressed as (mvQX_x, mvQX_y) and (mvPX_x, mvPX_y), and X represents a list and can be replaced with 0 or 1.

[0136] When both the Q block and the P block are blocks compiled in the IBC mode or blocks compiled in the inter-frame mode, the above Conditions 2 to 4 can be applied. When both the Q block and the P block are blocks compiled in the IBC mode, or the slice including the Q block and / or the P block is a P slice, BS can be determined as follows. That is, when the reference picture used to predict the Q block and the reference picture used to predict the P block are different from each other, BS can be determined as 1. When the reference picture used to predict the block Q and the reference picture used to predict the block P are the same, BS can be determined based on the difference between the block vectors or the motion vectors of the block Q and the block P. When the difference between the block vectors or the motion vectors is greater than or equal to a threshold value (nTh), BS can be determined as 1, and otherwise, BS can be determined as 0. The method for determining BS based on the difference between the motion vectors of two blocks can be expressed as Equation 1 below.

[0137] [Equation 1]

[0138]

[0139] In Equation 1, mvQ0_x and mvQ0_y respectively represent the x-component and y-component of the L0 motion vector of the Q block. mvP0_x and mvP0_y respectively represent the x-component and y-component of the L0 motion vector of the P block. nTh can be used to determine the threshold of the BS. When the difference between mvQ0_x and mvP0_x or the difference between mvQ0_y and mvP0_y is greater than or equal to nTh, the BS can be determined as 1, otherwise it can be determined as 0. That is, when the difference between the motion vectors of the P block and the Q block is greater than or equal to the threshold, the BS can be determined as 1, otherwise the BS can be determined as 0. Here, the difference can mean the absolute difference value, and can be calculated separately for the x-component and y-component of the motion vector.

[0140] When neither the Q block nor the P block is a block coded in the IBC mode and the slice including the Q block and / or the P block is a B slice, the BS can be determined as follows. That is, when the reference picture for predicting the Q block and the reference picture for predicting the P block are different from each other, the BS can be determined as 1. When the reference picture for predicting the Q block and the reference picture for predicting the P block are the same, the BS can be determined based on the difference between the motion vectors of the Q block and the P block. Hereinafter, a method for determining the BS when the reference pictures of the Q block and the P block are different from each other will be described.

[0141] First, it can be determined whether the reference picture of the P block is the same as the reference picture of the Q block. For example, when the L0 reference picture of the P block is the same as the L0 reference picture of the Q block and the L1 reference picture of the P block is the same as the L1 reference picture of the Q block, it can be determined that the reference picture of the P block is the same as the reference picture of the Q block. Alternatively, when the L1 reference picture of the P block is the same as the L0 reference picture of the Q block and the L0 reference picture of the P block is the same as the L1 reference picture of the Q block, it can be determined that the reference picture of the P block is the same as the reference picture of the Q block.

[0142] When the reference pictures of the P block and the Q block are determined to be the same, it can be checked whether the L0 reference picture and the L1 reference picture of the P block are the same. When the L0 reference picture and the L1 reference picture of the P block are different, the BS can be determined based on whether the L0 reference picture of the P block is the same as the L0 reference picture of the Q block. Specifically, when the L0 reference picture of the P block is the same as the L0 reference picture of the Q block, the BS can be determined as in Equation 2 below.

[0143] [Equation 2]

[0144]

[0145] In Equation 2, mvQ0_x and mvQ0_y can respectively represent the x - component and y - component of the L0 motion vector of the Q block. mvP0_x and mvP0_y can respectively represent the x - component and y - component of the L0 motion vector of the P block. mvQ1_x and mvQ1_y can respectively represent the x - component and y - component of the L1 motion vector of the Q block. mvP1_x and mvP1_y can respectively represent the x - component and y - component of the L1 motion vector of the P block. nTh is the threshold used to determine BS. Hereinafter, these can be explained with the same meaning.

[0146] According to Equation 2, when any one of the differences between mvQ0_x and mvP0_x, the difference between mvQ0_y and mvP0_y, the difference between mvQ1_x and mvP1_x, or the difference between mvQ1_y and mvP1_y is greater than or equal to nTh, BS can be determined as 1. Otherwise (i.e., when all of the above differences are less than nTh), BS can be determined as 0.

[0147] That is to say, according to Equation 2, when the difference between the motion vectors in the same direction (L0 or L1) between the P block and the Q block is greater than or equal to the threshold, BS can be determined as 1, and otherwise, BS can be determined as 0. Here, the difference can mean the absolute value of the difference, and can be calculated for each of the x - component and y - component of the motion vector.

[0148] When the L0 reference picture of the P block is different from the L0 reference picture of the Q block, BS can be determined as follows in Equation 3.

[0149] [Equation 3]

[0150]

[0151] According to Equation 3, when any one of the differences between mvQ1_x and mvP0_x, the difference between mvQ1_y and mvP0_y, the difference between mvQ0_x and mvP1_x, or the difference between mvQ0_y and mvP1_y is greater than or equal to nTh, BS can be determined as 1. Otherwise (i.e., when all of the above differences are less than nTh), BS can be determined as 0.

[0152] That is to say, according to Equation 3, when the difference between the motion vectors in different directions between the P block and the Q block is greater than or equal to the threshold, BS can be determined as 1, otherwise, BS can be determined as 0. Here, the difference can mean the absolute value of the difference, and can be calculated for each of the x - component and y - component of the motion vector.

[0153] When the L0 reference picture and the L1 reference picture of the P block are the same, BS can be determined as follows in Equation 4.

[0154] [Equation 4]

[0155]

[0156] According to Equation 4, when the difference between the motion vectors in the same direction between the P block and the Q block is greater than or equal to a threshold, and the difference between the motion vectors in different directions between the P block and the Q block is greater than or equal to a threshold, BS can be determined to be 1. Otherwise, BS can be determined to be 0. This is as described in Reference Equations 2 and 3, and redundant explanations will be omitted.

[0157] Example 2-B

[0158] In addition to up to two reference blocks for uni - directional or bi - directional prediction, a block to which multi - reference prediction is applied may also have additional reference blocks. Here, in order to distinguish up to two reference blocks from additional reference blocks, the up to two reference blocks are referred to as normal reference blocks. When the motion information of the corresponding block is different from that of the neighboring block (or when the difference in motion information between the corresponding block and the neighboring block is greater than a specific threshold), discontinuity may occur between the blocks. Therefore, when determining whether to perform block - boundary filtering on a block to which multi - reference prediction is applied, in addition to specifying the motion information of the normal reference blocks of the corresponding block, the motion information of the specified additional reference blocks may be further considered to determine the discontinuity of the block boundary.

[0159] As an example of multi - reference prediction, when the current block has motion information for bi - directional prediction, two reference blocks (P0, P1) for bi - directional prediction can be derived. Additionally, additional reference blocks (P2, P3) for multi - reference prediction can be derived. This is only an example, and the number of reference blocks for multi - reference prediction can be determined within a predefined maximum number N.

[0160] In this way, the current block has four reference blocks (P0 to P3), and the final predicted block of the current block can be generated by the weighted sum of the four reference blocks. In this case, the weight (W2) applied to the P2 reference block and the weight (W3) applied to the P3 block can be used. For example, the final predicted block (P) of the current block can be generated as follows in Equation 5.

[0161] [Equation 5]

[0162] P’ = (P0 + P1) / 2

[0163] P” = W2 * P2 + (1 - W2) * P’

[0164] P = W3 * P3 + (1 - W3) * P”

[0165] In the process of generating the final predicted block of the current block, since additional reference blocks for multi-reference prediction are used, in the process of determining the BS of the deblocking filter, in addition to the information about the reference blocks P0 and P1, at least one of the information about the additional reference blocks P2 and P3 or the information about multi-reference prediction can be further considered. Here, the information about the reference block can include at least one of the motion vector for specifying the corresponding reference block, the reference picture to which the corresponding reference block belongs, or the prediction mode used to derive the corresponding reference block. The information about multi-reference prediction can include at least one of the number of reference blocks used for multi-reference prediction, the motion vector for multi-reference prediction, the reference picture for multi-reference prediction, the index of the reference picture, or the output order (POC) of the reference picture.

[0166] For example, the BS can be determined based on the number of reference blocks used to predict the Q block and / or the P block. This can be applied when multi-reference prediction is applied to at least one of the Q block or the P block. However, it is not limited thereto, and it can also be applied when multi-reference prediction is not applied to the Q block and the P block.

[0167] Specifically, the number of reference blocks used to predict the Q block is referred to as numPred_Q, and the number of reference blocks used to predict the P block is referred to as numPred_P. When numPred_Q and numPred_P are different, the BS can be determined to be 1. Otherwise, the BS can be determined based on the difference between the motion vectors of the Q block and the P block. That is, when numPred_Q and numPred_P are the same, the BS can be determined based on whether the difference between the motion vectors of the Q block and the P block is less than a threshold. This is described in detail in condition 4 of Embodiment 2-A.

[0168] For example, assume that the Q block performs bi-directional prediction and has two additional reference blocks for multi-reference prediction, and the P block performs bi-directional prediction but does not perform multi-reference prediction. In this case, the number of reference blocks used to predict the Q block (numPred_Q) becomes 4, and the number of reference blocks used to predict the P block (numPred_P) becomes 2. In this way, when numPred_Q and numPred_P are different, the BS can be determined to be 1.

[0169] For example, it can be checked whether the P block and / or the Q block is a block to which BDPCM is applied. When both the P block and the Q block are blocks to which BDPCM is applied, the BS can be determined to be 0. Otherwise, it can be checked whether the P block and / or the Q block is a block compiled in the intra mode or the CIIP mode (combined inter-intra prediction mode).

[0170] When the P block and / or the Q block are blocks coded in the intra mode or the CIIP mode, the BS can be determined to be 2. Otherwise, it can be checked whether at least one of the block edges corresponds to a transform unit edge, or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0171] When the block edge corresponds to a transform unit edge and at least one of the P block or the Q block has non-zero transform coefficients, the BS can be determined to be 1. Otherwise, it can be checked whether at least one of the following conditions 1 to 4 is satisfied.

[0172] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0173] (Condition 2) Both the P block and the Q block are blocks coded in the IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0174] (Condition 3) The reference pictures of the P block and the Q block are different from each other, or the number of reference blocks used to predict the P block and the number of reference blocks used to predict the Q block are different from each other.

[0175] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0176] When at least one of the above conditions 1 to 4 is satisfied, the BS can be determined to be 1. When none of the above conditions 1 to 4 is satisfied, the BS can be determined to be 0.

[0177] For example, when multi-reference prediction is applied to the Q block and / or the P block, there may be additional reference blocks for multi-reference prediction. In this case, the prediction mode used to derive the additional reference blocks can be considered to determine the BS. Assume that multi-reference prediction is applied to the Q block coded in the inter mode. In this case, when the Q block uses a prediction block derived based on the intra mode or the IBC mode as an additional reference block for multi-reference prediction, the BS can be determined to be 2. Alternatively, assume that multi-reference prediction is applied to the P block coded in the intra mode. In this case, when the P block uses a prediction block derived based on the inter mode as an additional reference block for multi-reference prediction, the BS can be determined to be 2.

[0178] For example, when both the Q block and the P block are blocks to which multi-reference prediction is applied, the BS can be determined to be 0. The BS can be determined based on whether the information on multi-reference prediction between the Q block and the P block is the same. When the information on multi-reference prediction between the Q block and the P block is the same, the BS can be determined to be 0. Otherwise, the BS may not be determined to be 0. In this way, the complexity can be reduced by determining the BS to be 0.

[0179] When multi-reference prediction is applied to a Q block and / or a P block, the BS can be determined without performing a comparison of motion information between the Q block and the P block (e.g., whether the motion information is the same). Here, the motion information means a motion vector, a reference picture, the number of reference blocks, etc., as described above. This is because, in the case of multi-reference prediction, additional weights (W2, W3) can be used for the weighted sum with additional reference blocks, and even if the current block and an adjacent block have the same motion vector, the luminance of the prediction block for the current block may vary due to this weight.

[0180] Alternatively, when the additional weights (W2, W3) are less than a specific threshold, the difference between the prediction block before applying multi-reference prediction and the prediction block after applying multi-reference prediction may not be large. In this case, the application of multi-reference prediction may not affect the determination of the BS. Therefore, the above-described Example 2-A or Example 2-B can be selected according to the additional weights, and the BS can be determined based on the selected method. For example, when the additional weights (W2, W3) are less than a specific threshold, the BS can be determined based on Example 2-A. When the additional weights (W2, W3) are greater than or equal to a specific threshold, the BS can be determined based on Example 2-B. However, this is merely an example, and at least one of the information regarding multi-reference prediction can be further considered in addition to the additional weights. Depending on the additional weights and the information regarding multi-reference prediction, the above-described Example 2-A or Example 2-B can be selected, and the BS can be determined based on the selected method.

[0181] The above BS can be determined as any integer in the range of 0 to k, where k can mean the maximum value among the available BSs. When the additional weight is greater than or equal to a specific threshold, k can have a larger value than when multi-reference prediction is not applied, and the range of available BSs can be extended.

[0182] Alternatively, multiple weight intervals for multi-reference prediction can be defined, and the range of BSs available for each interval (i.e., the value of k) can be defined. Any one of the multiple weight intervals can have a different k value from the others. The weight interval to which the additional weight belongs can be determined, and the BS can be determined within the range of BSs corresponding to that weight interval.

[0183] Example 2-C

[0184] When a Q block and / or a P block is a block to which multi-reference prediction is applied, in addition to the reference blocks for uni-directional or bi-directional prediction (hereinafter referred to as normal reference blocks), the block can also have additional reference blocks. In this case, the motion information of the normal reference blocks and the additional reference blocks can be compared with each other to determine the BS.

[0185] In this embodiment, when multi-reference prediction is applied to a Q block and / or a P block, the boundary strength of the deblocking filter may change during this process. The boundary strength can be determined based on whether the reference picture to which the normal reference block belongs is the same as the reference picture to which the additional reference block belongs. The comparison between reference pictures can be applied between the normal reference block and the additional reference block of the Q block. The comparison between reference pictures can be applied between the normal reference block and the additional reference block of the P block. The comparison between reference pictures can be applied between the additional reference block of the Q block and the normal reference block of the P block. The comparison between reference pictures can be applied between the additional reference block of the P block and the normal reference block of the Q block. The comparison between reference pictures can be applied between the additional reference block of the P block and the additional reference block of the Q block.

[0186] In addition, the BS can be determined based on the difference between the motion vector indicating the normal reference block and the motion vector indicating the additional reference block. The difference between the motion vectors of the normal reference block and the additional reference block of the Q block can be calculated. The difference between the motion vectors of the normal reference block and the additional reference block of the P block can be calculated. The difference between the motion vectors of the additional reference block of the Q block and the normal reference block of the P block can be calculated. The difference between the motion vectors of the additional reference block of the Q block and the additional reference block of the P block can be calculated. The difference between the motion vectors of the additional reference block of the P block and the normal reference block of the Q block can be calculated. The difference between the motion vectors of the additional reference block of the P block and the additional reference block of the Q block can be calculated. The BS can be determined based on whether the difference between the motion vectors is less than a predefined threshold.

[0187] Assume that each of the Q block and the P block performs bidirectional prediction and has two additional reference blocks for multi-reference prediction, and in this case, a method for determining the BS will be described. Hereinafter, for convenience of explanation, the picture to which the normal reference block belongs is referred to as the normal reference picture, and the motion vector indicating the normal reference block is referred to as the normal motion vector. In addition, the picture to which the additional reference block belongs is referred to as the additional reference picture, and the motion vector indicating the additional reference block is referred to as the additional motion vector.

[0188] First, it can be determined whether the normal reference picture of the P block and the normal reference picture of the Q block are the same. For example, when the L0 normal reference picture of the P block and the L0 normal reference picture of the Q block are the same and the L1 normal reference picture of the P block and the L1 normal reference picture of the Q block are the same, the normal reference picture of the P block and the normal reference picture of the Q block can be determined to be the same. Alternatively, when the L1 normal reference picture of the P block and the L0 normal reference picture of the Q block are the same and the L0 normal reference picture of the P block and the L1 normal reference picture of the Q block are the same, the normal reference picture of the P block and the normal reference picture of the Q block can be determined to be the same.

[0189] When it is determined that the normal reference picture of the P block is the same as that of the Q block, it is possible to check whether the L0 normal reference picture and the L1 normal reference picture of the P block are the same. When the L0 normal reference picture and the L1 normal reference picture of the P block are different, the BS can be determined based on whether the L0 normal reference picture of the P block is the same as the L0 normal reference picture of the Q block. Specifically, when the L0 normal reference picture of the P block is the same as the L0 normal reference picture of the Q block, the BS can be determined as in Equation 6 below.

[0190] [Equation 6]

[0191]

[0192] In Equation 6, mvQ0_x and mvQ0_y can respectively represent the x component and the y component of the L0 normal motion vector of the Q block. mvP0_x and mvP0_y can respectively represent the x component and the y component of the L0 normal motion vector of the P block. mvQ1_x and mvQ1_y can respectively represent the x component and the y component of the L1 normal motion vector of the Q block. mvP1_x and mvP1_y can respectively represent the x component and the y component of the L1 normal motion vector of the P block. nTh can be a threshold for determining the BS. According to Equation 6, when any one of the differences between mvQ0_x and mvP0_x, the difference between mvQ0_y and mvP0_y, the difference between mvQ1_x and mvP1_x, or the difference between mvQ1_y and mvP1_y is greater than or equal to nTh, the BS can be determined as 1. Otherwise (that is, when all the above differences are less than nTh), the BS can be determined as 0.

[0193] That is to say, according to Equation 6, when the difference between the normal motion vectors of the P block and the Q block in the same direction (L0 or L1) is greater than or equal to the threshold, the BS can be determined as 1, otherwise, the BS can be determined as 0. Here, the difference can mean the absolute value of the difference, and can be calculated for each of the x component and the y component of the motion vector.

[0194] When the L0 normal reference picture of the P block is different from the L0 normal reference picture of the Q block, the BS can be determined as in Equation 7 below.

[0195] [Equation 7]

[0196]

[0197] According to Equation 7, when any one of the differences between mvQ1_x and mvP0_x, the difference between mvQ1_y and mvP0_y, the difference between mvQ0_x and mvP1_x, or the difference between mvQ0_y and mvP1_y is greater than or equal to nTh, BS can be determined as 1. Otherwise (i.e., when all of the above differences are less than nTh), BS can be determined as 0.

[0198] That is to say, according to Equation 7, when the difference between the normal motion vectors in different directions between the P block and the Q block is greater than or equal to the threshold, BS can be determined as 1, otherwise, BS can be determined as 0. Here, the difference can mean the absolute value of the difference, and can be calculated for each of the x and y components of the motion vector.

[0199] When the L0 normal reference picture and the L1 normal reference picture of the P block are the same, BS can be determined as follows in Equation 8.

[0200] [Equation 8]

[0201]

[0202] According to Equation 8, when the difference between the normal motion vectors in the same direction between the P block and the Q block is greater than or equal to the threshold, and the difference between the normal motion vectors in different directions between the P block and the Q block is greater than or equal to the threshold, BS can be determined as 1. Otherwise, BS can be determined as 0.

[0203] Then, at least one of the reference pictures or the motion vectors between the normal reference block and the additional reference block of the Q block can be compared. In addition, at least one of the reference pictures or the motion vectors between the additional reference block of the Q block and the normal reference block of the P block can be compared. The comparison process here can be repeatedly executed the same number of times as the number of additional reference blocks of the Q block. Through this, the predetermined BS can be updated.

[0204] Specifically, it can be determined whether the normal reference picture and the additional reference picture of the Q block are the same. When the L0 normal reference picture of the Q block is different from the additional reference picture of the Q block, or the L1 normal reference picture of the Q block is different from the additional reference picture of the Q block, it can be determined that the normal reference picture and the additional reference picture of the Q block are different.

[0205] It can be determined whether the normal reference picture of the P block and the additional reference picture of the Q block are the same. When the L0 normal reference picture of the P block is the same as the additional reference picture of the Q block, or the L1 normal reference picture of the P block is the same as the additional reference picture of the Q block, it can be determined that the normal reference picture of the P block and the additional reference picture of the Q block are the same.

[0206] When the normal reference picture of the P block is the same as the additional reference picture of the Q block (specifically, when the L0 normal reference picture of the P block is the same as the additional reference picture of the Q block), BS can be determined as in Equation 9 below.

[0207] [Equation 9]

[0208]

[0209] In Equation 9, mvQi_x and mvQi_y respectively represent the x - component and y - component of the additional motion vector of the Q block. mvP0_x and mvP0_y respectively represent the x - component and y - component of the L0 normal motion vector of the P block. nTh1 can represent the threshold for determining BS. When the difference between mvQi_x and mvP0_x or the difference between mvQi_y and mvP0_y is greater than or equal to nTh1, BS can be determined as 1. Otherwise (i.e., when both differences are less than nTh1), BS can be determined as 0.

[0210] When the normal reference picture of the P block is the same as the additional reference picture of the Q block (specifically, when the L1 normal reference picture of the P block is the same as the additional reference picture of the Q block), BS can be determined as in Equation 10 below.

[0211] [Equation 10]

[0212]

[0213] In Equation 10, mvQi_x and mvQi_y respectively represent the x - component and y - component of the additional motion vector of the Q block. mvP1_x and mvP1_y can respectively represent the x - component and y - component of the L1 normal motion vector of the P block. nTh1 represents the threshold for determining BS. When the difference between mvQi_x and mvP1_x or the difference between mvQi_y and mvP1_y is greater than or equal to nTh1, BS can be determined as 1. Otherwise (i.e., when both differences are less than nTh1), BS can be determined as 0.

[0214] On the other hand, when the normal reference picture of the P block and the additional reference picture of the Q block are determined to be different from each other, BS can be determined as 1.

[0215] Then, at least one of the reference pictures or motion vectors between the normal reference block of the P block and the additional reference block can be compared. Additionally, at least one of the reference pictures or motion vectors between the additional reference block of the P block and the normal reference block of the Q block can be compared. The comparison process here can be repeated the same number of times as the number of additional reference blocks of the Q block. By this, the predetermined BS can be updated.

[0216] Specifically, it can be determined whether the normal reference picture and the additional reference picture of the P block are the same. When the L0 normal reference picture of the P block is different from the additional reference picture of the P block, or the L1 normal reference picture of the P block is different from the additional reference picture of the P block, it can be determined that the normal reference picture and the additional reference picture of the P block are not the same.

[0217] When the normal reference picture and the additional reference picture of the P block are not the same, it can be determined whether the normal reference picture of the Q block and the additional reference picture of the P block are the same. When the L0 normal reference picture of the Q block is the same as the additional reference picture of the P block, or the L1 normal reference picture of the Q block is the same as the additional reference picture of the P block, the normal reference picture of the Q block and the additional reference picture of the P block can be determined to be the same.

[0218] When the normal reference picture of the Q block is the same as the additional reference picture of the P block (specifically, when the L0 normal reference picture of the Q block is the same as the additional reference picture of the P block), BS can be determined as in the following Equation 11.

[0219] [Equation 11]

[0220]

[0221] In Equation 11, mvQ0_x and mvQ0_y respectively represent the x component and the y component of the L0 normal motion vector of the Q block. mvPi_x and mvPi_y respectively represent the x component and the y component of the additional motion vector of the P block. nTh2 can represent the threshold for determining BS. When the difference between mvQ0_x and mvPi_x or the difference between mvQ0_y and mvPi_y is greater than or equal to nTh2, BS can be determined to be 1. Otherwise (that is, when the difference between the two is less than nTh2), BS can be determined to be 0.

[0222] When the normal reference picture of the Q block is the same as the additional reference picture of the P block (specifically, when the L1 normal reference picture of the Q block is the same as the additional reference picture of the P block), BS can be determined as in the following Equation 12.

[0223] [Equation 12]

[0224]

[0225] In Equation 12, mvQ1_x and mvQ1_y respectively represent the x-component and y-component of the L1 normal motion vector of the Q block. mvPi_x and mvPi_y respectively represent the x-component and y-component of the additional motion vector of the P block. nTh2 can represent the threshold for determining the BS. When the difference between mvQ1_x and mvPi_x or the difference between mvQ1_y and mvPi_y is greater than or equal to nTh2, the BS can be determined as 1. Otherwise (i.e., when the difference between the two is less than nTh2), the BS can be determined as 0.

[0226] On the other hand, when the normal reference picture of the Q block and the additional reference picture of the P block are determined to be different from each other, the BS can be determined as 1.

[0227] The BS can be determined based on at least one of the above Equations 6 to 12, and the corresponding BS can be determined as the final BS.

[0228] Example 2-D

[0229] The present disclosure defines an extended range for the BS of the deblocking filter, so as to be able to apply the deblocking filter more precisely. The present disclosure can determine the BS as any value within the extended range, and can subdivide the determination process of the deblocking filter for each step. According to the present disclosure, a range of 0 to 3 can be defined for the BS of the deblocking filter, and the BS can be determined as any value within this range.

[0230] Specifically, it can be checked whether the P block and / or the Q block is a block to which BDPCM is applied. When both the P block and the Q block are blocks to which BDPCM is applied, the BS can be determined as 0. Otherwise, it can be checked whether the P block and / or the Q block is a block compiled in the intra mode or the CIIP mode (combined inter-intra prediction mode).

[0231] When the P block and / or the Q block is a block compiled in the intra coding mode or the CIIP mode, the BS can be determined as 3. Otherwise, it can be checked whether at least one of the block edges corresponds to the transform unit edge, or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0232] When the block edge corresponds to the transform unit edge and at least one of the P block or the Q block has non-zero transform coefficients, the BS can be determined as 2. Otherwise, it can be checked whether at least one of the following Conditions 1 to 3 is satisfied.

[0233] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0234] (Condition 2) Both the P block and the Q block are blocks compiled in the IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold (for example, 8).

[0235] (Condition 3) The reference picture of the P block and the reference picture of the Q block are different from each other, or the number of motion vectors used to predict the P block and the number of motion vectors used to predict the Q block are different from each other.

[0236] When at least one of the above Conditions 1 to 3 is satisfied, BS can be determined to be 2. When none of the above Conditions 1 to 3 are satisfied, BS can be determined to be 1 or 0. When none of the above Conditions 1 to 3 are satisfied, it can be additionally checked whether the following Condition 4 is satisfied.

[0237] (Condition 4)The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold value (for example, 8).

[0238] When Condition 4 is satisfied, BS can be determined to be 1. When Condition 4 is not satisfied, BS can be determined to be 0.

[0239] Example 2-E

[0240] According to the present disclosure, a range of 0 to 4 can be defined for BS of the deblocking filter, and BS can be determined to be any value within this range.

[0241] Specifically, it can be checked whether the P block and / or the Q block is a block to which BDPCM is applied. When both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined to be 0. Otherwise, it can be checked whether the P block and / or the Q block is a block compiled in the intra mode or the CIIP mode (combined inter-intra prediction mode).

[0242] When the P block and / or the Q block is a block compiled in the intra mode or the CIIP mode, BS can be determined to be 4. Otherwise, it can be checked whether at least one of the block edges corresponds to a transform unit edge, or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0243] When the block edge corresponds to the transform unit edge and at least one of the P block or the Q block has non-zero transform coefficients, BS can be determined to be 3. Otherwise, it can be checked whether the following Condition 1 is satisfied.

[0244] (Condition 1)The prediction mode of the P block is different from the prediction mode of the Q block.

[0245] When Condition 1 is satisfied, BS can be determined to be 3. When Condition 1 is not satisfied, BS can be determined to be 2, 1, or 0. When Condition 1 is not satisfied, it can be additionally checked whether the following Condition 2 is satisfied.

[0246] Condition 2: Both the P block and the Q block are blocks compiled in the IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0247] When the above Condition 2 is satisfied, the BS can be determined to be 2. When the above Condition 2 is not satisfied, the BS can be determined to be 2, 1, or 0. When the above Condition 2 is not satisfied, it can be additionally checked whether the following Condition 3 is satisfied.

[0248] Condition 3: The reference pictures of the P block and the Q block are different from each other, or the number of motion vectors used to predict the P block is different from the number of motion vectors used to predict the Q block.

[0249] When the above Condition 3 is satisfied, the BS can be determined to be 2. When the above Condition 3 is not satisfied, the BS can be determined to be 1 or 0. When the above Condition 3 is not satisfied, it can be additionally checked whether the following Condition 4 is satisfied.

[0250] Condition 4: The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0251] When the above Condition 4 is satisfied, the BS can be determined to be 1. When the above Condition 4 is not satisfied, the BS can be determined to be 0.

[0252] Example 2-F

[0253] The present disclosure relates to a method for determining a deblocking filter BS when applying OBMC, by which subjective image quality can be improved. When applying OBMC, the motion information of adjacent blocks is used to smooth the boundaries between blocks (e.g., CU boundaries, sub-block boundaries), so that the discontinuity between blocks can be removed. Therefore, when the deblocking filter is applied to blocks to which OBMC has been applied, the boundaries between blocks may be overly smoothed.

[0254] Specifically, it can be checked whether the P block and / or the Q block is a block to which BDPCM or OBMC has been applied. For example, when both the P block and the Q block are blocks to which OBMC has been applied, the BS can be determined to be 0. Otherwise (i.e., when at least one of the P block and the Q block is a block to which OBMC has not been applied), based on whether the P block and / or the Q block is a block compiled in the intra mode or the CIIP mode, the BS can be determined to be 2, 1, or 0. Alternatively, regardless of whether the P block is a block to which OBMC has been applied, when the Q block is a block to which OBMC has been applied, the BS can be determined to be 0. Otherwise (i.e., when the Q block is a block to which OBMC has not been applied), based on whether the P block and / or the Q block is a block compiled in the intra mode or the CIIP mode, the BS can be determined to be 2, 1, or 0.

[0255] When the P block and / or the Q block are blocks encoded in the intra mode or the CIIP mode, the BS can be determined to be 2. Otherwise (i.e., when neither the P block nor the Q block is a block encoded in the intra mode and the CIIP mode), it can be checked whether at least one of the block edges corresponds to a transform unit edge, or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0256] When the block edge corresponds to a transform unit edge and at least one of the P block or the Q block has non-zero transform coefficients, the BS can be determined to be 1. Otherwise (i.e., when the block edge does not correspond to a transform unit edge, or neither the P block nor the Q block has non-zero transform coefficients), it can be checked whether at least one of the following conditions 1 to 4 is satisfied.

[0257] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block. Here, each of the P block and the Q block has one of a plurality of predefined prediction modes, and the plurality of prediction modes can include at least one of the intra mode, the inter mode, the IBC mode, or the palette mode.

[0258] (Condition 2) Both the P block and the Q block are blocks encoded in the IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0259] (Condition 3) The reference pictures of the P block and the Q block are different from each other, or the number of motion vectors used to predict the P block and the number of motion vectors used to predict the Q block are different from each other.

[0260] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0261] When at least one of the above conditions 1 to 4 is satisfied, the BS can be determined to be 1. When none of the above conditions 1 to 4 is satisfied, the BS can be determined to be 0. The method for determining the BS depending on whether the above conditions 2 to 4 are satisfied is as described in Example 2-A.

[0262] Example 2-G

[0263] The present disclosure relates to a method for determining a deblocking filter BS when applying OBMC, by which the subjective image quality can be improved. When applying OBMC, the motion information of adjacent blocks is used to smooth the boundaries between blocks (e.g., CU boundaries, sub-block boundaries), so that the discontinuities between blocks can be removed. Therefore, when the deblocking filter is applied to the blocks to which OBMC is applied, the boundaries between blocks may be over-smoothed.

[0264] The present disclosure defines an extended range of BS for the deblocking filter, enabling more precise application of the deblocking filter. The present disclosure can determine BS as any value within the extended range and can subdivide the determination process of the deblocking filter for each step. According to the present disclosure, a range of 0 to 4 can be defined for BS of the deblocking filter, and BS can be determined as any value within this range.

[0265] Specifically, it can be checked whether the P block and / or the Q block are blocks to which BDPCM is applied. When both the P block and the Q block are blocks to which BDPCM is applied, BS can be determined as 0. Otherwise, it can be checked whether the P block and / or the Q block are blocks compiled in the intra mode or the CIIP mode (combined inter-intra prediction mode).

[0266] When the P block and / or the Q block are blocks compiled in the intra mode or the CIIP mode, BS can be determined as 4. Otherwise, it can be checked whether at least one of the block edges corresponds to a transform unit edge, or whether at least one of the P block or the Q block has non-zero transform coefficients.

[0267] When the block edge corresponds to a transform unit edge and at least one of the P block or the Q block has non-zero transform coefficients, BS can be determined as 3. Otherwise, it can be checked whether the following condition 1 is satisfied.

[0268] (Condition 1) The prediction mode of the P block is different from the prediction mode of the Q block.

[0269] When the above condition 1 is satisfied, BS can be determined as 3. When the above condition 1 is not satisfied, BS can be determined as 2, 1, or 0. When the above condition 1 is not satisfied, it can be additionally checked whether the following condition 2 is satisfied.

[0270] (Condition 2) Both the P block and the Q block are blocks compiled in the IBC mode, and the difference between the block vectors of the P block and the Q block is greater than or equal to a threshold value (e.g., 8).

[0271] When the above condition 2 is satisfied, BS can be determined as 2. When the above condition 2 is not satisfied, BS can be determined as 2, 1, or 0. When the above condition 2 is not satisfied, it can be additionally checked whether the following condition 3 is satisfied.

[0272] (Condition 3) Neither the P block nor the Q block is a block to which OBMC is applied, and the reference picture of the P block is different from the reference picture of the Q block. Alternatively, neither the P block nor the Q block is a block to which OBMC is applied, and the number of motion vectors used for P block prediction is different from the number of motion vectors used for Q block prediction. Alternatively, the Q block is not a block to which OBMC is applied, and the reference picture of the P block is different from the reference picture of the Q block. Alternatively, the Q block is not a block to which OBMC is applied, and the number of motion vectors used for P block prediction is different from the number of motion vectors used for Q block prediction.

[0273] When the above Condition 3 is satisfied, BS can be determined to be 2. When the above Condition 3 is not satisfied, BS can be determined to be 1 or 0. When the above Condition 3 is not satisfied, it can be additionally checked whether the following Condition 4 is satisfied.

[0274] (Condition 4) The difference between the motion vectors of the P block and the Q block is greater than or equal to a threshold (e.g., 8).

[0275] When the above Condition 4 is satisfied, BS can be determined to be 1. When the above Condition 4 is not satisfied, BS can be determined to be 0.

[0276] Reference Figure 4 , a deblocking filter with a predetermined type and strength can be determined based on the filter length of the deblocking filter and BS (S420).

[0277] Specifically, based on at least one of the sample changes within the block or the sample changes at the edge between the blocks, it can be determined whether the edge corresponds to an actual edge of the block or an edge caused by block artifacts. Based on this determination, the type and / or strength of the deblocking filter can be determined. As an example of the type of deblocking filter, a short filter, a long filter, etc. can be used. As an example of the strength of the deblocking filter, a strong filter, a weak filter, etc. can be used.

[0278] The above deblocking filter can be determined based on a predetermined variable (e.g., β, t C ). The predetermined variable can be compared with at least one of the sample changes within the Q block and the P block or the sample changes at the edge, so that the type and / or strength of the deblocking filter can be determined.

[0279] The variables β and t can be calculated based on the offsets for each variable (e.g., qpOffset, sh_luma_beta_offset_div2, sh_luma_tc_offset_div2) and the quantization parameter (Qp) of the Q / P block C . For example, the variables β and t can be calculated as shown in Equation 13 and Table 1 C .

[0280] [Equation 13]

[0281]

[0282]

[0283]

[0284]

[0285]

[0286] [Table 1]

[0287]

[0288] In Equation 13, Qp Q and Qp P respectively denote the quantization parameters of the Q block and the P block. In Equation 13, qpOffset can be derived as shown in Table 2 below.

[0289] [Table 2]

[0290]

[0291] The semantics of the above syntax elements (sps_ladf_qp_offset, sh_luma_beta_offset_div2, sh_luma_tc_offset_div2) are shown in Table 3 below.

[0292] [Table 3]

[0293]

[0294] When the range of BS is changed, the variables β and t C can be changed as follows. For example, when defining the range of 0 to 3 for BS of the deblocking filter, the variables β and t C can be changed as in Equation 14. When defining the range of 0 to 2, the variables of Q multiplied by BS become 2, 0, -2. On the other hand, when defining the range of 0 to 3, it can be applied by changing the variables of Q multiplied by BS to 3, 1, -1, -3.

[0295] [Equation 14]

[0296]

[0297]

[0298]

[0299]

[0300]

[0301] Alternatively, when defining a range of 0 to 4 for BS of the deblocking filter, the variables β and t C can be changed as in Equation 15. When defining a range of 0 to 4, it can be applied by multiplying the variable of Q by BS and changing it to 4, 2, 0, -2, -4.

[0302] [Equation 15]

[0303]

[0304]

[0305]

[0306]

[0307]

[0308] Alternatively, when defining a range of 0 to 3 or 0 to 4 for BS of the deblocking filter, the variables β and t C can be changed as in Equation 16. When defining a range of 0 to 3 or 0 to 4, it can be applied by multiplying the change amount of Q by BS and changing it to 3, 2, 1, 0 or 4, 3, 2, 1, which are the same as the value of BS.

[0309] [Equation 16]

[0310]

[0311]

[0312]

[0313]

[0314]

[0315] As described above, the value of Q in Table 1 can be adjusted according to the value of BS. Alternatively, the mapping table between Q and the variables β and t C can be adjusted more finely.

[0316] Reference Figure 4 , the current picture can be updated by applying the deblocking filter to the block boundaries within the current picture (S430).

[0317] Figure 5 FIG. illustrates a schematic configuration of a decoding apparatus (300) that executes an image decoding method according to the present disclosure.

[0318] Reference Figure 5 , the decoding device (300) may include a filter length determiner (500), a boundary strength determiner (510), a filter determiner (520), and a filter applicator (530).

[0319] The filter length determiner (500) may determine the filter length of the deblocking filter. The filter length determiner (500) may determine the filter length of the deblocking filter based on the reference Figure 4 described filter length determination method, and any repeated description will be omitted here.

[0320] The boundary strength determiner (510) may determine the boundary strength (BS) of the deblocking filter. The boundary strength determiner (510) may determine the boundary strength (BS) of the deblocking filter based on the reference Figure 4 described boundary strength determination method, and any repeated description will be omitted here.

[0321] The filter determiner (520) may determine a deblocking filter having a predetermined type and strength based on the filter length and BS of the deblocking filter. Here, the deblocking filter may be determined based on predetermined variables (e.g., β, t C ), and the method for deriving the variables is as described in the reference Figure 4 stated.

[0322] The filter applicator (530) may update the current picture by applying the deblocking filter to the block boundary within the current picture.

[0323] Figure 6 The figure illustrates an image encoding method performed in the encoding device (200) according to the present disclosure.

[0324] Reference Figure 6 , the filter length of the deblocking filter may be determined (S600). The method for determining the filter length of the deblocking filter is as described in the reference Figure 4 stated.

[0325] Reference Figure 6 , the boundary strength (BS) of the deblocking filter may be determined (S610). The BS determination method is as described in the reference Figure 4 stated.

[0326] Reference Figure 6 , based on the filter length and BS of the deblocking filter, a deblocking filter having a predetermined type and strength may be determined (S620). Here, the deblocking filter may be determined based on predetermined variables (e.g., β, t C ), and the method for deriving the variables is as described in the reference Figure 4 stated.

[0327] Reference Figure 6 ,the deblocking filter can be applied to the block boundaries within the current picture to modify the current picture (S630). The modified current picture can be stored in the memory (270) within the encoding device (200) and can be used as a reference for inter prediction of another picture.

[0328] Figure 7 FIG. shows a schematic configuration of an encoding device (200) that executes an image encoding method according to the present disclosure.

[0329] Reference Figure 7 ,the encoding device (200) may include a filter length determiner (700), a boundary strength determiner (710), a filter determiner (720), and a filter applicator (730).

[0330] The filter length determiner (700) may determine the filter length of the deblocking filter. The filter length determiner (700) may determine the filter length of the deblocking filter based on the filter length determination method described in Reference Figure 4 and any repeated description will be omitted here.

[0331] The boundary strength determiner (710) may determine the boundary strength (BS) of the deblocking filter. The boundary strength determiner (710) may determine the BS of the deblocking filter based on the BS determination method described in Reference Figure 4 and the repeated description will be omitted here.

[0332] The filter determiner (720) may determine a deblocking filter having a predetermined type and strength based on the filter length and BS of the deblocking filter. Here, the deblocking filter may be determined based on a predetermined variable (e.g., β, t C ), and the method of deriving the variable is as described in Reference Figure 4 .

[0333] The filter applicator (730) may modify the current picture by applying the deblocking filter to the block boundaries in the current picture.

[0334] In the above embodiments, the method is described as a series of steps or blocks based on a flowchart, but the corresponding embodiments are not limited to the order of the steps, and some steps may occur simultaneously or in a different order from other steps as described above. Additionally, those skilled in the art can understand that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of the present disclosure.

[0335] The above method according to an embodiment of the present disclosure can be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure can be included in a device that performs image processing, such as a TV, a computer, a smart phone, a set-top box, a display device, etc.

[0336] In the present disclosure, when the embodiment is implemented as software, the above method can be implemented as a module (process, function, etc.) that performs the above functions. The module can be stored in a memory and can be executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various well-known means. The processor can include an application specific integrated circuit (ASIC), another chipset, logic circuits, and / or a data processing device. The memory can include a read only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. In other words, the embodiments described herein can be executed by being implemented on a processor, a microprocessor, a controller, or a chip. For example, each functional unit shown in each drawing can be executed by being implemented on a computer, a processor, a microprocessor, a controller, or a chip. In this case, the information for implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.

[0337] In addition, the decoding device and the encoding device applying the embodiments of the present disclosure can be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital cinema video device, a surveillance camera, a video session device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a device for providing a video on demand (VoD) service, an over-the-top (OTT) device, a device for providing an Internet streaming service, a three-dimensional (3D) video device, a virtual reality (VR) device, an augmented reality (AR) device, a videophone video device, a vehicle terminal (e.g., a vehicle (including an autonomous vehicle) terminal, an aircraft terminal, a ship terminal, etc.), and a medical video device, etc., and can be used to process video signals or data signals. For example, an over-the-top (OTT) device can include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smart phone, a tablet, a digital video recorder (DVR), and so on.

[0338] In addition, the processing method according to an embodiment of the present disclosure can be generated in the form of a program executable by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to an embodiment of the present disclosure can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices that store computer-readable data. The computer-readable recording medium may include, for example, Blu-ray Disc (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical media storage devices. In addition, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (e.g., transmitted via the Internet). In addition, a bitstream generated by an encoding method can be stored in a computer-readable recording medium or can be transmitted via a wired or wireless communication network.

[0339] In addition, an embodiment of the present disclosure can be implemented by a computer program product through program code, and the program code can be executed on a computer by an embodiment of the present disclosure. The program code can be stored on a computer-readable carrier.

[0340] Figure 8 An example of a content streaming system to which an embodiment of the present disclosure can be applied is shown.

[0341] Referring to Figure 8 , a content streaming system to which an embodiment of the present disclosure is applied may mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0342] The encoding server generates a bitstream by compressing content input from a multimedia input device such as a smartphone, a camera, or a video camera into digital data and sends it to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, or a video camera directly generates a bitstream, the encoding server can be omitted.

[0343] A bitstream can be generated by applying an encoding method or a bitstream generation method according to an embodiment of the present disclosure, and the streaming server can temporarily store the bitstream during the process of sending or receiving the bitstream.

[0344] The streaming server sends multimedia data to the user device via the web server based on the user's request, and the web server serves as a medium for notifying the user of what services are available. When the user requests a required service from the web server, the web server delivers it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls the commands / responses between each device in the content streaming system.

[0345] The streaming server can receive content from a media storage and / or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming media service, the streaming server can store the bitstream for a certain period of time.

[0346] Examples of user devices can include mobile phones, smartphones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0347] Each server in the content streaming system can be operated as a distributed server, and in this case, the data received from each server can be distributed and processed.

[0348] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of the present disclosure can be combined and implemented as a device, and the technical features of the device claims of the present disclosure can be combined and implemented as a method. Additionally, the technical features of the method claims of the present disclosure and the technical features of the device claims of the present disclosure can be combined and implemented as a device, and the technical features of the method claims of the present disclosure and the technical features of the device claims of the present disclosure can be combined and implemented as a method.

Claims

1. A method for decoding an image, comprising: Determining a filter length of a deblocking filter applied to a block boundary between a first block and a second block in a current picture; Determining a boundary strength of the deblocking filter; Determining the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength; And Applying the deblocking filter to the block boundary between the first block and the second block, wherein the filter length of the deblocking filter is determined based on whether multi-reference prediction is applied to at least one of the first block or the second block.

2. The method according to claim 1, wherein, The filter length of the deblocking filter is determined based on at least one of the size of the second block or whether the second block is a block compiled in an inter prediction mode based on sub-blocks.

3. The method according to claim 1, wherein, The boundary strength is determined based on whether the number of multiple reference blocks for the first block is the same as the number of multiple reference blocks for the second block.

4. The method according to claim 1, wherein, The boundary strength is determined based on the weights for the multi-reference prediction.

5. The method according to claim 1, wherein When the multi-reference prediction is applied to the second block, the boundary strength is determined based on whether the picture to which the normal reference block of the second block belongs is the same as the picture to which the additional reference block of the second block belongs.

6. The method according to claim 1, wherein When the multi-reference prediction is applied to the second block, the boundary strength is determined based on whether the picture to which the additional reference block of the second block belongs is the same as the picture to which the normal reference block of the first block belongs.

7. The method according to claim 1, wherein When the multi-reference prediction is applied to the second block, the boundary strength is determined based on the difference between the motion vector associated with the additional reference block of the second block and the motion vector associated with the normal reference block of the first block.

8. The method according to claim 1, wherein, When the multi-reference prediction is applied to the second block, the boundary strength is determined based on the difference between the motion vector associated with the normal reference block of the second block and the motion vector associated with the additional reference block of the first block.

9. The method according to claim 1, wherein A predefined maximum value for the boundary strength is 3 or 4.

10. A method for encoding an image, comprising: Determining a filter length of a deblocking filter applied to a block boundary between a first block and a second block in a current picture; Determining a boundary strength of the deblocking filter; Determining the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength; And Applying the deblocking filter to the block boundary between the first block and the second block, wherein the filter length of the deblocking filter is determined based on whether the multi-reference prediction is applied to at least one of the first block or the second block.

11. A computer-readable storage medium storing a bitstream generated according to the image encoding method of claim 10.

12. A method for sending data, comprising: Obtain a bitstream for image information, wherein the bitstream is generated by: determining a filter length of a deblocking filter applied to a block boundary between a first block and a second block in a current picture, determining a boundary strength of the deblocking filter, determining the deblocking filter having a predetermined type and strength based on the filter length and the boundary strength, and applying the deblocking filter to the block boundary between the first block and the second block; and Transmit the data including the bitstream, wherein the filter length of the deblocking filter is determined based on whether the multi-reference prediction is applied to at least one of the first block or the second block.

Images