Image coding apparatus and method for controlling loop filtering

Abstract

To provide an image decoding method.SOLUTION: A method includes the steps of: acquiring image information; generating a residual sample; deriving a prediction sample; generating a reconstructed sample of a current picture based on the prediction sample and the residual sample; and generating a corrected reconstructed sample, the image information including SPS. Information related to a virtual boundary includes a virtual boundary enabled flag. The SPS includes an SPS virtual boundary present flag based on the virtual boundary enabled flag. Whether signaling of the information related to the virtual boundary is present in the SPS is determined based on the virtual boundary enabled flag. Whether an in-loop filtering process is performed across the virtual boundary is determined based on the virtual boundary enabled flag. The SPS includes information on the number of vertical virtual boundaries and the number of horizontal virtual boundaries, based on that a value of the SPS virtual boundary present flag is 1.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] This document relates to an image coding apparatus and method for controlling loop filtering.

Background Art

[0002] In recent years, the demand for high-resolution and high-quality images / videos such as 4K or UHD (Ultra High Definition) images / videos of 8K or higher has been increasing in various fields. As the image / video data becomes higher in resolution and quality, the amount of information or bits transmitted relatively increases compared to existing image / video data. Therefore, when transmitting image data using a medium such as an existing wired or wireless broadband line, or storing image / video data using an existing storage medium, the transmission cost and storage cost increase.

[0003] Also, in recent years, the interest and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality) contents, and holograms have been increasing, and the broadcasting of images / videos having image characteristics different from real images, such as game images, has been increasing.

[0004] Thus, there is a need for a highly efficient image / video compression technology to effectively compress, transmit, store, and reproduce the information of high-resolution and high-quality images / videos having various characteristics as described above.

[0005] Specifically, there is a discussion about a scheme for efficiently controlling loop filtering executed across virtual boundaries.

Summary of the Invention

Means for Solving the Problems

[0006] According to an embodiment of this document, a method and apparatus for enhancing the efficiency of image / video coding are provided.

[0007] According to one embodiment of this document, an efficient filtering method and apparatus are provided.

[0008] According to one embodiment of this document, a method and apparatus for efficiently applying deblocking, SAO (Sample Adaptive Loop), and ALF (Adaptive Loop Filtering) are provided.

[0009] According to one embodiment of this document, in-loop filtering can be performed based on a virtual boundary.

[0010] According to one embodiment of this document, the Sequence Parameter Set (SPS) may have a virtual boundary enable flag for the SPS that indicates whether in-loop filtering is performed across the virtual boundary.

[0011] According to one embodiment of this document, in-loop filtering can be performed across virtual boundaries based on the virtual boundary enable flag of the SPS.

[0012] According to one embodiment of this document, an encoding device for performing video / image encoding is provided.

[0013] According to one embodiment of this document, a computer-readable digital storage medium is provided which stores encoded video / image information generated by a video / image encoding method disclosed in at least one embodiment of this document.

[0014] According to one embodiment of this document, a computer-readable digital storage medium is provided which stores encoded information or encoded video / image information, which is configured to be used by a decoding device to perform the video / image decoding method disclosed in at least one embodiment of this document. [Effects of the Invention]

[0015] According to one embodiment of this document, the overall image / video compression efficiency can be improved.

[0016] According to one embodiment of this document, subjective / objective visual quality can be enhanced through efficient filtering.

[0017] One embodiment of this document provides an in-loop filtering procedure (process) based on a virtual boundary that can save hardware resources.

[0018] According to one embodiment of this document, an in-loop filtering procedure based on a virtual boundary can be efficiently performed, and filtering performance can be improved.

[0019] According to one embodiment of this document, information for in-loop filtering based on virtual boundaries can be efficiently signaled. [Brief explanation of the drawing]

[0020] [Figure 1] This figure schematically illustrates an example of a video / image coding system that can be applied to the embodiments described herein. [Figure 2] This figure schematically illustrates the configuration of a video / image encoding device that can be applied to the embodiments described herein. [Figure 3] This figure schematically illustrates the configuration of a video / image decoding device that can be applied to the embodiments described herein. [Figure 4] This diagram exemplifies the hierarchical structure of coded images / videos. [Figure 5] This figure shows an example of an ALF filter shape. [Figure 6] This is a flowchart illustrating a filtering-based encoding method in an encoding device. [Figure 7] It is a flowchart for explaining a decoding method based on filtering in a decoding device. [Figure 8] It is a diagram schematically showing an example of a video / image encoding method and related components according to an embodiment of this document. [Figure 9] It is a diagram schematically showing an example of a video / image encoding method and related components according to an embodiment of this document. [Figure 10] It is a diagram schematically showing an example of an image / video decoding method and related components according to an embodiment of this document. [Figure 11] It is a diagram schematically showing an example of an image / video decoding method and related components according to an embodiment of this document. [Figure 12] It is a diagram showing an example of a content streaming system to which the embodiments disclosed in this document can be applied.

Mode for Carrying Out the Invention

[0021] This document can be modified in various ways, can have various embodiments, and specific embodiments are illustrated in the drawings and will be described in detail. However, this is not intended to limit this document to specific embodiments. The terms commonly used in this specification are merely used to describe specific embodiments and are not used with the intention of limiting the technical idea of this document. Singular expressions include plural expressions unless the context clearly indicates otherwise. Terms such as "including" or "having" in this specification are intended to specify the existence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and it should be understood that the existence or addition possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof, etc., is not precluded in advance.

[0022] On the other hand, each configuration shown in the diagrams described in this document is illustrated independently for the purpose of explaining its distinct characteristic functions, and does not mean that each configuration is implemented with separate hardware or separate software. For example, two or more of the configurations may be combined to form a single configuration, and one configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of this document as long as they do not deviate from the essence of this document.

[0023] Preferred embodiments of this document will be described in more detail below with reference to the attached drawings. Hereafter, the same reference numerals will be used for identical components in the drawings, and redundant descriptions of identical components will be omitted.

[0024] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document relate to the VVC (Versatile Video Coding) standard (ITU-T Rec.H.266), next-generation video / image coding standards after VVC, or other video coding-related standards (e.g., HEVC (High Efficiency Video Coding) standard (ITU-T Rec.H.265), EVC (Essential Video Coding) standard, AVS2 standard, etc.).

[0025] This document presents various embodiments relating to video / image coding, and unless otherwise noted, these embodiments can be implemented in combination with each other.

[0026] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a single image representing a specific time period, while "slice" or "tile" is a unit that constitutes part of a picture in coding. A slice or tile may contain one or more Coding Tree Units (CTUs). A single picture may consist of one or more slices or tiles. A single picture may consist of one or more tile groups. A tile group may contain one or more tiles.

[0027] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" may be used as a counterpart to pixel. A sample generally refers to a pixel or a pixel value, and may refer only to the luma component pixel / pixel value, or only to the chroma component pixel / pixel value. Alternatively, a sample can refer to a pixel value in the spatial domain, and if such a pixel value is converted to the frequency domain, it can also refer to the conversion coefficient in the frequency domain.

[0028] A unit represents a basic unit of image processing. A unit contains at least one of a specific region of a picture and information about that region. One unit contains one luma block and two chroma (e.g., cb, cr) blocks. The term unit may sometimes be used interchangeably with terms such as block or area. In general, an M×N block contains a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0029] In this document, " / " and "," are interpreted as "and / or". For example, "A / B" is interpreted as "A and / or B", and "A, B" is interpreted as "A and / or B". Additionally, "A / B / C" means "at least one of A, B and / or C". Similarly, "A, B, C" also means "at least one of A, B and / or C".

[0030] Additionally, in this document, "or" is interpreted as "and / or". For example, "A or B" could mean 1) "A" only, 2) "B" only, or 3) "A and B". In other words, "or" in this document can mean "additionally or alternatively".

[0031] In this specification, "at least one of A and B" may mean "A only," "B only," or "both A and B." Furthermore, in this specification, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted similarly to "at least one of A and B."

[0032] Furthermore, in this specification, "at least one of A, B and C" may mean "A only," "B only," "C only," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0033] Furthermore, parentheses used in this specification may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be proposed as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be proposed as an example of "prediction." Also, when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be proposed as an example of "prediction."

[0034] Technical features described individually within a single drawing in this specification may be implemented individually or simultaneously.

[0035] Figure 1 schematically shows an example of a video / image coding system to which this document can be applied.

[0036] As shown in Figure 1, a video / image coding system may comprise a source device and a receiving device. The source device can transmit encoded video / image information or data to the receiving device in file or streaming form via a digital storage medium or network.

[0037] The source device may comprise a video source, an encoding device, and a transmitter. The receiving device may comprise a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be provided in the encoding device. The receiver may be provided in the decoding device. The renderer may comprise a display unit, which may consist of a separate device or external component.

[0038] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source can include video / image capture devices and / or video / image generation devices. Video / image capture devices can include, for example, one or more cameras, or a video / image archive containing previously captured video / images. Video / image generation devices can include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated via a computer, in which case the video / image capture process can be replaced by the process of generating the associated data.

[0039] An encoding device can encode input video / images. For compression and coding efficiency, the encoding device can perform a series of steps, including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in bitstream format.

[0040] The transmitting unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium can include various types of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitting unit may include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0041] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of the encoding device.

[0042] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0043] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which this document applies. Hereinafter, the term "video encoding device" may include an image encoding device.

[0044] As shown in Figure 2, the encoding device 200 can be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-prediction unit 221 and an intra-prediction unit 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be called a reconstructor or a reconstructed block generator. The aforementioned image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. Furthermore, the memory 270 may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The hardware components may also further include the memory 270 as an internal / external component.

[0045] The image splitting unit 210 can split the input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, coding units can be recursively split from a coding tree unit (CTU) or a largeest coding unit (LCU) using a QTBTTT (Quad-Tree Binary-Tree Ternary-Tree) structure. For example, one coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary-tree structure and / or the ternary-tree structure. Alternatively, the binary-tree structure may be applied first. The coding procedure according to this disclosure may be performed based on a final coding unit that cannot be further divided. In this case, based on coding efficiency due to image characteristics, the largest coding unit may be used as the final coding unit, or, if necessary, the coding unit may be recursively divided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, as described later. As another example, the processing unit may further comprise a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit may each be divided or partitioned from the final coding unit described above.The above prediction unit may be a unit of sample prediction, and the above conversion unit may be a unit for deriving conversion coefficients and / or a unit for deriving residual signals from conversion coefficients.

[0046] The term "unit" can sometimes be confused with terms such as "block" or "area." Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a pixel or pel (which constitutes) a picture (or image).

[0047] The subtraction unit 231 can generate a residual signal (residual block, residual sample, or residual sample array) by subtracting the predicted signal (predicted block, predicted sample, or predicted sample array) output from the prediction unit 220 from the input image signal (original block, original sample, or original sample array), and the generated residual signal is transmitted to the conversion unit 232. The prediction unit 220 can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes the predicted sample for the current block. The prediction unit 220 can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. As will be described later in the explanation of each prediction mode, the prediction unit can generate various information related to prediction, such as prediction mode information, and transmit it to the entropy coding unit 240. The information related to prediction can be encoded by the entropy coding unit 240 and output in bitstream form.

[0048] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample can be located adjacent to the current block or at a distance, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Directional modes can include, for example, 33 or 65 directional prediction modes, depending on the degree of fineness of prediction direction. However, this is illustrative, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 222 can also determine the prediction mode to apply to the current block using the prediction modes applied to adjacent blocks.

[0049] The interprediction unit 221 can derive predicted blocks relative to the current block based on reference blocks (reference sample arrays) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the above reference blocks and the reference picture containing the above temporal neighboring blocks may be the same or different. The above temporal neighboring blocks may be called collocated reference blocks, collocated CUs, etc., and the reference picture containing the above temporal neighboring blocks may be called collocated pictures (colPic). For example, the inter-prediction unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-prediction can be performed based on various prediction modes; for example, in skip mode and merge mode, the inter-prediction unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In Motion Vector Prediction (MVP) mode, the motion vector of the adjacent block is used as the motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0050] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called Combined Inter and Intra Prediction (CIIP). The prediction unit can also perform intra-block copy (IBC) for predictions on blocks. The above intra-block copy can be used for content image / video coding such as in games, for example, in SCC (Screen Content Coding). IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can use at least one of the inter-prediction techniques described in this document.

[0051] The prediction signal generated via the interpretation unit 221 and / or the intrapretation unit 222 (including) can be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can apply a transformation technique to the residual signal to generate transformation coefficients. For example, the transformation technique may include DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph when relational information between pixels is represented in a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels and based on that. The transformation process may also be applied to pixel blocks of the same size and square shape, or to non-square blocks of variable size.

[0052] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 233 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector form of the quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form in units of Network Abstraction Layer (NAL) units. The video / image information may further include information about various parameter sets, such as the Adaptation Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). The video / image information may also further include general constraint information. In this document, the signaling / transmitted information and / or syntax elements described later can be encoded via the encoding procedure described above and included in the bitstream. The bitstream can be transmitted over a network or stored on a digital storage medium.Here, the network may include broadcasting networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. For signals output from the entropy encoding unit 240, a transmitting unit (not shown) and / or a storage unit (not shown) that transmits and / or stores the signals may be configured as internal / external elements of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0053] The quantized conversion coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 234 and the inverse transformation unit 235. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample, or reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the prediction unit 220. If there are no residuals for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, and can also be used for inter-prediction of the next picture after filtering, as described later.

[0054] On the other hand, LMCS (Luma Mapping with Chroma Scaling) can also be applied during the picture encoding and / or restoration process.

[0055] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering-related information and transmit it to the entropy encoding unit 240, as will be described later in the explanation of each filtering method. The filtering-related information can be encoded by the entropy encoding unit 240 and output in bitstream format.

[0056] The corrected restored picture sent to memory 270 can be used as a reference picture in the interpretation unit 221. When interpretation is applied via this, the encoding device can avoid prediction mismatches between the encoding device 100 and the decoding device, and can also improve encoding efficiency.

[0057] The DPB in memory 270 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in already restored pictures. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0058] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which this document can be applied.

[0059] As shown in Figure 3, the decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-prediction unit 331 and an intra-prediction unit 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The aforementioned entropy decoder 310, residual processor 320, predictor 330, adder 340, and filtering unit 350 can be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 360 may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The above hardware components may also include Memory 360 as an internal / external component.

[0060] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image corresponding to the process by which the video / image information was processed in the encoding device shown in Figure 3. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Therefore, the decoding processing unit can be, for example, a coding unit, which can be divided from a coding tree unit or a maximum coding unit according to a quadtree structure, a binary tree structure, and / or a ternary tree structure. One or more transformation units can be derived from the coding unit. The reconstructed image signal decoded and output via the decoding device 300 can then be reproduced via a playback device.

[0061] The decoding device 300 can receive the signal output from the encoding device shown in Figure 3 in bitstream form, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can further decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of transformation coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded, the decoded information of adjacent and decoded blocks, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, performs arithmetic decoding of the bins, and generates symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 310, information related to prediction is provided to the prediction unit 330, and residual information from the entropy decoding performed by the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 321. In addition, of the information decoded by the entropy decoding unit 310, information related to filtering can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit can be a component of the entropy decoding unit 310. On the other hand, the decoding device relating to this document can be called a video / image / picture decoding device, and the above decoding device can also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The above information decoder may include the above entropy decoding unit 310, and the above sample decoder may include at least one of the above inverse quantization unit 321, inverse transform unit 322, prediction unit 330, addition unit 340, filtering unit 350, and memory 360.

[0062] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0063] The inverse transform unit 322 performs an inverse transform on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0064] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0065] The prediction unit can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called Combined Inter and Intra Prediction (CIIP). The prediction unit can also perform intra-block copy (IBC) for predictions on blocks. The above intra-block copy can be used for content image / video coding in games, for example, as in SCC (Screen Content Coding). IBC basically performs prediction within the current picture, but can be done similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can be considered an example of intra-coding or intra-prediction.

[0066] The intra-prediction unit 331 can predict the current block by referring to a sample within the current picture. The referenced sample can be located adjacent to or far from the current block, depending on the prediction mode. In intra-prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-prediction unit 331 can also determine the prediction mode to be applied to the current block using the prediction modes applied to adjacent blocks.

[0067] The interprediction unit 332 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the information regarding the prediction may include information indicating the mode of interprediction for the current block.

[0068] The summing unit 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block.

[0069] The summing unit 340 may be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and can be output after filtering as described later, or it can be used for intra-prediction of the next picture.

[0070] On the other hand, LMCS (Luma Mapping with Chroma Scaling) can also be applied during the picture decoding process.

[0071] The filtering unit 350 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0072] The restored picture stored (modified) in the DPB of memory 360 can be used as a reference picture by the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in already restored pictures. The stored motion information can be transmitted to the inter-prediction unit 332 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0073] In this specification, embodiments described in relation to the prediction unit 330, inverse quantization unit 321, inverse transform unit 322, and filtering unit 350 of the decoding device 300 can be applied identically to or corresponding to the prediction unit 220, inverse quantization unit 234, inverse transform unit 235, and filtering unit 260 of the encoding device 200, respectively.

[0074] As mentioned above, prediction is performed to improve compression efficiency when performing video coding. Through this, a predicted block containing predicted samples for the current block, which is the block to be coded, can be generated. Here, the predicted block contains predicted samples in the spatial region (or pixel region). The predicted block is derived in the same way by the encoding device and the decoding device, and the encoding device can improve image coding efficiency by signaling the decoding device information about the residuals between the original block and the predicted block (residual information), which is not the original sample value of the original block itself. The decoding device can derive a residual block containing residual samples based on the residual information, and can generate a restored block containing restored samples by combining the residual block and the predicted block, and can generate a restored picture containing the restored block.

[0075] The residual information described above can be generated through transformation and quantization procedures. For example, an encoding device can signal the relevant residual information to a decoding device (via a bitstream) by deriving a residual block between the original block and the predicted block, performing a transformation procedure on the residual samples (residual sample array) contained in the residual block to derive transformation coefficients, and then performing a quantization procedure on the transformation coefficients to derive quantized transformation coefficients. Here, the residual information may include information such as the value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. The decoding device can derive residual samples (or residual blocks) by performing an inverse quantization / inverse transformation procedure based on the residual information. The decoding device can generate a reconstructed picture based on the predicted block and the residual block. Alternatively, the encoding device can derive a residual block by inverse quantization / inverse transformation of the quantized transformation coefficients for reference for subsequent picture interpretations, and generate a reconstructed picture based on this.

[0076] In this document, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. If the above quantization / inverse quantization is omitted, the above quantized transformation coefficients may be called transformation coefficients. If the above transformation / inverse transformation is omitted, the above transformation coefficients may also be called coefficients or residual coefficients, or for consistency of expression, they may still be called transformation coefficients.

[0077] In this document, quantized transformation coefficients and transformation coefficients can be referred to as transformation coefficients and scaled transformation coefficients, respectively. In this case, residual information can include information about the transformation coefficient(s), and such information about the transformation coefficient(s) can be signaled via residual coding syntax. Based on the residual information (or information about the transformation coefficient(s)), the transformation coefficient(s) can be derived, and the scaled transformation coefficient(s) can be derived via an inverse transformation (scaling) of the transformation coefficient(s). Based on an inverse transformation (transformation) of the scaled transformation coefficient(s), residual samples can be derived. This can be applied / expressed similarly in other parts of this document.

[0078] The prediction unit of the encoding / decoding device can perform interpretation on a block-by-block basis to derive predicted samples. Interpretation can indicate predictions derived in a way that depends on data elements of pictures other than the current picture (e.g., sample values ​​or motion information). When interpretation is applied to the current block, a predicted block (predicted sample array) for the current block can be derived (induced) based on the reference block (reference sample array) identified by the motion vector on the reference picture pointed to by the index of the reference picture. In this case, in order to reduce the amount of motion information transmitted in interpretation mode, the motion information of the current block can be predicted on a block, subblock, or sample basis based on the correlation of motion information between neighboring blocks and the current block. The above motion information may include motion vectors and the index of the reference picture. The above motion information may further include information on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). When interpretation is applied, neighboring blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the above-mentioned reference block and the reference picture containing the above-mentioned time-adjacent block may be the same or different. The above-mentioned time-adjacent block may be called a collocated reference block or collocated CU (colCU), and the reference picture containing the above-mentioned time-adjacent block may be called a collocated picture (colPic). For example, a candidate list of motion information may be constructed based on the adjacent blocks of the current block, and flags or index information may be signaled to indicate which candidate is selected (used) in order to derive the motion vector and / or the index of the reference picture of the current block.Interpretation is performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block may be identical to the motion information of the selected adjacent block. In skip mode, unlike merge mode, a residual signal may not be transmitted. In Motion Vector Prediction (MVP) mode, the motion vector of the selected adjacent block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0079] The above motion information may include L0 motion information and / or L1 motion information, depending on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be called an L0 motion vector or MVL0, and a motion vector in the L1 direction may be called an L1 motion vector or MVL1. A prediction based on an L0 motion vector may be called an L0 prediction, a prediction based on an L1 motion vector may be called an L1 prediction, and a prediction based on both the above L0 motion vector and the above L1 motion vector may be called a Bi (Bi) prediction. Here, an L0 motion vector may represent a motion vector associated with the reference picture list L0 (L0), and an L1 motion vector may represent a motion vector associated with the reference picture list L1 (L1). The reference picture list L0 may include pictures earlier in the output order than the above current picture, and the reference picture list L1 may include pictures later in the output order than the above current picture. Pictures prior to the above may be called forward (reference) pictures, and pictures after the above may be called reverse (reference) pictures. The above reference picture list L0 may include pictures that are later in the output order than the above current picture as reference pictures. In this case, the above earlier pictures may be indexed first in the above reference picture list L0, and the above later pictures may be indexed afterward. The above reference picture list L1 may include pictures that are earlier in the output order than the above current picture as reference pictures. In this case, the above later pictures may be indexed first in the above reference picture list L1, and the above earlier pictures may be indexed afterward. Here, the output order may correspond to the POC (Picture Order Count) order.

[0080] Figure 4 illustrates the hierarchical structure for coded images / videos.

[0081] As shown in Figure 4, coded images / videos are divided into the VCL (Video Coding Layer), which handles the decoding process and the images / videos themselves; a lower-level system that transmits and stores the coded information; and the NAL (Network Abstraction Layer), which exists between the VCL and the lower-level system and is responsible for network adaptation functions.

[0082] VCL can generate VCL data containing compressed image data (slice data), or generate parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages additionally required during the image decoding process.

[0083] In NAL, a NAL unit can be generated by adding header information (NAL unit header) to an RBSP (Raw Byte Sequence Payload) generated by VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc., generated by VCL. The NAL unit header can include NAL unit type information, which is identified by the RBSP data contained in the NAL unit.

[0084] As shown in the diagram above, NAL units can be divided into VCL NAL units and Non-VCL NAL units by the RBSP generated in VCL. VCL NAL units can represent NAL units that contain information about the image (slice data), while Non-VCL NAL units can represent NAL units that contain information necessary to decode the image (parameter set or SEI message).

[0085] The aforementioned VCL NAL units and Non-VCL NAL units can be transmitted over a network with header information added according to the data standards of the lower-level system. For example, NAL units can be transformed into data formats of predetermined standards such as H.266 / VVC file format, RTP (Real-time Transport Protocol), and TS (Transport Stream) and transmitted over various networks.

[0086] As mentioned above, the NAL unit type can be identified by the RBSP data structure contained within the NAL unit, and this information about the NAL unit type can be stored in the NAL unit header and signaled.

[0087] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether or not they contain image information (slice data). VCL NAL unit types can be further classified by the nature and type of picture they contain, while Non-VCL NAL unit types can be further classified by the type of parameter set.

[0088] The following is an example of a NAL unit type identified by the type of parameter set included in the Non-VCL NAL unit type.

[0089] -APS (Adaptation Parameter Set) NAL unit: Type for NAL units that include APS

[0090] -DPS (Decoding Parameter Set) NAL unit: Type for NAL units including DPS

[0091] -VPS (Video Parameter Set) NAL unit: Type for NAL unit including VPS

[0092] -SPS (Sequence Parameter Set) NAL unit: Type for NAL units that include SPS

[0093] -PPS (Picture Parameter Set) NAL unit: Type for NAL units that include PPS

[0094] -PH (Picture Header) NAL unit: Type for NAL units containing PH

[0095] The aforementioned NAL unit type has syntax information for the NAL unit type, and this syntax information can be stored in the NAL unit header and signaled. For example, the syntax information is nal_unit_type, and the NAL unit type can be identified by the nal_unit_type value.

[0096] On the other hand, as mentioned above, a single picture can contain multiple slices, and a single slice can contain a slice header and slice data. In this case, a single picture header may be added to each of the multiple slices (slice headers and slice data sets) within a single picture. The above picture header (picture header syntax) may contain information / parameters that are commonly applicable to the above picture. In this document, slices may be mixed with or substituted for tile groups. Also in this document, slice headers may be mixed with or substituted for type group headers.

[0097] The above slice header (slice header syntax, slice header information) may include information / parameters that can be applied in common to the above slice. The above APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be applied in common to one or more slices or pictures. The above SPS (SPS syntax) may include information / parameters that can be applied in common to one or more sequences. The above VPS (VPS syntax) may include information / parameters that can be applied in common to multiple layers. The above DPS (DPS syntax) may include information / parameters that can be applied in common to video in general. The above DPS may include information / parameters related to the concatenation of CVS (Coded Video Sequence). In this document, High Level Syntax (HLS) may include at least one of the above APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, picture header syntax, and slice header syntax.

[0098] In this document, the image / video information encoded from the encoding device to the decoding device and signaled in bitstream form may include not only partitioning-related information, intra / inter prediction information, residual information, and in-loop filtering information within the picture, but also information contained in the slice header, picture header, APS, PPS, SPS, VPS, and / or DPS. Furthermore, the image / video information may further include information from the NAL unit header.

[0099] On the other hand, to compensate for differences between the original image and the reconstructed image due to errors that occur during the compression encoding process, such as quantization, an in-loop filtering procedure can be performed on the reconstructed sample or reconstructed picture, as described above. As described above, in-loop filtering can be performed in the filter section of the encoding device and the filter section of the decoding device, and a deblocking filter, SAO, and / or adaptive loop filter (ALF) can be applied. For example, the ALF procedure can be performed after the deblocking filtering procedure and / or SAO procedure are completed. However, even in this case, the deblocking filtering procedure and / or SAO procedure may be omitted.

[0100] The following provides a detailed explanation of picture restoration and filtering. In image / video coding, a restored block can be generated based on intra-prediction / inter-prediction for each block, and a restored picture containing the restored block can be generated. If the current picture / slice is an I-picture / slice, the blocks contained in the current picture / slice can be restored based solely on intra-prediction. On the other hand, if the current picture / slice is a P or B-picture / slice, the blocks contained in the current picture / slice can be restored based on either intra-prediction or inter-prediction. In this case, intra-prediction may be applied to some of the blocks in the current picture / slice, while inter-prediction may be applied to the remaining blocks.

[0101] Intra prediction can represent a prediction that generates prediction samples for the current block based on reference samples within the picture to which the current block belongs (hereinafter referred to as the current picture). When intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block can be derived. The adjacent reference samples of the current block described above may include a total of 2 × nH samples adjacent to the left boundary and bottom-left of the current block of size nW × nH, a total of 2 × nW samples adjacent to the top boundary and top-right of the current block, and one sample adjacent to the top-left of the current block. Alternatively, the adjacent reference samples of the current block described above may also include multiple columns of upper adjacent samples and multiple rows of left adjacent samples. Furthermore, the adjacent reference samples of the current block may also include a total of nH samples adjacent to the right boundary of the current block, which is of size nW × nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0102] However, some of the adjacent reference samples in the current block may not yet be decoded or available. In this case, the decoder can construct adjacent reference samples to be used for prediction by substituting the unavailable samples as available samples. Alternatively, it can construct adjacent reference samples to be used for prediction through interpolation of available samples.

[0103] If neighboring reference samples are derived, (i) predicted samples can be derived based on the average or interpolation of neighboring reference samples in the current block, or (ii) predicted samples can be derived based on reference samples in the current block that are located in a specific (prediction) direction relative to the predicted sample. Case (i) is called the non-directional mode or non-angular mode, and case (ii) is called the directional mode or angular mode. In addition, the predicted sample can also be generated by interpolation between the first neighboring sample and a second neighboring sample located in the opposite direction to the prediction direction of the intra-prediction mode of the current block, based on the predicted sample of the current block among the neighboring reference samples. In the above case, it can be called Linear Interpolation Intra Prediction (LIP). In addition, chroma predicted samples can be generated based on chroma samples using a linear model. In this case, it can be called the LM mode. Alternatively, a temporary (provisional) predicted sample for the current block can be derived based on filtered adjacent reference samples, and the predicted sample for the current block can be derived by performing a weighted sum of the temporary predicted sample and at least one reference sample derived by the intra-prediction mode from the existing adjacent reference samples, i.e., the unfiltered adjacent reference samples. In the case described above, this can be called PDPC (Position Dependent Intra Prediction). In addition, intra-predictive coding can be performed by selecting the reference sample line with the highest prediction accuracy (accuracy) from the adjacent multiple reference sample lines of the current block, deriving the predicted sample using the reference sample located in the prediction direction on that line, and then instructing (signaling) the decoding device with the reference sample line used at that time.In the aforementioned cases, this can be called Multi-Reference Line (MRL) intra prediction or MRL-based intra prediction. Furthermore, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, allowing for the deriving and use of adjacent reference samples on a subpartition basis. That is, in this case, the intra prediction mode for the current block is also applied to the subpartition, and by deriving and using adjacent reference samples on a subpartition basis, intra prediction performance can sometimes be improved. Such prediction methods can be called Intra Sub-Partitions (ISP) or ISP-based intra prediction. The aforementioned intra prediction methods can be distinguished from the intra prediction modes in sections 1 and 2 of the table of contents and referred to as intra prediction types. These intra prediction types can be referred to by various terms, such as intra prediction techniques or additional intra prediction modes. For example, the above intra prediction type (or additional intra prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP. A general intra-prediction method that excludes specific intra-prediction types such as LIP, PDPC, MRL, and ISP can be called a normal intra-prediction type. A normal intra-prediction type can be generally applied when the specific intra-prediction types mentioned above are not applicable, and predictions can be performed based on the aforementioned intra-prediction modes. Furthermore, post-processing filtering can be performed on the derived prediction samples as needed.

[0104] Specifically, the intra-prediction procedure may include an intra-prediction mode / type determination step, an adjacent reference sample derivation step, and an intra-prediction mode / type-based predictive sample derivation step. Additionally, a post-filtering step may be performed on the derived predictive samples as needed.

[0105] The in-loop filtering procedure generates a modified restored picture, which is output from the decoding device as a decoded picture and stored in the decoded picture buffer or memory of the encoding / decoding device, and can subsequently be used as a reference picture in the interpretation procedure during picture encoding / decoding. The in-loop filtering procedure includes, as described above, a deblocking filtering procedure, an SAO (Sample Adaptive Offset) procedure, and / or an ALF (Adaptive Loop Filter) procedure. In this case, one or part of the deblocking filtering procedure, SAO (Sample Adaptive Offset) procedure, ALF (Adaptive Loop Filter) procedure, and bi-lateral filter procedure may be applied sequentially, or all of them may be applied sequentially. For example, the SAO procedure may be performed after the deblocking filtering procedure is applied to the restored picture. Alternatively, for example, the ALF procedure may be performed after the deblocking filtering procedure is applied to the restored picture. This is also done in the encoding device.

[0106] Deblocking filtering is a filtering technique that removes distortion that occurs at the boundaries between blocks in a reconstructed picture. The deblocking filtering procedure can, for example, involve deriving a target boundary in the reconstructed picture, determining a boundary strength (bs) for the target boundary, and performing deblocking filtering on the target boundary based on the bS. The bS can be determined based on the prediction modes of two adjacent blocks, the difference in motion vectors, whether the reference pictures are identical, and whether a non-zero effectiveness coefficient exists.

[0107] SAO is a method for compensating for the offset difference between a restored picture and the original picture on a sample-by-sample basis, and can be applied based on types such as Band Offset and Edge Offset. According to SAO, each SAO type classifies samples into different categories, and an offset value can be added to each sample based on its category. Filtering information for SAO can include information on whether SAO is applicable, SAO type information, SAO offset value information, etc. SAO can also be applied to restored pictures after the above deblocking filtering has been applied.

[0108] ALF (Adaptive Loop Filter) is a technique that filters a restored picture on a sample-by-sample basis based on filter coefficients determined by the filter shape. The encoding device can determine whether ALF can be applied, the ALF shape, and / or ALF filtering coefficients by comparing the restored picture with the original picture, and can signal this to the decoding device. In other words, filtering information for ALF can include information on whether ALF can be applied, ALF filter shape information, ALF filtering coefficient information, etc. ALF can also be applied to the restored picture after the above-mentioned deblocking filtering has been applied.

[0109] Figure 5 shows an example of an ALF filter shape.

[0110] Figure 5(a) shows a 7x7 diamond filter shape, and (b) shows a 5x5 diamond filter shape. In Figure 5, Cn in the filter shape represents the filter coefficient. When n is the same in Cn, this indicates that the same filter coefficient can be assigned. In this document, the position and / or unit to which the filter coefficient is assigned according to the ALF filter shape may be called a filter tab. In this case, one filter coefficient is assigned to each filter tab, and the arrangement of the filter tabs may correspond to a filter shape. A filter tab located in the center of the filter shape may be called a center filter tab. Two filter tabs with the same n value located at corresponding positions relative to the center filter tab may be assigned the same filter coefficient. For example, in the case of a 7x7 diamond filter shape, there are 25 filter tabs, and the filter coefficients C0 to C11 are assigned in a centrally symmetrical manner, so only 13 filter coefficients can be used to assign the filter coefficients to the above 25 filter tabs. Furthermore, for example, in the case of a 5x5 diamond filter shape, since it includes 13 filter tabs and the filter coefficients C0 to C5 are assigned in a centrally symmetrical manner, only 7 filter coefficients are needed to assign the filter coefficients to the 13 filter tabs. For example, to reduce the amount of data regarding the signaled filter coefficients, 12 of the 13 filter coefficients for a 7x7 diamond filter shape can be (explicitly) signaled, and one filter coefficient can be (implicitly) derived. Also, for example, 6 of the 7 filter coefficients for a 5x5 diamond filter shape can be (explicitly) signaled, and one filter coefficient can be (implicitly) derived.

[0111] Figure 6 is a flowchart illustrating a filtering-based encoding method in an encoding device. The method in Figure 6 may include steps S600 to S630.

[0112] In step S600, the encoding device can generate a restored picture. Step S600 can be performed based on the procedure for generating a restored picture (or restored sample) described above.

[0113] In step S610, the encoding device can determine whether in-loop filtering is applied (across the virtual boundary) based on in-loop filtering-related information. Here, in-loop filtering may include at least one of the deblocking filtering, SAO, or ALF described above.

[0114] In step S620, the encoding device can generate a modified restored picture (modified restored sample) based on the above decision in step S610. Here, the modified restored picture (modified restored sample) may be a filtered restored picture (filtered restored sample).

[0115] In step S630, the encoding device can encode image / video information including in-loop filtering-related information based on the in-loop filtering procedure.

[0116] Figure 7 is a flowchart illustrating a filtering-based decoding method in a decoding device. The method in Figure 7 may include steps S700 to S730.

[0117] In step S700, the decoding device can obtain image / video information, including in-loop filtering-related information, from the bitstream. Here, the bitstream may be based on encoded image / video information transmitted from the encoding device.

[0118] In step S710, the decoding device can generate a restored picture. Step S710 can be performed based on the procedure for generating a restored picture (or restored sample) described above.

[0119] In step S720, the decoding device can determine whether in-loop filtering is applied (across the virtual boundary) based on in-loop filtering-related information. Here, in-loop filtering may include at least one of the deblocking filtering, SAO, or ALF described above.

[0120] In step S730, the decoding device can generate a modified restored picture (modified restored sample) based on the above determination in step S720. Here, the modified restored picture (modified restored sample) may be a filtered restored picture (filtered restored sample).

[0121] As mentioned above, an in-loop filtering procedure can be applied to the restored picture. In this case, a virtual boundary can be defined to further enhance the subjective / objective visual quality of the restored picture, and the in-loop filtering procedure can be applied across the virtual boundary. The virtual boundary may include discontinuous edges such as 360-degree images, VR images, or PIPs (Picture In Picture). For example, the virtual boundary may exist at a predetermined, agreed-upon location, and its presence and / or location may be signaled. As an example, the virtual boundary may be located above the fourth sample line from the top in the CTU row (specifically, for example, above the fourth sample line from the top in the CTU row). As another example, information regarding the presence and / or location of the virtual boundary may be signaled via the HLS. The HLS may include SPS, PPS, picture headers, slice headers, etc., as mentioned above.

[0122] The following describes the signaling and semantics of the higher-level syntax for the embodiments of this document.

[0123] One embodiment of this document may include a method for controlling a loop filter. This method for controlling a loop filter may be applied to a restored picture. The in-loop filter (loop filter) can be used for decoding an encoded bitstream. The loop filter may include the deblocking, SAO, and ALF mentioned above. The SPS may include flags associated with each of the deblocking, SAO, and ALF. These flags may indicate whether each tool is available for coding a CLVS (Coded Layer Video Sequence) or CVS (Coded Video Sequence) that references the SPS.

[0124] If the above loop filter is available for CVS, its application can be controlled to avoid crossing specific boundaries. For example, whether the loop filter crosses subpicture boundaries can be controlled. Also, whether the loop filter crosses tile boundaries can be controlled. In addition, whether the loop filter crosses virtual boundaries can be controlled, where virtual boundaries can be defined on the CTU based on the availability of line buffers.

[0125] Related to whether an in-loop filtering procedure is performed across a virtual boundary, in-loop filtering related information may include at least one of the following: the SPS virtual boundary enable flag (virtual boundary enable flag within the SPS), the SPS virtual boundary existence flag, the picture header virtual boundary existence flag, the SPS picture header virtual boundary existence flag, and information regarding the location of the virtual boundary.

[0126] In the embodiments described herein, information regarding the position of a virtual boundary may include information regarding the x-coordinate of a vertical virtual boundary and / or information regarding the y-coordinate of a horizontal virtual boundary. Specifically, information regarding the position of a virtual boundary may include information regarding the x-coordinate of a vertical virtual boundary and / or information regarding the y-coordinate of a horizontal virtual boundary in luma sample units. Furthermore, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the x-coordinate of a vertical virtual boundary present in the SPS. Furthermore, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the y-coordinate of a horizontal virtual boundary present in the SPS. Alternatively, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the x-coordinate of a vertical virtual boundary present in the picture header. Furthermore, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the y-coordinate of a horizontal virtual boundary present in the picture header.

[0127] The following table shows exemplary syntax and semantics of SPS according to this embodiment.

[0128] [Table 1]

[0129] [Table 2]

[0130] The following table shows exemplary syntax and semantics of the Picture Parameter Set (PPS) according to this embodiment.

[0131] [Table 3]

[0132] [Table 4]

[0133] The following table shows exemplary syntax and semantics of a picture header according to this embodiment.

[0134] [Table 5-1]

[0135] [Table 5-2]

[0136] [Table 6-1]

[0137] [Table 6-2]

[0138] The following table shows exemplary syntax and semantics of a slice header according to this embodiment.

[0139] [Table 7]

[0140] [Table 8]

[0141] The following section describes the signaling of information about virtual boundaries that can be used in in-loop filtering.

[0142] In existing designs, to disable loop filters that cross virtual boundaries, i) the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is set to 0, and the PH virtual boundary presence flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag) exists for all picture headers and is set to 0, or ii) the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is set to 1, and information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_vertical_boudnaries) and information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_vertical_boudnaries) may both be set to 0.

[0143] In existing designs, the above ii) can cause the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) to be set to 1, which can lead to problems in the decoding procedure because the decoder anticipates signaling for the location of the virtual boundary.

[0144] The embodiments described in the following paragraphs can offer solutions to the aforementioned problems. The embodiments may be applied independently, or at least two or more embodiments may be applied in combination.

[0145] In one embodiment of this document, whether or not syntax elements for indicating virtual boundaries are included in the SPS can be controlled by flags. For example, there may be two such flags (e.g., an SPS virtual boundaries enabled flag and an SPS virtual boundaries present flag).

[0146] In one example according to this embodiment, the virtual boundary enable flag of the SPS may be called sps_loop_filter_across_virtual_boundaries_disabled_flag (or sps_virtual_boundaries_enabled_flag). The virtual boundary enable flag of the SPS can indicate whether the feature for disabling the loop filter across a virtual boundary is enabled.

[0147] In one example according to this embodiment, the virtual boundary presence flag of the SPS may be referred to as sps_loop_filter_across_virtual_boundaries_disabled_present_flag (or sps_virtual_boundaries_present_flag). The virtual boundary presence flag of the SPS can indicate whether signaling information for virtual boundaries is included in the SPS or picture header (PH).

[0148] In one example according to this embodiment, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 0, then signaling information to disable loop filters crossing virtual boundaries may be included in the picture header.

[0149] In one example according to this embodiment, if information regarding the location of virtual boundaries (e.g., vertical virtual boundaries, horizontal virtual boundaries) is included in the SPS, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be restricted to be greater than 0.

[0150] In one example according to this embodiment, a variable can be derived that indicates whether the filter is disabled at the virtual boundary for the current picture. For example, the above variable may include VirtualBoundairesDisabledFlag.

[0151] In one example in this illustration, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the above SPS is 1 and the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the above SPS is 1, then VirtualBoundairesDisabledFlag may be 1.

[0152] In another example, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the above SPS is 1, the virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the above SPS is 0, and the sum of the information regarding the number of vertical virtual boundaries (e.g., ph_num_ver_virtual_boundaries) and the information regarding the number of horizontal virtual boundaries (e.g., ph_num_hor_virtual_boundaries) is greater than 0, then VirtualBoundairesDisabledFlag may be 1.

[0153] In other cases in this example, VirtualBoundairesDisabledFlag may be 0.

[0154] The following table shows an exemplary syntax of SPS according to this embodiment.

[0155] [Table 9]

[0156] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0157] [Table 10]

[0158] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0159] [Table 11]

[0160] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0161] [Table 12]

[0162] In embodiments relating to Tables 9 to 12, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). The SPS may also include a virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) based on the virtual boundary enable flag.

[0163] For example, the above SPS may include a virtual boundary existence flag if the value of the virtual boundary enable flag is 1. Based on the virtual boundary enable flag and the virtual boundary existence flag of the above SPS, the above SPS may include information about the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information about the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information about the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information about the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]). For example, if the value of the virtual boundary enable flag and the value of the virtual boundary existence flag of the above SPS are 1, the above SPS may include information about the number of vertical virtual boundaries of the SPS, information about the position of the vertical virtual boundaries of the SPS, information about the number of horizontal virtual boundaries of the SPS, and information about the position of the horizontal virtual boundaries of the SPS.

[0164] In one example, the number of pieces of information regarding the position of the vertical virtual boundary of the SPS can be determined based on the number of pieces of information regarding the vertical virtual boundary of the SPS, and the number of pieces of information regarding the position of the horizontal virtual boundary of the SPS can be determined based on the number of pieces of information regarding the horizontal virtual boundary of the SPS. The picture header may include information regarding the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundary of the PH (ph_virtual_boundaries_pos_y[i]), based on the virtual boundary enable flag and the virtual boundary existence flag of the SPS.

[0165] For example, if the value of the virtual boundary enable flag is 1 and the value of the virtual boundary existence flag for the SPS is 0, the picture header may include information about the number of vertical virtual boundaries of the PH, information about the positions of the vertical virtual boundaries of the PH, information about the number of horizontal virtual boundaries of the PH, and information about the positions of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information about the positions of the vertical virtual boundaries of the PH can be determined based on the information about the number of vertical virtual boundaries of the PH, and the number of pieces of information about the positions of the horizontal virtual boundaries of the PH can be determined based on the information about the number of horizontal virtual boundaries of the PH.

[0166] In another embodiment of this document, each of the picture headers (picture headers) referencing an SPS may include a virtual boundary presence flag for the PH, ph_loop_filter_across_virtual_boundaries_disabled_present_flag (or ph_virtual_boundaries_present_flag). This embodiment can also be described together with the virtual boundary enable flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_flag) and the virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag), as in the previous embodiment.

[0167] In one example according to this embodiment, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 0, then each of the picture header information (picture header) of a picture referencing the SPS may include the virtual boundary existence flag ph_loop_filter_across_virtual_boundaries_disalbed_present_flag (or ph_virtual_boundaries_present_flag) of the PH.

[0168] In one example according to this embodiment, if information regarding the location of virtual boundaries (e.g., vertical virtual boundaries, horizontal virtual boundaries) is included in the SPS, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be restricted to be greater than 0.

[0169] In one example according to this embodiment, a variable can be derived that indicates whether the filter is disabled at the virtual boundary for the current picture. For example, the above variable may include VirtualBoundairesDisabledFlag.

[0170] In one example in this illustration, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the above SPS is 1 and the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the above SPS is 1, then VirtualBoundairesDisabledFlag may be 1.

[0171] In another example, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the above SPS is 1 and the virtual boundary existence flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the above PH is 1, then VirtualBoundairesDisabledFlag may be 1.

[0172] In other cases in this example, VirtualBoundairesDisabledFlag may be 0.

[0173] The following table shows an exemplary syntax of SPS according to this embodiment.

[0174] [Table 13]

[0175] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0176] [Table 14]

[0177] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0178] [Table 15]

[0179] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0180] [Table 16-1]

[0181] [Table 16-2]

[0182] In embodiments relating to Tables 13 to 16, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary enable flag, the SPS may include a virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, if the value of the virtual boundary enable flag is 1, the SPS may include a virtual boundary presence flag. Based on the virtual boundary enable flag and the virtual boundary existence flag for the SPS, the above SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]).

[0183] For example, if the value of the virtual boundary enable flag is 1 and the value of the virtual boundary existence flag for the SPS is 1, the SPS may include information about the number of vertical virtual boundaries, information about the positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about the positions of the horizontal virtual boundaries. In one example, the number of pieces of information about the positions of the vertical virtual boundaries of the SPS can be determined based on the information about the number of vertical virtual boundaries of the SPS, and the number of pieces of information about the positions of the horizontal virtual boundaries of the SPS can be determined based on the information about the number of horizontal virtual boundaries of the SPS. The picture header may include the virtual boundary existence flag for the PH based on the virtual boundary enable flag and the virtual boundary existence flag for the SPS.

[0184] For example, if the value of the virtual boundary enable flag is 1 and the value of the virtual boundary existence flag for the SPS is 0, the picture header may include the virtual boundary existence flag for the PH. Based on the virtual boundary existence flag for the PH, the picture header may include information about the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries), information about the position of the vertical virtual boundaries of the PH (ph_virtual_boundaries_pos_x[i]), information about the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries), and information about the position of the horizontal virtual boundaries of the PH (ph_virtual_boundaries_pos_y[i]).

[0185] For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information about the number of vertical virtual boundaries of the PH, information about the positions of the vertical virtual boundaries of the PH, information about the number of horizontal virtual boundaries of the PH, and information about the positions of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information about the positions of the vertical virtual boundaries of the PH can be determined based on the information about the number of vertical virtual boundaries of the PH, and the number of pieces of information about the positions of the horizontal virtual boundaries of the PH can be determined based on the information about the number of horizontal virtual boundaries of the PH.

[0186] In another embodiment of this document, whether a syntax element for indicating a virtual boundary is included in the SPS can be controlled by a flag. For example, there may be two such flags (e.g., an SPS virtual boundaries present flag and an SPS PH virtual boundaries present flag).

[0187] In one example according to this embodiment, the virtual boundary presence flag of the SPS may be called sps_loop_filter_across_virtual_boundaries_disabled_present_flag (or sps_virtual_boundaries_present_flag). The virtual boundary presence flag of the SPS can indicate whether or not virtual boundary information is included in the SPS.

[0188] In one example according to this embodiment, the virtual boundary presence flag of the SPS PH may be called sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag. The virtual boundary presence flag of the SPS PH can indicate whether or not virtual boundary information is included in the Picture Header (PH).

[0189] In one example according to this embodiment, if the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 1, then the virtual boundary existence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS PH does not exist and may be restricted to being inferred to 0.

[0190] In one example according to this embodiment, if the virtual boundary presence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS PH is 1, signaling information to disable loop filters crossing virtual boundaries may be included in the picture header.

[0191] The following table shows an exemplary syntax of SPS according to this embodiment.

[0192] [Table 17]

[0193] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0194] [Table 18]

[0195] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0196] [Table 19]

[0197] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0198] [Table 20-1]

[0199] [Table 20-2]

[0200] In embodiments relating to Tables 17 to 20, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). Based on the virtual boundary presence flag for the SPS, the SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]).

[0201] For example, if the value of the virtual boundary existence flag of the SPS is 1, the above SPS may include information about the number of vertical virtual boundaries of the SPS, information about the positions of the vertical virtual boundaries of the SPS, information about the number of horizontal virtual boundaries of the SPS, and information about the positions of the horizontal virtual boundaries of the SPS. In one example, the number of pieces of information about the positions of the vertical virtual boundaries of the SPS can be determined based on the information about the number of vertical virtual boundaries of the SPS, and the number of pieces of information about the positions of the horizontal virtual boundaries of the SPS can be determined based on the information about the number of horizontal virtual boundaries of the SPS. The above SPS may include a virtual boundary existence flag for SPS PH based on the virtual boundary existence flag of the SPS.

[0202] For example, if the value of the virtual boundary existence flag for the above SPS is 0, the above SPS may include the virtual boundary existence flag for SPS PH. The above picture header may include the virtual boundary existence flag for PH based on the virtual boundary existence flag for SPS PH. For example, if the value of the virtual boundary existence flag for the above SPS PH is 1, the above picture header may include the virtual boundary existence flag for PH. The above picture header may include information about the number of vertical virtual boundaries of PH (ph_num_ver_virtual_boundaries), information about the position of the vertical virtual boundaries of PH (ph_virtual_boundaries_pos_x[i]), information about the number of horizontal virtual boundaries of PH (ph_num_hor_virtual_boundaries), and information about the position of the horizontal virtual boundaries of PH (ph_virtual_boundaries_pos_y[i]) based on the virtual boundary existence flag of PH.

[0203] For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information about the number of vertical virtual boundaries of the PH, information about the positions of the vertical virtual boundaries of the PH, information about the number of horizontal virtual boundaries of the PH, and information about the positions of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information about the positions of the vertical virtual boundaries of the PH can be determined based on the information about the number of vertical virtual boundaries of the PH, and the number of pieces of information about the positions of the horizontal virtual boundaries of the PH can be determined based on the information about the number of horizontal virtual boundaries of the PH.

[0204] In another embodiment of this document, if Gradual Decoding Refresh (GDR) is available (i.e., if the value of gdr_enabled_flag is 1), the feature that loop filters are disabled at virtual boundaries is enabled, and virtual boundary information may be signaled in the picture header (may be included in the picture header).

[0205] In another embodiment of this document, when the function to disable loop filtering across a virtual boundary is enabled, information regarding the signaling of the virtual boundary's location may be included in one or more parameter sets. For example, when the function to disable loop filtering across a virtual boundary is enabled, information regarding the signaling of the virtual boundary's location may be included in the SPS and picture headers.

[0206] In this embodiment, if the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and signaling information regarding the location of the virtual boundary is included in one or more parameter sets, then the following applies:

[0207] a) Signaling information regarding the location of the virtual boundary may be included only in the SPS, only in the picture header, or in both the SPS and the picture header.

[0208] b) The derivation of VirtualBoundariesDisabledFlag for each picture is as follows:

[0209] - If sps_loop_filter_across_virtual_boundaries_disabled_flag is 0, then VirtualBoundariesDisabledFlag may be set to 0.

[0210] - In another case in this example, if information about the location of virtual boundaries is not signaled in the SPS or any picture header associated with the picture, VirtualBoundariesDisabledFlag may be set to 0.

[0211] - In other cases in this example (where the virtual boundary location is signaled only in the SPS, only in the picture header, or in both the SPS and the picture header), VirtualBoundariesDisabledFlag may be set to 1.

[0212] c) The virtual boundary applied to a picture may include a union of virtual boundaries signaled by parameter sets that the picture directly or indirectly references. For example, the above virtual boundary may include (if present) a virtual boundary signaled by the SPS. For example, the above virtual boundary may include (if present) a virtual boundary signaled by the picture header associated with the picture.

[0213] d) Restrictions may be applied to ensure that the maximum number of virtual boundaries per picture does not exceed a predefined value. For example, the predefined value may be 8.

[0214] e) Information regarding the location of virtual boundaries signaled in the picture header may be restricted (if any) so as not to be the same as information regarding the location of virtual boundaries included in other parameter sets (e.g., SPS or PPS).

[0215] - Alternatively, the location of a virtual boundary applied to the current picture (for example, the location of the same virtual boundary signaled by the SPS and the picture header associated with the picture) may be included in two different sets of parameters.

[0216] f) If the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) for the above SPS is 1, then the virtual boundary existence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) for the above SPS PH does not exist and may be restricted to being inferred to 0.

[0217] The following table shows an exemplary syntax of SPS according to this embodiment.

[0218] [Table 21]

[0219] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0220] [Table 22]

[0221] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0222] [Table 23]

[0223] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0224] [Table 24-1]

[0225] [Table 24-2]

[0226] In embodiments relating to Tables 21 to 24, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary enable flag, the SPS may include a virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, if the value of the virtual boundary enable flag is 1, the SPS may include a virtual boundary presence flag. Based on the virtual boundary enable flag and the virtual boundary existence flag for the SPS, the above SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]).

[0227] For example, if the value of the virtual boundary enable flag is 1 and the value of the virtual boundary existence flag for the SPS is 1, the SPS may include information about the number of vertical virtual boundaries, information about the positions of the vertical virtual boundaries, information about the number of horizontal virtual boundaries, and information about the positions of the horizontal virtual boundaries. In one example, the number of pieces of information about the positions of the vertical virtual boundaries of the SPS can be determined based on the information about the number of vertical virtual boundaries of the SPS, and the number of pieces of information about the positions of the horizontal virtual boundaries of the SPS can be determined based on the information about the number of horizontal virtual boundaries of the SPS. The picture header may include the virtual boundary existence flag for the PH based on the virtual boundary enable flag.

[0228] For example, if the value of the virtual boundary enable flag is 1, the picture header may include the virtual boundary existence flag for the PH. Based on the virtual boundary existence flag for the PH, the picture header may include information about the number of vertical virtual boundaries for the PH (ph_num_ver_virtual_boundaries), information about the position of the vertical virtual boundaries for the PH (ph_virtual_boundaries_pos_x[i]), information about the number of horizontal virtual boundaries for the PH (ph_num_hor_virtual_boundaries), and information about the position of the horizontal virtual boundaries for the PH (ph_virtual_boundaries_pos_y[i]). For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information about the number of vertical virtual boundaries for the PH, information about the position of the vertical virtual boundaries for the PH, information about the number of horizontal virtual boundaries for the PH, and information about the position of the horizontal virtual boundaries for the PH. In one example, the number of pieces of information regarding the position of the vertical virtual boundary of the PH can be determined based on the number of pieces of information regarding the vertical virtual boundary of the PH, and the number of pieces of information regarding the position of the horizontal virtual boundary of the PH can be determined based on the number of pieces of information regarding the horizontal virtual boundary of the PH.

[0229] In another embodiment of this document, loop filtering can be performed as in the embodiment described above, but without the restriction that the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries must be greater than 0.

[0230] In another embodiment of this document, virtual boundary information can be signaled in both the SPS and PH. In one example of this embodiment, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1, then information regarding the number of vertical virtual boundaries, the number of horizontal virtual boundaries, and / or information regarding the locations of virtual boundaries may be included in the SPS. In addition, if the virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1, then information regarding the number of vertical virtual boundaries, the number of horizontal virtual boundaries, and / or information regarding the delta values ​​of the virtual boundary locations may be included in the picture header. The delta values ​​of the virtual boundary locations may represent the difference between the virtual boundary locations. The picture header may also include information regarding the sign of the virtual boundary locations.

[0231] According to one example of this embodiment, in order to derive the virtual boundary position for each picture, if the delta value of the virtual boundary position is not present in the picture header, the information about the virtual boundary position signaled by SPS may be used for loop filtering. If the delta value of the virtual boundary position is present in the picture header, the virtual boundary position may be derived based on the sum of the information about the virtual boundary position signaled by SPS and the associated delta value.

[0232] The following table shows an exemplary syntax of SPS according to this embodiment.

[0233] [Table 25]

[0234] The following table shows exemplary semantics regarding the syntax elements included in the above syntax.

[0235] [Table 26]

[0236] The following table shows exemplary syntax of the header information (picture header) according to this embodiment.

[0237] [Table 27]

[0238] The following table shows exemplary semantics regarding the syntax elements included in the above syntax.

[0239] [Table 28-1]

[0240] [Table 28-2]

[0241] In embodiments relating to Tables 25 to 28, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary enable flag, the SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]). For example, if the value of the virtual boundary enable flag is 1, the above SPS may include information regarding the number of horizontal virtual boundaries of the SPS, information regarding the positions of the horizontal virtual boundaries of the SPS, information regarding the number of vertical virtual boundaries of the SPS, and information regarding the positions of the vertical virtual boundaries of the SPS.

[0242] For example, the number of pieces of information regarding the position of the horizontal virtual boundary of the SPS can be determined based on the number of horizontal virtual boundaries of the SPS, and the number of pieces of information regarding the position of the vertical virtual boundary of the SPS can be determined based on the number of vertical virtual boundaries of the SPS. The picture header may include a virtual boundary existence flag for the PH based on the virtual boundary enable flag. For example, if the value of the virtual boundary enable flag is 1, the picture header may include a virtual boundary existence flag for the PH. The picture header may include information regarding the delta value of the position of the horizontal virtual boundary of the PH (ph_virtual_boundaries_pos_x_delta[i]), information regarding the sign of the position of the horizontal virtual boundary of the PH (ph_virtual_boundaries_pos_x_sign[i]), information regarding the delta value of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_y_delta[i]), and information regarding the sign of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_y_sign[i]), based on the virtual boundary existence flag for the PH.

[0243] For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information regarding the delta value of the position of the vertical virtual boundary of the PH, information regarding the sign of the position of the vertical virtual boundary of the PH, information regarding the delta value of the position of the horizontal virtual boundary of the PH, and information regarding the sign of the position of the horizontal virtual boundary of the PH. In one example, based on information regarding the number of vertical virtual boundaries of the SPS, the number of pieces of information regarding the delta value of the position of the vertical virtual boundary of the PH and the number of pieces of information regarding the sign of the position of the vertical virtual boundary of the PH can be determined, and based on information regarding the number of horizontal virtual boundaries of the SPS, the number of pieces of information regarding the delta value of the position of the horizontal virtual boundary of the PH and the number of pieces of information regarding the sign of the position of the horizontal virtual boundary of the PH can be determined.

[0244] In further embodiments of this document, signaling of information regarding the location of virtual boundaries for each picture is described. For example, if information regarding the location of virtual boundaries is included in the SPS and information regarding the delta value of the location of virtual boundaries is not included in the picture header, the information regarding the location of virtual boundaries included in the SPS may be used for loop filtering. If information regarding the location of virtual boundaries is not included in the SPS but information regarding the delta value of the location of virtual boundaries is included in the picture header, the information regarding the location of virtual boundaries included in the picture header may be used for loop filtering. If information regarding the location of virtual boundaries is included in the SPS and information regarding the delta value of the location of virtual boundaries is included in the picture header, the location of virtual boundaries may be derived based on the sum of the information regarding the location of virtual boundaries signaled in the SPS and the associated delta value. If information regarding the location of virtual boundaries is not included in the SPS and information regarding the delta value of the location of virtual boundaries is not included in the picture header, a virtual boundary may not be applied to the picture.

[0245] The following table shows an exemplary syntax of SPS according to this embodiment.

[0246] [Table 29]

[0247] The following table shows exemplary semantics for the syntax elements included in the above syntax.

[0248] [Table 30]

[0249] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0250]

Table 31

[0251] The following table shows exemplary semantics regarding the syntax elements included in the above syntax.

[0252]

Table 32-1

[0253]

Table 32-2

[0254] In embodiments related to Tables 29 to 32, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH).

[0255] The above SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the above virtual boundary enable flag, the above SPS may include a virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, if the value of the above virtual boundary enable flag is 1, the above SPS may include a virtual boundary presence flag for the SPS. Based on the above virtual boundary enable flag and the above SPS virtual boundary presence flag, the above SPS may include information about the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information about the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information about the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information about the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]).

[0256] For example, if the value of the virtual boundary enable flag is 1 and the value of the virtual boundary existence flag of the SPS is 1, the SPS may include information about the number of horizontal virtual boundaries, information about the positions of the horizontal virtual boundaries, information about the number of vertical virtual boundaries, and information about the positions of the vertical virtual boundaries. In one example, the number of pieces of information about the positions of the horizontal virtual boundaries can be determined based on the information about the number of horizontal virtual boundaries, and the number of pieces of information about the positions of the vertical virtual boundaries can be determined based on the information about the number of vertical virtual boundaries. The picture header may include the virtual boundary existence flag of the PH based on the virtual boundary enable flag.

[0257] For example, if the value of the virtual boundary enable flag is 1, the picture header may include the virtual boundary existence flag for the PH. Based on the virtual boundary existence flag for the PH and the information regarding the number of vertical virtual boundaries of the SPS, the picture header may include information regarding the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries). For example, if the value of the virtual boundary existence flag for the PH is 1 and the value of the information regarding the number of vertical virtual boundaries of the SPS is 0, the picture header may include information regarding the number of vertical virtual boundaries of the PH. In one example, based on the information regarding the number of vertical virtual boundaries of the PH, the picture header may include information regarding the delta value of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_x_delta[i]) and information regarding the sign of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_x_sign[i]). In one example, based on the information regarding the number of vertical virtual boundaries of the PH, the number of pieces of information regarding the delta value of the position of the vertical virtual boundary of the PH and the number of pieces of information regarding the sign of the position of the vertical virtual boundary of the PH can be determined. The picture header described above may include information regarding the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries) based on the virtual boundary existence flag of the PH and the information regarding the number of horizontal virtual boundaries of the SPS.

[0258] For example, if the value of the virtual boundary existence flag for the above PH is 1 and the value of the information regarding the number of horizontal virtual boundaries for the above SPS is 0, the picture header may include information regarding the number of horizontal virtual boundaries for the above PH. In one example, the picture header may include information regarding the delta value of the position of the horizontal virtual boundary of the above PH (ph_virtual_boundaries_pos_y_delta[i]) and information regarding the sign of the position of the horizontal virtual boundary of the above PH (ph_virtual_boundaries_pos_y_sign[i]) based on the information regarding the number of horizontal virtual boundaries of the above PH. In one example, the number of pieces of information regarding the delta value of the position of the horizontal virtual boundary of the above PH and the number of pieces of information regarding the sign of the position of the horizontal virtual boundary of the above PH can be determined based on the information regarding the number of horizontal virtual boundaries of the above PH.

[0259] Along with the table above, according to the embodiments of this document, the coding device can efficiently signal the information necessary to control in-loop filtering performed across virtual boundaries. In one example, information related to whether in-loop filtering is available across virtual boundaries can be signaled.

[0260] Figures 8 and 9 schematically illustrate an example of a video / image encoding method and related components according to the embodiments described in this document.

[0261] The method disclosed in Figure 8 can be performed by the encoding device disclosed in Figure 2 or Figure 9. Specifically, for example, steps S800 and S810 in Figure 8 can be performed by the residual processing unit 230 of the encoding device in Figure 9, steps S820 and / or S830 in Figure 8 can be performed by the filtering unit 260 of the encoding device in Figure 9, and step S840 in Figure 8 can be performed by the entropy encoding unit 240 of the encoding device in Figure 9. Although not shown in Figure 8, the prediction unit 220 of the encoding device in Figure 8 can derive prediction samples or prediction-related information, and the entropy encoding unit 240 of the encoding device can generate a bitstream from the residual information or prediction-related information. The method disclosed in Figure 8 may include the embodiments described above.

[0262] Referring to Figure 8, the encoding device can derive residual samples (S800). The encoding device can derive residual samples for the current block, and these residual samples for the current block can be derived based on the original and predicted samples of the current block. Specifically, the encoding device can derive predicted samples for the current block based on the prediction mode. In this case, various prediction methods disclosed in this document, such as interpretation or intraprediction, can be applied. Residual samples can be derived based on the predicted and original samples.

[0263] The encoding device can derive conversion coefficients. The encoding device can derive conversion coefficients based on the conversion procedure for the residual sample described above. For example, the conversion procedure may include at least one of DCT, DST, GBT, or CNT.

[0264] The encoding device can derive quantized transformation coefficients. The encoding device can derive quantized transformation coefficients based on a quantization procedure for the above transformation coefficients. The quantized transformation coefficients may take the form of a one-dimensional vector based on the coefficient scan order.

[0265] The encoding device can generate residual information (S810). The encoding device can generate residual information based on the residual samples for the current block. The encoding device can generate residual information showing the quantized conversion coefficients. The residual information can be generated through various encoding methods such as exponential Golomb, CAVLC, and CABAC.

[0266] The encoding device can generate reconstructed samples. The encoding device can generate reconstructed samples based on the residual information described above. The reconstructed samples can be generated by adding the residual samples based on the residual information to the predicted samples. Specifically, the encoding device can perform a prediction (intra or inter prediction) for the current block and generate reconstructed samples based on the original samples and the predicted samples generated from the predictions.

[0267] The reconstructed sample may include a reconstructed luma sample and a reconstructed chroma sample. Specifically, the residual sample may include a residual luma sample and a residual chroma sample. The residual luma sample can be generated based on the original luma sample and the predicted luma sample. The residual chroma sample can be generated based on the original chroma sample and the predicted chroma sample. The encoding device can derive conversion coefficients (luma conversion coefficients) for the residual luma sample and / or conversion coefficients (chroma conversion coefficients) for the residual chroma sample. The quantized conversion coefficients may include quantized luma conversion coefficients and / or quantized chroma conversion coefficients.

[0268] The encoding device can determine whether the in-loop filtering procedure described above is performed across the virtual boundary (S820). Here, the virtual boundary may be the same as the virtual boundary described above. The in-loop filtering procedure may include at least one of the deblocking procedure, the SAO procedure, or the ALF procedure.

[0269] The encoding device can generate virtual boundary-related information (S830). Based on the above determination in step S820, the encoding device can generate virtual boundary-related information. The virtual boundary-related information may be included in the in-loop filtering-related information. Here, the in-loop filtering-related information may refer to the information used to perform the above in-loop filtering procedure. For example, the virtual boundary-related information may include the information about the virtual boundary mentioned in this document (such as the virtual boundary enable flag of the SPS, the virtual boundary enable flag of the picture header, the virtual boundary existence flag of the SPS, the virtual boundary existence flag of the picture header, and information about the location of the virtual boundary).

[0270] The encoding device can encode video / image information (S840). The image information may include residual information, prediction-related information, and / or in-loop filtering-related information. The encoded video / image information can be output in the form of a bitstream. The bitstream can be transmitted to a decoding device via a network or storage medium.

[0271] The above image / video information may include a variety of information according to the embodiments described herein. For example, the above image / video information may include information disclosed in at least one of Tables 1 to 32 described above.

[0272] In one embodiment, the image information may include an SPS (Sequence Parameter Set) and picture header information referencing the SPS. The virtual boundary-related information may include a virtual boundary enable flag (or a virtual boundary enable flag for the SPS). Based on the virtual boundary enable flag, it can be determined whether signaling for the virtual boundary-related information exists in the SPS or the picture header information. The in-loop filtering procedure may be executed across the virtual boundary (or not executed across the virtual boundary) based on the virtual boundary enable flag. For example, the virtual boundary enable flag may indicate whether it is possible to disable the in-loop filtering procedure across the virtual boundary.

[0273] In one embodiment, the SPS includes a virtual boundary enable flag and a virtual boundary existence flag for the SPS, and based on the virtual boundary existence flag, it can be determined whether the SPS contains information regarding the location of the virtual boundary and information regarding the number of virtual boundaries.

[0274] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of vertical virtual boundaries.

[0275] In one embodiment, the SPS may include information regarding the location of a vertical virtual boundary. Furthermore, the number of pieces of information regarding the location of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0276] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of horizontal virtual boundaries.

[0277] In one embodiment, the SPS may include information about the location of a horizontal virtual boundary. Furthermore, the number of pieces of information about the location of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0278] In one embodiment, based on the value of the virtual boundary enable flag being 1 and the value of the virtual boundary existence flag of the SPS being 0, the picture header information may include the virtual boundary existence flag of the picture header.

[0279] In one embodiment, based on the value of the virtual boundary existence flag in the picture header being 1, the picture header information may include information regarding the number of vertical virtual boundaries.

[0280] In one embodiment, the picture header information may include information about the position of a vertical virtual boundary. Furthermore, the number of pieces of information about the position of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0281] In one embodiment, based on the value of the virtual boundary existence flag in the picture header being 1, the picture header information may include information regarding the number of horizontal virtual boundaries.

[0282] In one embodiment, the picture header information may include information about the position of a horizontal virtual boundary. Furthermore, the number of pieces of information about the position of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0283] In one embodiment, based on the fact that the SPS includes information about the position of vertical virtual boundaries and information about the position of horizontal virtual boundaries, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be greater than 0.

[0284] In one embodiment, the in-loop filtering-related information (and / or virtual boundary-related information) may further include a virtual boundary presence flag for the SPS, a virtual boundary presence flag for the picture header, and a Gradual Decoding Refresh (GDR) enable flag. For example, based on the value of the GDR enable flag being 1, the value of the virtual boundary enable flag for the SPS (virtual boundary enable flag) may be 1, the value of the virtual boundary presence flag for the SPS may be 0, and the value of the virtual boundary presence flag for the picture header may be 1 (signaling of virtual boundary information may be present in the picture header).

[0285] Figures 10 and 11 schematically illustrate an example of a video / image decoding method and related components according to the embodiments described in this document.

[0286] The method disclosed in Figure 10 can be performed by the decoding device disclosed in Figure 3 or Figure 11. Specifically, for example, S1000 in Figure 10 can be performed by the entropy decoding unit 310 of the decoding device, S1010 can be performed by the residual processing unit 320 and / or adder 340 of the decoding device, and S1020 can be performed by the filtering unit 350 of the decoding device. The method disclosed in Figure 10 may include the embodiments described above in this document.

[0287] Referring to Figure 10, the decoding device can receive / acquire video / image information (S1000). The video / image information may include residual information, prediction-related information, and / or in-loop filtering-related information. The decoding device can receive / acquire the above image / video information via a bitstream.

[0288] The above image / video information may include a variety of information according to the embodiments described herein. For example, the above image / video information may include information disclosed in at least one of Tables 1 to 32 mentioned above.

[0289] The decoding device can derive quantized transformation coefficients. The decoding device can derive quantized transformation coefficients based on the residual information described above. The quantized transformation coefficients may take the form of a one-dimensional vector based on the coefficient scan order. The quantized transformation coefficients may include quantized luma transformation coefficients and / or quantized chroma transformation coefficients.

[0290] The decoding device can derive conversion coefficients. The decoding device can derive conversion coefficients based on an inverse quantization procedure for the above quantized conversion coefficients. The decoding device can derive Luma conversion coefficients via inverse quantization based on the quantized Luma conversion coefficients. The decoding device can derive Chroma conversion coefficients via inverse quantization based on the quantized Chroma conversion coefficients.

[0291] The decoding device can generate / derive residual samples. The decoding device can derive residual samples based on the inverse transformation procedure for the above transformation coefficients. The decoding device can derive residual luma samples via the inverse transformation procedure based on luma transformation coefficients. The decoding device can derive residual chroma samples via the inverse transformation procedure based on chroma transformation coefficients.

[0292] The decoding device can generate / derive a reconstructed sample (S1010). For example, the decoding device can generate / derive a reconstructed luminal sample and / or a reconstructed chromatic sample. The decoding device can generate a reconstructed luminal sample and / or a reconstructed chromatic sample based on the residual information. The decoding device can generate a reconstructed sample based on the residual information. The reconstructed sample may include a reconstructed luminal sample and / or a reconstructed chromatic sample. The luminal component of the reconstructed sample may correspond to the reconstructed luminal sample, and the chromatic component of the reconstructed sample may correspond to the reconstructed chromatic sample. The decoding device can generate a predicted luminal sample and / or a predicted chromatic sample via a prediction procedure. The decoding device can generate a reconstructed luminal sample based on the predicted luminal sample and the residual luminal sample. The decoding device can generate a reconstructed chromatic sample based on the predicted chromatic sample and the residual chromatic sample.

[0293] The decoding device can generate a corrected (filtered) reconstructed sample (S1020). The decoding device can generate a corrected reconstructed sample based on an in-loop filtering procedure on the reconstructed sample. The decoding device can generate a corrected reconstructed sample based on in-loop filtering related information. The decoding device can use a deblocking procedure, an SAO procedure, and / or an ALF procedure to generate a corrected reconstructed sample.

[0294] In one embodiment, the image information may include an SPS and picture header information referencing the SPS. The virtual boundary-related information may include a virtual boundary enable flag (or a virtual boundary existence flag for the SPS). Based on the virtual boundary enable flag, it can be determined whether signaling for the virtual boundary-related information exists in the SPS or the picture header information. The in-loop filtering procedure may be executed across the virtual boundary (or not executed) based on the virtual boundary enable flag. For example, the virtual boundary enable flag may indicate whether it is possible to disable the in-loop filtering procedure across the virtual boundary.

[0295] In one embodiment, the SPS may include a virtual boundary enable flag and / or a virtual boundary existence flag for the SPS. Based on the virtual boundary existence flag for the SPS, it can be determined whether the SPS contains information regarding the location of the virtual boundary and information regarding the number of the virtual boundary.

[0296] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of vertical virtual boundaries.

[0297] In one embodiment, the SPS may include information regarding the location of a vertical virtual boundary. Furthermore, the number of pieces of information regarding the location of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0298] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of horizontal virtual boundaries.

[0299] In one embodiment, the SPS may include information about the location of a horizontal virtual boundary. Furthermore, the number of pieces of information about the location of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0300] In one embodiment, based on the value of the virtual boundary enable flag being 1 and the value of the virtual boundary existence flag of the SPS being 0, the picture header information may include the virtual boundary existence flag of the picture header.

[0301] In one embodiment, based on the value of the virtual boundary existence flag in the picture header being 1, the picture header information may include information regarding the number of vertical virtual boundaries.

[0302] In one embodiment, the picture header information may include information about the position of a vertical virtual boundary. Furthermore, the number of pieces of information about the position of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0303] In one embodiment, based on the value of the virtual boundary existence flag in the picture header being 1, the picture header information may include information regarding the number of horizontal virtual boundaries.

[0304] In one embodiment, the picture header information may include information about the position of a horizontal virtual boundary. Furthermore, the number of pieces of information about the position of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0305] In one embodiment, based on the fact that the SPS includes information about the position of vertical virtual boundaries and information about the position of horizontal virtual boundaries, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be greater than 0.

[0306] In one embodiment, the in-loop filtering-related information (and / or virtual boundary-related information) may further include a virtual boundary presence flag for the SPS, a virtual boundary presence flag for the picture header, and a Gradual Decoding Refresh (GDR) enable flag. For example, based on the value of the GDR enable flag being 1, the value of the virtual boundary enable flag for the SPS (virtual boundary enable flag) may be 1, the value of the virtual boundary presence flag for the SPS may be 0, and the value of the virtual boundary presence flag for the picture header may be 1 (signaling of virtual boundary information may be present in the picture header).

[0307] The decoding device can receive information about residuals relative to the current block, if residual samples exist for the current block. This residual information may include transformation coefficients for the residual samples. Based on the residual information, the decoding device can derive residual samples (or residual sample arrays) for the current block. Specifically, the decoding device can derive quantized transformation coefficients based on the residual information. These quantized transformation coefficients may have a one-dimensional vector form based on the coefficient scan order. The decoding device can derive transformation coefficients based on an inverse quantization procedure applied to the above quantized transformation coefficients. Based on these transformation coefficients, the decoding device can derive residual samples.

[0308] The decoding device can generate a reconstructed sample based on the (intra) predicted sample and the residual sample, and can derive a reconstructed block or reconstructed picture based on the reconstructed sample. Specifically, the decoding device can generate a reconstructed sample based on the sum of the (intra) predicted sample and the residual sample. Subsequently, as described above, the decoding device can apply deblocking filtering and / or in-loop filtering procedures such as the SAO procedure to the reconstructed picture as needed to improve subjective / objective image quality.

[0309] For example, a decoding device can decode a bitstream or encoded information to obtain image information that includes all or part of the aforementioned information (or syntax elements). Furthermore, the bitstream or encoded information can be stored in a computer-readable storage medium, and the aforementioned decoding method can be performed.

[0310] In the embodiments described above, the method is explained based on a flowchart as a series of steps or blocks, but the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and that different steps may be included, or one or more steps in the flowchart may be omitted without affecting the scope of the embodiments described herein.

[0311] The methods according to the embodiments of this document described above can be implemented in software form, and the encoding and / or decoding devices relating to this document may be included in, for example, image processing devices such as TVs, computers, smartphones, set-top boxes, and display devices.

[0312] In this document, when embodiments are implemented in software, the methods described above can be implemented by modules (processes, functions, etc.) that perform the functions described above. These modules are stored in memory and can be executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by various well-known means. The processor may include an ASIC (Application-Specific Integrated Circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (Read-Only Memory), RAM (Random Access Memory), flash memory, memory cards, storage media, and / or other storage devices. In other words, the embodiments described in this document may be implemented on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing may be implemented on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0313] Furthermore, the decoding and encoding devices to which the embodiments of this document apply may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communication devices, mobile streaming devices, storage media, camcorders, video-on-demand (VoD) service providers, OTT video (Over The Top video) devices, internet streaming service providers, 3D video devices, VR (Virtual Reality) devices, AR (Augmented Reality) devices, image-phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video equipment, and may be used to process video signals or data signals. For example, OTT video (Over The Top video) devices may include game consoles, Blu-ray players, internet-access TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0314] Furthermore, the processing methods to which the embodiments of this document apply can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having a data structure according to the embodiments of this document can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices on which data to be read by a computer is stored. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), Universal Serial Bus (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method can be stored on a computer-readable recording medium or transmitted over a wireless network.

[0315] Furthermore, the embodiments described in this document can be implemented as a computer program product using program code, and the program code can be executed on a computer according to the embodiments described in this document. The program code can be stored on a computer-readable carrier.

[0316] Figure 12 shows an example of a content streaming system to which the embodiments disclosed in this document may be applied.

[0317] Referring to Figure 12, the content streaming system to which the embodiments described in this document apply can broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0318] The above-mentioned encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then transmitting this bitstream to the above-mentioned streaming server. As an alternative example, if the multimedia input device such as a smartphone, camera, or camcorder generates the bitstream directly, the above-mentioned encoding server may be omitted.

[0319] The bitstream described above can be generated by an encoding method or a bitstream generation method to which the embodiments of this document apply, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0320] The streaming server transmits multimedia data to user devices based on user requests via the web server, and the web server acts as an intermediary to inform the user about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0321] The above-mentioned streaming server can receive content from a media storage device (storage) and / or an encoding server. For example, when receiving content from the above-mentioned encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the above-mentioned streaming server can store the above-mentioned bitstream for a certain period of time.

[0322] Examples of user devices mentioned above include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), navigation systems, slate PCs, tablet PCs, ultrabooks (ULTRABOOK®), wearable devices (such as smartwatches, smart glasses, and HMDs (Head Mounted Displays)), digital TVs, desktop computers, and digital signatures (Signiji).

[0323] Each server within the above content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0324] The claims described herein can be combined in various ways. For example, the technical features of the method claims herein can be combined to realize an apparatus, and the technical features of the apparatus claims herein can be combined to realize a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize an apparatus, and the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize a method.

Claims

1. In a decoding device for image decoding, Memory and The system comprises at least one processor connected to the memory, The aforementioned at least one processor is Image information, including residual information, virtual boundary-related information, and prediction-related information, is acquired via a bitstream. Based on the residual information, a residual sample is generated. Based on the aforementioned prediction-related information, a prediction sample is derived. Based on the predicted sample and the residual sample, a restored sample of the current picture is generated. It is configured to generate a corrected restored sample based on an in-loop filtering procedure on the restored sample, The aforementioned image information includes an SPS (Sequence Parameter Set) and picture header information that references the SPS. The information relating to the virtual boundary includes a virtual boundary enable flag, The SPS includes an SPS virtual boundary existence flag based on the virtual boundary enable flag, Based on the virtual boundary enable flag, it is determined whether the signaling of the information related to the virtual boundary exists in the SPS, or whether the signaling exists in the picture header information. Based on the virtual boundary enable flag, it is determined whether the in-loop filtering procedure is performed across the virtual boundary. The SPS includes information regarding the number of vertical virtual boundaries and information regarding the number of horizontal virtual boundaries, based on the value of the SPS virtual boundary existence flag being 1. The SPS further includes information regarding the position of a vertical virtual boundary and information regarding the position of a horizontal virtual boundary, based on the value of the SPS virtual boundary existence flag being 1.

2. In an encoding device for image encoding, Memory and The system comprises at least one processor connected to the memory, The aforementioned at least one processor is Derive prediction-related information for the current block, Based on the aforementioned prediction-related information, a prediction sample is generated for the current block. Based on the predicted sample, the residual sample for the current block is derived, Based on the residual sample for the current block, residual information is generated. Based on the predicted sample and the residual sample, a restored sample is generated for the current picture. Determine whether the in-loop filtering procedure is performed across the virtual boundary. Based on the above decision, information related to the virtual boundary is generated, It is configured to encode image information including the prediction-related information, the residual information, and the information related to the virtual boundary, The aforementioned image information includes an SPS (Sequence Parameter Set) and picture header information that references the SPS. The information relating to the virtual boundary includes a virtual boundary enable flag, The SPS includes an SPS virtual boundary existence flag based on the virtual boundary enable flag, Based on the virtual boundary enable flag, it is determined whether the signaling of the information related to the virtual boundary exists in the SPS, or whether the signaling exists in the picture header information. Based on the virtual boundary enable flag, it is determined whether the in-loop filtering procedure is performed across the virtual boundary. The SPS includes information regarding the number of vertical virtual boundaries and information regarding the number of horizontal virtual boundaries, based on the value of the SPS virtual boundary existence flag being 1. The SPS further includes information regarding the position of a vertical virtual boundary and information regarding the position of a horizontal virtual boundary, based on the value of the SPS virtual boundary existence flag being 1.

3. In a device for transmitting image-related data, At least one processor configured to acquire a bitstream relating to the aforementioned image, wherein the bitstream is Steps include: deriving prediction-related information for the current block, The steps include generating prediction samples for the current block based on the prediction-related information, The steps include: deriving a residual sample for the current block based on the predicted sample; The steps include generating residual information based on the residual sample for the current block, A step of generating a reconstructed sample for the current picture based on the predicted sample and the residual sample, A step to determine whether the in-loop filtering procedure is performed across the virtual boundary, Based on the above decision, the steps include generating information related to the virtual boundary, A processor generated based on the step of encoding image information including the prediction-related information, the residual information, and the information related to the virtual boundary, A transmitting unit configured to transmit the data including the bitstream, The aforementioned image information includes an SPS (Sequence Parameter Set) and picture header information that references the SPS. The information relating to the virtual boundary includes a virtual boundary enable flag, The SPS includes an SPS virtual boundary existence flag based on the virtual boundary enable flag, Based on the virtual boundary enable flag, it is determined whether the signaling of the information related to the virtual boundary exists in the SPS, or whether the signaling exists in the picture header information. Based on the virtual boundary enable flag, it is determined whether the in-loop filtering procedure is performed across the virtual boundary. The SPS includes information regarding the number of vertical virtual boundaries and information regarding the number of horizontal virtual boundaries, based on the value of the SPS virtual boundary existence flag being 1. The SPS further includes information regarding the position of a vertical virtual boundary and information regarding the position of a horizontal virtual boundary, based on the value of the SPS virtual boundary existence flag being 1.

Images