Image coding apparatus and method for controlling loop filtering

Abstract

To provide a decoding method by a decoding device.SOLUTION: In a decoding method by a decoding device according to an embodiment of this document, it can be determined whether signaling of information related to a virtual boundary exists in an SPS or picture header information on the basis of a virtual boundary availability flag.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] This document relates to an image coding apparatus and method for controlling loop filtering.

Background Art

[0002] In recent years, the demand for high-resolution and high-quality images / videos such as 4K or UHD (Ultra High Definition) images / videos of 8K or higher has been increasing in various fields. As the image / video data becomes higher in resolution and quality, the amount of information or bits transmitted relatively increases compared to the existing image / video data. Therefore, when transmitting image data using a medium such as an existing wired or wireless broadband line or storing image / video data using an existing storage medium, the transmission cost and storage cost increase.

[0003] In addition, in recent years, the interest and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality) contents, and holograms have been increasing, and the broadcast of images / videos having image characteristics different from real images, such as game images, has been increasing.

[0004] Thus, a highly efficient image / video compression technique is required to effectively compress, transmit, store, and reproduce the information of high-resolution and high-quality images / videos having various characteristics as described above.

[0005] Specifically, loop filtering can be used for image / video compression. There has been a discussion on a scheme for efficiently signaling the information for controlling loop filtering.

Summary of the Invention

Means for Solving the Problems

[0006] According to one embodiment of this document, a method and apparatus for enhancing the efficiency of image / video coding are provided.

[0007] According to one embodiment of this document, an efficient filtering method and apparatus are provided.

[0008] According to one embodiment of this document, a method and apparatus for efficiently applying deblocking, SAO (sample adaptive loop), and ALF (adaptive loop filtering) are provided.

[0009] According to one embodiment of this document, in-loop filtering is performed based on a virtual boundary.

[0010] According to one embodiment of this document, the SPS (sequence parameter set) may include a virtual boundary availability flag for the SPS that indicates whether in-loop filtering is performed across the virtual boundary.

[0011] According to one embodiment of this document, in-loop filtering can be performed across virtual boundaries based on the virtual boundary availability flag of the SPS.

[0012] According to one embodiment of this document, an encoding device for video / image encoding is provided.

[0013] According to one embodiment of this document, a computer-readable digital storage medium is provided which stores encoded video / image information generated by a video / image encoding method disclosed in at least one embodiment of this document.

[0014] According to one embodiment of this document, a computer-readable digital storage medium is provided which stores encoded information or encoded video / image information that causes a decoding device to perform a video / image decoding method disclosed in at least one embodiment of this document. [Effects of the Invention]

[0015] According to one embodiment of this document, the overall image / video compression efficiency can be improved.

[0016] According to one embodiment of this document, subjective / objective visual quality can be enhanced through efficient filtering.

[0017] One embodiment of this document provides an in-loop filtering procedure based on a virtual boundary that can save hardware resources.

[0018] According to one embodiment of this document, an in-loop filtering procedure based on a virtual boundary can be efficiently performed, and filtering performance can be improved.

[0019] According to one embodiment of this document, information for in-loop filtering based on virtual boundaries can be efficiently signaled. [Brief explanation of the drawing]

[0020] [Figure 1] An example of a video / image coding system that can be applied to the embodiments described herein is schematically shown. [Figure 2] This diagram schematically illustrates the configuration of a video / image encoding device that can be applied to the embodiments described herein. [Figure 3] This diagram schematically illustrates the configuration of a video / image decoding device that can be applied to the embodiments described herein. [Figure 4] This illustrates the hierarchical structure for coded images / videos. [Figure 5] An example of an ALF filter shape is shown. [Figure 6] One embodiment of this document provides an example of an ALF procedure using a virtual boundary. [Figure 7] This is a flowchart illustrating a filtering-based encoding method in an encoding device. [Figure 8]It is a flowchart for explaining a decoding method based on filtering in a decoding device. [Figure 9] An example of a video / image encoding method and related components according to an embodiment of this document is schematically shown. [Figure 10] An example of a video / image encoding method and related components according to an embodiment of this document is schematically shown. [Figure 11] An example of an image / video decoding method and related components according to an embodiment of this document is schematically shown. [Figure 12] An example of an image / video decoding method and related components according to an embodiment of this document is schematically shown. [Figure 13] An example of a content streaming system to which the embodiments disclosed in this document can be applied is shown.

Embodiments for Carrying Out the Invention

[0021] This document can be modified in various ways, can have various embodiments, and specific embodiments are illustrated in the drawings and will be described in detail. However, this is not intended to limit this document to specific embodiments. The terms commonly used in this specification are merely used to describe specific embodiments and are not intended to limit the technical idea of this document. Singular expressions include plural expressions unless the context clearly indicates otherwise. Terms such as "including" or "having" in this specification are intended to specify the presence of the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and it should be understood that the presence or possibility of addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

[0022] On the other hand, each configuration shown in the diagrams described in this document is illustrated independently for the purpose of explaining its distinct characteristic functions, and does not mean that each configuration is implemented with separate hardware or separate software. For example, two or more of the configurations may be combined to form a single configuration, and one configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of the rights of this document, as long as they do not deviate from the essence of this document.

[0023] Preferred embodiments of this document will be described in more detail below with reference to the attached drawings. Hereafter, the same reference numerals will be used for identical components in the drawings, and redundant descriptions of identical components will be omitted.

[0024] This document relates to video / image coding. For example, the methods / embodiments disclosed in this document relate to the VVC (Versatile Video Coding) standard (ITU-T Rec.H.266), next-generation video / image coding standards after VVC, or other video coding-related standards (e.g., HEVC (High Efficiency Video Coding) standard (ITU-T Rec.H.265), EVC (essential video coding) standard, AVS2 standard, etc.).

[0025] This document presents various embodiments relating to video / image coding, and unless otherwise noted, these embodiments may be implemented in combination with each other.

[0026] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a single image representing a specific time period, while "slice" or "tile" is a unit that constitutes part of a picture in coding. A slice or tile may contain one or more CTUs (coding tree units). A single picture may consist of one or more slices or tiles. A single picture may consist of one or more tile groups. A tile group may contain one or more tiles.

[0027] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" may be used as a counterpart to pixel. A sample generally refers to a pixel or a pixel value, and may refer only to the pixel / pixel value of the luma component, or only to the pixel / pixel value of the chroma component. Alternatively, a sample can refer to a pixel value in the spatial domain, and if such a pixel value is converted to the frequency domain, it can also refer to the conversion coefficient in the frequency domain.

[0028] A unit represents a basic unit of image processing. A unit contains at least one of the following: a specific region of a picture and information about that region. One unit contains one luma block and two chroma (e.g., cb, cr) blocks. The term unit may be used interchangeably with terms such as block or area. In general, an M×N block contains a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0029] In this document, " / " and "," are interpreted as "and / or". For example, "A / B" is interpreted as "A and / or B", and "A, B" is interpreted as "A and / or B". Additionally, "A / B / C" means "at least one of A, B and / or C". Similarly, "A, B, C" also means "at least one of A, B and / or C".

[0030] Additionally, in this document, "or" is interpreted as "and / or". For example, "A or B" could mean 1) only "A", 2) only "B", or 3) "A and B". In other words, "or" in this document can mean "additionally or alternatively".

[0031] In this specification, "at least one of A and B" may mean "A only," "B only," or "both A and B." Furthermore, in this specification, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted similarly to "at least one of A and B."

[0032] Furthermore, in this specification, "at least one of A, B and C" may mean "A only," "B only," "C only," or "any combination of A, B and C." Also, "at least one of A, B or C" or "at least one of A, B and C" may mean "at least one of A, B and C."

[0033] Furthermore, parentheses used in this specification may mean "for example." Specifically, when "prediction (intra-prediction)" is indicated, "intra-prediction" may be proposed as an example of "prediction." In other words, "prediction" in this specification is not limited to "intra-prediction," and "intra-prediction" may be proposed as an example of "prediction." Also, when "prediction (i.e., intra-prediction)" is indicated, "intra-prediction" may be proposed as an example of "prediction."

[0034] Technical features described individually within a single drawing in this specification may be implemented individually or simultaneously.

[0035] Figure 1 schematically shows an example of a video / image coding system to which this document can be applied.

[0036] As shown in Figure 1, a video / image coding system may comprise a source device and a receiving device. The source device can transmit encoded video / image information or data to the receiving device in file or streaming form via a digital storage medium or network.

[0037] The source device may comprise a video source, an encoding device, and a transmitter. The receiving device may comprise a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be provided in the encoding device. The receiver may be provided in the decoding device. The renderer may comprise a display unit, which may consist of a separate device or external component.

[0038] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by the process of generating the associated data.

[0039] An encoding device can encode input video / images. For compression and coding efficiency, the encoding device can perform a series of steps including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in bitstream format.

[0040] The transmitting unit can transmit encoded video / image information or data output in bitstream format to the receiving unit of a receiving device via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitting unit may include elements for generating media files via a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0041] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of the encoding device.

[0042] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0043] Figure 2 is a schematic diagram illustrating the configuration of a video / image encoding device to which this document applies. Hereinafter, the term "video encoding device" may include an image encoding device.

[0044] As shown in Figure 2, the encoding device 200 can be configured to include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-prediction unit 221 and an intra-prediction unit 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be called a reconstructor or a reconstructed block generator. The aforementioned image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, addition unit 250, and filtering unit 260 can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. The memory 270 may also include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware components may further include the memory 270 as an internal / external component.

[0045] The image splitting unit 210 can split an input image (or picture, frame) input to the encoding device 200 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, the coding units can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary tree structure and / or the ternary structure. Alternatively, the binary tree structure may be applied first. The coding procedure according to this disclosure may be performed based on the final coding unit that is not further split. In this case, based on coding efficiency due to image characteristics, the largest coding unit can be used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further comprise a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can each be separated or partitioned from the final coding unit described above.The prediction unit may be a unit of sample prediction, and the conversion unit may be a unit for deriving conversion coefficients and / or a unit for deriving a residual signal from conversion coefficients.

[0046] The term "unit" can sometimes be used interchangeably with terms such as "block" or "area." Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used to refer to a single picture (or image) as a pixel or pel.

[0047] The subtraction unit 231 can generate a residual signal (residual block, residual sample, or residual sample array) by subtracting the predicted signal (predicted block, predicted sample, or predicted sample array) output from the prediction unit 220 from the input image signal (original block, original sample, or original sample array), and the generated residual signal is transmitted to the conversion unit 232. The prediction unit 220 can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes a predicted sample for the current block. The prediction unit 220 can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. As will be described later in the explanation of each prediction mode, the prediction unit can generate various information related to prediction, such as prediction mode information, and transmit it to the entropy encoding unit 240. The information related to prediction can be encoded by the entropy encoding unit 240 and output in bitstream form.

[0048] The intra-prediction unit 222 can predict the current block by referring to a sample in the current picture. The referenced sample can be located adjacent to the current block or at a distance, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of prediction direction. However, this is illustrative, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 222 can also determine the prediction mode to apply to the current block using the prediction modes applied to adjacent blocks.

[0049] The interprediction unit 221 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, col CU, etc., and the reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the inter-prediction unit 221 can construct a motion information candidate list based on adjacent blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-prediction can be performed based on various prediction modes; for example, in skip mode and merge mode, the inter-prediction unit 221 can use the motion information of adjacent blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of an adjacent block is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0050] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra-prediction (CIIP). The prediction unit can also perform intra-block copy (IBC) for predictions on blocks. The intra-block copy can be used for content image / video coding in games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can use at least one of the inter-prediction techniques described in this document.

[0051] The prediction signal generated via the interpretation unit 221 and / or intrapretation unit 222 can be used to generate a reconstructed signal or a residual signal. The transformation unit 232 can apply a transformation technique to the residual signal to generate transformation coefficients. For example, the transformation technique may include DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph when relational information between pixels is represented by this graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels and based on that. The transformation process may also be applied to pixel blocks of the same size and square shape, or to non-square blocks of variable size.

[0052] The quantization unit 233 quantizes the conversion coefficients and transmits them to the entropy encoding unit 240, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients can be called residual information. The quantization unit 233 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector form of the quantized conversion coefficients. The entropy encoding unit 240 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy encoding unit 240 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together with or separately from the quantized conversion coefficients. Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form in network abstraction layer (NAL) units. The video / image information may further include information about various parameter sets, such as adaptation parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. In this document, the signaling / transmitted information and / or syntax elements described later may be encoded via the encoding procedure described above and included in the bitstream. The bitstream may be transmitted over a network or stored on a digital storage medium.Here, the network may include broadcasting networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 240 can be transmitted by a transmitting unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 200, or the transmitting unit may be included in the entropy encoding unit 240.

[0053] The quantized conversion coefficients output from the quantization unit 233 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 234 and the inverse transformation unit 235. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample, or reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the prediction unit 220. If there is no residual for the block to be processed, as in the case of skip mode, the predicted block can be used as the reconstructed block. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, and can also be used for inter-prediction of the next picture after filtering, as described later.

[0054] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture encoding and / or restoration process.

[0055] The filtering unit 260 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 260 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 270, specifically in the DPB of the memory 270. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset (SAO), adaptive loop filter, and bilateral filter. The filtering unit 260 can generate various filtering-related information and transmit it to the entropy encoding unit 240, as will be described later in the description of each filtering method. The filtering-related information can be encoded by the entropy encoding unit 240 and output in bitstream form.

[0056] The corrected restored picture sent to memory 270 can be used as a reference picture in the interpretation unit 221. When interpretation is applied via this, the encoding device can avoid prediction mismatches between the encoding device 100 and the decoding device, and can also improve encoding efficiency.

[0057] The DPB in memory 270 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 221. Memory 270 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 221 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 270 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 222.

[0058] Figure 3 is a schematic diagram illustrating the configuration of a video / image decoding device to which this document can be applied.

[0059] As shown in Figure 3, the decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-prediction unit 331 and an intra-prediction unit 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 321. The aforementioned entropy decoder 310, residual processor 320, predictor 330, adder 340, and filtering unit 350 can be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The aforementioned hardware component may also further include memory 360 as an internal / external component.

[0060] When a bitstream containing video / image information is input, the decoding device 300 can reconstruct the image corresponding to the process by which the video / image information was processed in the encoding device shown in Figure 3. For example, the decoding device 300 can derive units / blocks based on block division-related information obtained from the bitstream. The decoding device 300 can perform decoding using the processing units applied in the encoding device. Therefore, the decoding processing unit can be, for example, a coding unit, which can be divided from a coding tree unit or a maximum coding unit according to a quad-tree structure, a binary tree structure, and / or a terminally tree structure. One or more conversion units can be derived from the coding unit. The reconstructed image signal decoded and output via the decoding device 300 can then be reproduced via a playback device.

[0061] The decoding device 300 can receive the signal output from the encoding device shown in Figure 3 in bitstream form, and the received signal can be decoded via the entropy decoding unit 310. For example, the entropy decoding unit 310 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can further decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 310 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded and the decoded information of adjacent and decoded blocks or symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, performs arithmetic decoding of the bins, and generates symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 310, information related to prediction is provided to the prediction unit 330, and residual information that has been entropy decoded by the entropy decoding unit 310, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 321. In addition, of the information decoded by the entropy decoding unit 310, information related to filtering can be provided to the filtering unit 350. On the other hand, a receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external element of the decoding device 300, or the receiving unit can be a component of the entropy decoding unit 310. On the other hand, the decoding device relating to this document can be called a video / image / picture decoding device, and the decoding device can also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 310, and the sample decoder may include at least one of the inverse quantization unit 321, inverse transformation unit 322, prediction unit 330, addition unit 340, filtering unit 350, and memory 360.

[0062] The inverse quantization unit 321 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 321 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. The inverse quantization unit 321 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain the transformation coefficients.

[0063] In the inverse conversion unit 322, the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0064] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 310, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode.

[0065] The prediction unit can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for a single block, and can also apply intra-prediction and inter-prediction simultaneously. This can be called combined inter and intra prediction (CIIP). The prediction unit can also perform intra-block copying (IBC) for predictions on blocks. This intra-block copying can be used for content image / video coding in games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but can be done similarly to inter-prediction in that it derives a reference block within the current picture. That is, IBC can utilize at least one of the inter-prediction techniques described in this document. Palette mode can be considered an example of intra-coding or intra-prediction.

[0066] The intra-prediction unit 331 can predict the current block by referring to a sample in the current picture. The referenced sample can be located adjacent to or far from the current block depending on the prediction mode. In intra-prediction, the prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The intra-prediction unit 331 can also determine the prediction mode to be applied to the current block using the prediction modes applied to adjacent blocks.

[0067] The interprediction unit 332 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between adjacent blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, adjacent blocks may include spatially adjacent blocks that exist in the current picture and temporally adjacent blocks that exist in the reference picture. For example, the interprediction unit 332 can construct a motion information candidate list based on adjacent blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the mode of interprediction for the current block.

[0068] The summing unit 340 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the restored block.

[0069] The addition unit 340 may be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and can be output after filtering as described later, or it can be used for intra-prediction of the next picture.

[0070] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture decoding process.

[0071] The filtering unit 350 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 350 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 360, specifically to the DPB of the memory 360. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0072] The (modified) restored picture stored in the DPB of memory 360 can be used as a reference picture by the inter-prediction unit 332. Memory 360 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 332 for use as motion information of spatially adjacent blocks or motion information of temporally adjacent blocks. Memory 360 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 331.

[0073] In this specification, embodiments described in relation to the prediction unit 330, inverse quantization unit 321, inverse transform unit 322, and filtering unit 350 of the decoding device 300 can be applied identically to or corresponding to the prediction unit 220, inverse quantization unit 234, inverse transform unit 235, and filtering unit 260 of the encoding device 200, respectively.

[0074] As mentioned above, prediction is performed to improve compression efficiency when performing video coding. Through this, a predicted block containing predicted samples for the current block, which is the block to be coded, can be generated. Here, the predicted block contains predicted samples in the spatial domain (or pixel domain). The predicted block is derived in both the encoding and decoding devices, and the encoding device can improve image coding efficiency by signaling the decoding device with information (residual information) about the residual between the original block and the predicted block, which is not the original sample value of the original block itself. The decoding device can derive a residual block containing residual samples based on the residual information, and can generate a restored block containing restored samples by combining the residual block and the predicted block, and can generate a restored picture containing the restored block.

[0075] The residual information can be generated through transformation and quantization procedures. For example, an encoding device can signal the relevant residual information (via a bitstream) to a decoding device by deriving a residual block between the original block and the predicted block, performing a transformation procedure on the residual samples (residual sample array) contained in the residual block to derive transformation coefficients, and performing a quantization procedure on the transformation coefficients to derive quantized transformation coefficients. Here, the residual information may include information such as the value information, position information, transformation technique, transformation kernel, and quantization parameters of the quantized transformation coefficients. The decoding device can derive a residual sample (or residual block) by performing an inverse quantization / inverse transformation procedure based on the residual information. The decoding device can generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also derive a residual block by inverse quantization / inverse transformation of the quantized transformation coefficients for reference for subsequent interpretation of the picture, and generate a reconstructed picture based on this.

[0076] In this document, at least one of quantization / inverse quantization and / or transformation / inverse transformation may be omitted. If quantization / inverse quantization is omitted, the quantized transformation coefficients may be called transformation coefficients. If transformation / inverse transformation is omitted, the transformation coefficients may also be called coefficients or residual coefficients, or for consistency of expression, they may still be called transformation coefficients.

[0077] In this document, quantized transformation coefficients and transformation coefficients may be referred to as transformation coefficients and scaled transformation coefficients, respectively. In this case, residual information may include information about the transformation coefficients, which may be signaled via residual coding syntax. Transformation coefficients may be derived based on the residual information (or information about the transformation coefficients), and scaled transformation coefficients may be derived via an inverse transformation (scaling) of the transformation coefficients. Residual samples may be derived based on an inverse transformation (transformation) of the scaled transformation coefficients. This may be applied / expressed similarly in other parts of this document.

[0078] The prediction unit of the encoding / decoding device can perform interpretation on a block-by-block basis to derive predicted samples. Interpretation can indicate predictions derived in a manner dependent on data elements (e.g., sample values ​​or motion information) of pictures other than the current picture. When interpretation is applied to the current block, a predicted block (predicted sample array) for the current block can be derived based on the reference block (reference sample array) identified by the motion vector on the reference picture pointed to by the index of the reference picture. In this case, in order to reduce the amount of motion information transmitted in interpretation mode, the motion information of the current block can be predicted on a block, subblock, or sample basis based on the correlation of motion information between neighboring blocks and the current block. The motion information may include motion vectors and the index of the reference picture. The motion information may further include information on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). When interpretation is applied, neighboring blocks may include spatial neighboring blocks that exist in the current picture and temporal neighboring blocks that exist in the reference picture. The reference picture containing the aforementioned reference block and the reference picture containing the aforementioned temporally adjacent block may be the same or different. The temporally adjacent block may be referred to by names such as collocated reference block, colCU, etc., and the reference picture containing the temporally adjacent block may be referred to as collocated picture (colPic). For example, a candidate list of motion information can be constructed based on the adjacent blocks of the current block, and flags or index information can be signaled to indicate which candidate is selected (used) in order to derive the motion vector and / or index of the reference picture of the current block. Interpretation is performed based on various prediction modes; for example, in skip mode and merge mode, the motion information of the current block may be the same as the motion information of the selected adjacent block.In skip mode, unlike merge mode, the residual signal may not be transmitted. In motion vector prediction (MVP) mode, the motion vector of the selected adjacent block can be used as a motion vector predictor, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0079] The motion information may include L0 motion information and / or L1 motion information depending on the interpretation type (L0 prediction, L1 prediction, Bi prediction, etc.). A motion vector in the L0 direction may be called an L0 motion vector or MVL0, and a motion vector in the L1 direction may be called an L1 motion vector or MVL1. A prediction based on an L0 motion vector may be called an L0 prediction, a prediction based on an L1 motion vector may be called an L1 prediction, and a prediction based on both the L0 motion vector and the L1 motion vector may be called a bi (Bi) prediction. Here, an L0 motion vector may represent a motion vector associated with a reference picture list L0 (L0), and an L1 motion vector may represent a motion vector associated with a reference picture list L1 (L1). The reference picture list L0 may include pictures earlier in the output order than the current picture, and the reference picture list L1 may include pictures later in the output order than the current picture. The aforementioned earlier picture may be called a forward (reference) picture, and the aforementioned later picture may be called a reverse (reference) picture. The reference picture list L0 may include further reference pictures that are later in the output order than the current picture. In this case, the earlier picture may be indexed first in the reference picture list L0, and the later picture may be indexed afterward. The reference picture list L1 may include further reference pictures that are earlier in the output order than the current picture. In this case, the later picture may be indexed first in the reference picture list L1, and the earlier picture may be indexed afterward. Here, the output order may correspond to the POC (picture order count) order.

[0080] Figure 4 illustrates the hierarchical structure for coded images / videos.

[0081] As shown in Figure 4, coded images / videos are divided into the VCL (video coding layer), which handles the decoding process and the images / videos themselves; a lower-level system that transmits and stores the coded information; and the NAL (network abstraction layer), which exists between the VCL and the lower-level system and is responsible for network adaptation functions.

[0082] VCL can generate VCL data containing compressed image data (slice data), or generate parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages that are additionally necessary during the image decoding process.

[0083] In NAL, a NAL unit can be generated by adding header information (NAL unit header) to the RBSP (Raw Byte Sequence Payload) generated by VCL. In this case, the RBSP refers to the slice data, parameter set, SEI message, etc., generated by VCL. The NAL unit header can include NAL unit type information, which is identified by the RBSP data contained in the NAL unit.

[0084] As shown in the above diagram, NAL units can be divided into VCL NAL units and Non-VCL NAL units by the RBSP generated in VCL. VCL NAL units can mean NAL units that contain information about the image (slice data), and Non-VCL NAL units can mean NAL units that contain information necessary for decoding the image (parameter set or SEI message).

[0085] The aforementioned VCL NAL units and Non-VCL NAL units can be transmitted over a network with header information added according to the data standards of the lower-level system. For example, NAL units can be transformed into data formats of predetermined standards such as H.266 / VVC file format, RTP (Real-time Transport Protocol), and TS (Transport Stream) and transmitted over various networks.

[0086] As mentioned above, the NAL unit type can be identified by the RBSP data structure contained within the NAL unit, and information about such NAL unit types can be stored in the NAL unit header and signaled.

[0087] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether or not they contain information (slice data) about the image. VCL NAL unit types can be further classified by the nature and type of picture they contain, while Non-VCL NAL unit types can be further classified by the type of parameter set.

[0088] The following is an example of a NAL unit type identified by the type of parameter set included in the Non-VCL NAL unit type.

[0089] -APS (Adaptation Parameter Set) NAL unit: Type for NAL units that include APS

[0090] -DPS (Decoding Parameter Set) NAL unit: Type for NAL units including DPS

[0091] -VPS (Video Parameter Set) NAL unit: Type for NAL unit including VPS

[0092] -SPS (Sequence Parameter Set) NAL unit: Type for NAL units that include SPS

[0093] -PPS (Picture Parameter Set) NAL unit: Type for NAL units that include PPS

[0094] -PH (Picture header) NAL unit: Type for NAL units that include PH

[0095] The aforementioned NAL unit type has syntax information for the NAL unit type, and this syntax information can be stored in the NAL unit header and signaled. For example, the syntax information is nal_unit_type, and the NAL unit type can be identified by the nal_unit_type value.

[0096] On the other hand, as mentioned above, a single picture can contain multiple slices, and a single slice can contain a slice header and slice data. In this case, a picture header may be added to each of the multiple slices (slice headers and slice data sets) within a single picture. The picture header (picture header syntax) may contain information / parameters that can be commonly applied to the picture. In this document, slices may be mixed with or replaced by tile groups. Also in this document, slice headers may be mixed with or replaced by type group headers.

[0097] The slice header (slice header syntax, slice header information) may include information / parameters that can be commonly applied to the slice. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be commonly applied to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that can be commonly applied to one or more sequences. The VPS (VPS syntax) may include information / parameters that can be commonly applied to multiple layers. The DPS (DPS syntax) may include information / parameters that can be commonly applied to video in general. The DPS may include information / parameters related to the concatenation of CVS (coded video sequence). In this document, High-level syntax (HLS) may include at least one of the APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, picture header syntax, and slice header syntax.

[0098] In this document, the image / video information encoded from the encoding device to the decoding device and signaled in bitstream form may include not only partitioning-related information, intra / inter prediction information, residual information, and in-loop filtering information within the picture, but also information contained in the slice header, the picture header, the APS, the PPS, the SPS, the VPS, and / or the DPS. Furthermore, the image / video information may further include information from the NAL unit header.

[0099] On the other hand, to compensate for differences between the original image and the reconstructed image due to errors that occur during the compression encoding process, such as quantization, an in-loop filtering procedure can be performed on the reconstructed sample or reconstructed picture, as described above. As described above, in-loop filtering can be performed in the filter section of the encoding device and the filter section of the decoding device, and a deblocking filter, SAO, and / or adaptive loop filter (ALF) can be applied. For example, the ALF procedure can be performed after the deblocking filtering procedure and / or SAO procedure are completed. However, even in this case, the deblocking filtering procedure and / or SAO procedure may be omitted.

[0100] The following provides a detailed explanation of picture restoration and filtering. In image / video coding, a restored block can be generated based on intra-prediction / inter-prediction for each block, and a restored picture containing the restored block can be generated. If the current picture / slice is an I-picture / slice, the blocks contained in the current picture / slice can be restored based solely on intra-prediction. On the other hand, if the current picture / slice is a P or B-picture / slice, the blocks contained in the current picture / slice can be restored based on either intra-prediction or inter-prediction. In this case, intra-prediction may be applied to some blocks within the current picture / slice, while inter-prediction may be applied to the remaining blocks.

[0101] Intra prediction can represent a prediction that generates prediction samples for the current block based on reference samples within the picture to which the current block belongs (hereinafter referred to as the current picture). When intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block can be derived. The adjacent reference samples of the current block may include samples adjacent to the left boundary of the current block of size nW × nH and a total of 2 × nH samples adjacent to the bottom left, samples adjacent to the top boundary of the current block and a total of 2 × nW samples adjacent to the top right, and one sample adjacent to the top left of the current block. Alternatively, the adjacent reference samples of the current block may include multiple columns of upper adjacent samples and multiple rows of left adjacent samples. Furthermore, the adjacent reference samples of the current block may also include a total of nH samples adjacent to the right boundary of the current block, which is of size nW × nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right side of the current block.

[0102] However, some of the adjacent reference samples in the current block may not yet be decoded or available. In this case, the decoder can construct adjacent reference samples to be used for prediction by substituting the unavailable samples as available samples, or by constructing adjacent reference samples to be used for prediction through interpolation of available samples.

[0103] If neighboring reference samples are derived, (i) predicted samples can be derived based on the average or interpolation of neighboring reference samples in the current block, or (ii) predicted samples can be derived based on reference samples in the current block that are located in a specific (predicted) direction relative to the predicted sample. Case (i) is called the non-directional mode or non-angular mode, and case (ii) is called the directional mode or angular mode. Alternatively, the predicted sample can be generated by interpolation between the first neighboring sample and a second neighboring sample located in the opposite direction to the prediction direction of the current block's intra-prediction mode, based on the predicted sample of the current block. In this case, it can be called linear interpolation intra-prediction (LIP). Alternatively, chroma predicted samples can be generated based on chroma samples using a linear model. In this case, it can be called the LM mode. Alternatively, a temporary predicted sample for the current block can be derived based on filtered adjacent reference samples, and the predicted sample for the current block can be derived by performing a weighted sum of the temporary predicted sample and at least one reference sample derived by the intra-prediction mode from the existing adjacent reference samples, i.e., unfiltered adjacent reference samples. In the above case, it can be called PDPC (Position dependent intra prediction). In addition, intra-predictive coding can be performed by selecting the reference sample line with the highest prediction accuracy from the adjacent multiple reference sample lines of the current block, deriving the predicted sample using the reference sample located in the prediction direction on that line, and instructing (signaling) the decoding device with the reference sample line used at this time.In the aforementioned cases, this can be called multi-reference line (MRL) intra prediction or MRL-based intra prediction. Furthermore, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode, with adjacent reference samples derived and available for use on a subpartition basis. That is, in this case, the intra prediction mode for the current block is also applied to the subpartition, and by deriving and using adjacent reference samples on a subpartition basis, intra prediction performance can be improved in some cases. Such prediction methods can be called intra subpartitions (ISP) or ISP-based intra prediction. The aforementioned intra prediction methods can be distinguished from the intra prediction modes in Table of Contents 1 and 2 and referred to as intra prediction types. These intra prediction types can be referred to by various terms, such as intra prediction techniques or additional intra prediction modes. For example, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, and ISP. A general intra-prediction method that excludes specific intra-prediction types such as LIP, PDPC, MRL, and ISP can be called a normal intra-prediction type. The normal intra-prediction type can be generally applied when the aforementioned specific intra-prediction types are not applicable, and predictions can be performed based on the intra-prediction modes described above. Meanwhile, post-processing filtering can be performed on the derived prediction samples as needed.

[0104] Specifically, the intra-prediction procedure may include an intra-prediction mode / type determination step, an adjacent reference sample derivation step, and an intra-prediction mode / type-based predictive sample derivation step. Additionally, a post-filtering step may be performed on the derived predictive samples as needed.

[0105] A modified restored picture is generated by the in-loop filtering procedure, and the decoder outputs the modified restored picture as a decoded picture. This modified restored picture is also stored in the decoded picture buffer or memory of the encoding / decoding device and can be used as a reference picture in the interpretation procedure during subsequent encoding / decoding of the picture. The in-loop filtering procedure includes, as described above, a deblocking filtering procedure, an SAO (sample adaptive offset) procedure, and / or an ALF (adaptive loop filter) procedure. In this case, one or part of the deblocking filtering procedure, the SAO (sample adaptive offset) procedure, the ALF (adaptive loop filter) procedure, and the bi-lateral filter procedure may be applied sequentially, or all of them may be applied sequentially. For example, the SAO procedure may be performed after the deblocking filtering procedure is applied to the restored picture. Or, for example, the ALF procedure may be performed after the deblocking filtering procedure is applied to the restored picture. This is also done in the encoding device.

[0106] Deblocking filtering is a filtering technique that removes distortion occurring at the boundaries between blocks in a restored picture. The deblocking filtering procedure can, for example, involve deriving a target boundary in the restored picture, determining a boundary strength (bS) for the target boundary, and performing deblocking filtering on the target boundary based on the bS. The bS can be determined based on the prediction modes of two adjacent blocks, the difference in motion vectors, whether the reference picture is identical, and whether a non-zero effectiveness coefficient exists.

[0107] SAO is a method for compensating for the offset difference between a restored picture and the original picture on a sample-by-sample basis, and can be applied based on types such as Band Offset and Edge Offset. According to SAO, each SAO type can classify samples into different categories, and an offset value can be added to each sample based on the category. Filtering information for SAO can include information on whether SAO is applicable, SAO type information, SAO offset value information, etc. SAO can also be applied to the restored picture after the deblocking filtering has been applied.

[0108] ALF (Adaptive Loop Filter) is a technique that filters a restored picture on a sample-by-sample basis based on filter coefficients determined by the filter shape. The encoding device can determine whether ALF is applicable, the ALF shape, and / or ALF filtering coefficients by comparing the restored picture with the original picture, and can signal this to the decoding device. That is, filtering information for ALF can include information on whether ALF is applicable, ALF filter shape information, ALF filtering coefficient information, etc. ALF can also be applied to the restored picture after the deblocking filtering has been applied.

[0109] Figure 5 shows an example of an ALF filter shape.

[0110] Figure 5(a) shows a 7x7 diamond filter shape, and (b) shows a 5x5 diamond filter shape. In Figure 5, Cn in the filter shape represents the filter coefficient. When n is the same in Cn, this indicates that the same filter coefficient can be assigned. In this document, the position and / or unit to which the filter coefficient is assigned according to the ALF filter shape may be called a filter tab. In this case, one filter coefficient is assigned to each filter tab, and the arrangement of the filter tabs may correspond to a filter shape. A filter tab located in the center of the filter shape may be called a center filter tab. Two filter tabs with the same n value located at corresponding positions relative to the center filter tab may be assigned the same filter coefficient. For example, in the case of a 7x7 diamond filter shape, there are 25 filter tabs, and the filter coefficients C0 to C11 are assigned in a centrally symmetrical manner, so only 13 filter coefficients are needed to assign the filter coefficients to the 25 filter tabs. Furthermore, for example, in the case of a 5x5 diamond filter shape, since it includes 13 filter tabs and the filter coefficients C0 to C5 are assigned in a centrally symmetrical manner, the filter coefficients can be assigned to the 13 filter tabs using only 7 filter coefficients. For example, to reduce the amount of data regarding the signaled filter coefficients, 12 of the 13 filter coefficients for a 7x7 diamond filter shape can be (explicitly) signaled, and one filter coefficient can be (implicitly) derived. Also, for example, 6 of the 7 filter coefficients for a 5x5 diamond filter shape can be (explicitly) signaled, and one filter coefficient can be (implicitly) derived.

[0111] Figure 6 shows an example of an ALF procedure using a virtual boundary according to one embodiment of this document.

[0112] The virtual boundary may be a line defined by shifting the horizontal CTU boundary by only N samples. In one example, N may be 4 for the luma component and / or N may be 2 for the chroma component.

[0113] The modified block classification can be applied to the Luma component. Only samples on the virtual boundary can be used to calculate the 1D Laplacian gradient of a 4x4 block on the virtual boundary. Similarly, only samples below the virtual boundary can be used to calculate the 1D Laplacian gradient of a 4x4 block below the virtual boundary. The quantization of the activity value A can be scaled by considering the reduced number of samples used in the calculation of the 1D Laplacian gradient.

[0114] For the filtering procedure, symmetric padding operations at the virtual boundary can be used for the luma and chroma components. Referring to Figure 6, if a filtered sample is located below the virtual boundary, adjacent samples located on the virtual boundary may be padded. Conversely, the other such sample may also be padded symmetrically.

[0115] The procedure illustrated in Figure 6 can also be used for slice, brick, and / or tile boundaries when filtering across the boundary is not possible. For ALF block classification, only samples contained in the same slice, brick, and / or tile can be used, and the activity value can be scaled accordingly. For ALF filtering, symmetrical padding can be applied to the horizontal and / or vertical directions, respectively, for horizontal and / or vertical boundaries.

[0116] Figure 7 is a flowchart illustrating a filtering-based encoding method in an encoding device. The method in Figure 7 may include steps S700 to S730.

[0117] In step S700, the encoding device can generate a restored picture. Step S700 can be performed based on the procedure for generating a restored picture (or restored sample) described above.

[0118] In step S710, the encoding device can determine whether in-loop filtering is applied (across the virtual boundary) based on in-loop filtering-related information. Here, in-loop filtering may include at least one of the deblocking filtering, SAO, or ALF described above.

[0119] In step S720, the encoding device can generate a modified restored picture (modified restored sample) based on the decision made in step S710. Here, the modified restored picture (modified restored sample) may be a filtered restored picture (filtered restored sample).

[0120] In step S730, the encoding device can encode image / video information including in-loop filtering-related information based on the in-loop filtering procedure.

[0121] Figure 8 is a flowchart illustrating a filtering-based decoding method in a decoding device. The method in Figure 8 may include steps S800 to S830.

[0122] In step S800, the decoding device can obtain image / video information, including in-loop filtering-related information, from the bitstream. Here, the bitstream can be based on encoded image / video information transmitted from the encoding device.

[0123] In step S810, the decoding device can generate a restored picture. Step S810 can be performed based on the procedure for generating a restored picture (or restored sample) described above.

[0124] In step S820, the decoding device can determine whether in-loop filtering is applied (across the virtual boundary) based on in-loop filtering-related information. Here, in-loop filtering may include at least one of the deblocking filtering, SAO, or ALF described above.

[0125] In step S830, the decoding device can generate a modified restored picture (modified restored sample) based on the decision made in step S820. Here, the modified restored picture (modified restored sample) may be a filtered restored picture (filtered restored sample).

[0126] As mentioned above, an in-loop filtering procedure can be applied to the restored picture. In this case, a virtual boundary can be defined to further enhance the subjective / objective visual quality of the restored picture, and the in-loop filtering procedure can be applied across the virtual boundary. The virtual boundary may include discontinuous edges such as 360-degree images, VR images, or PIPs (picture in picture). For example, the virtual boundary may exist at a predetermined, agreed-upon location, and its presence and / or location may be signaled. As an example, the virtual boundary may be located at the fourth sample line from the top in the CTU row (specifically, for example, above the fourth sample line from the top in the CTU row). As another example, information regarding the presence and / or location of the virtual boundary may be signaled via an HLS. The HLS may include SPS, PPS, picture headers, slice headers, etc., as mentioned above.

[0127] The following describes the signaling and semantics of the higher-level syntax for the embodiments of this document.

[0128] One embodiment of this document may include a method for controlling a loop filter. This method for controlling a loop filter may be applied to a restored picture. The in-loop filter (loop filter) can be used for decoding an encoded bitstream. The loop filter may include the deblocking, SAO, and ALF described above. The SPS may include flags associated with each of the deblocking, SAO, and ALF. The flags may indicate whether each tool is available for coding a CLVS (coded layer video sequence) or CVS (coded video sequence) that references the SPS.

[0129] If the loop filter is available for CVS, its application can be controlled to avoid crossing specific boundaries. For example, whether the loop filter crosses subpicture boundaries can be controlled. Also, whether the loop filter crosses tile boundaries can be controlled. In addition, whether the loop filter crosses virtual boundaries can be controlled, where virtual boundaries may be defined on the CTU based on line buffer availability.

[0130] In relation to whether an in-loop filtering procedure is performed across a virtual boundary, the in-loop filtering-related information may include at least one of the following: the SPS virtual boundary availability flag (virtual boundary availability flag within the SPS), the SPS virtual boundary existence flag, the picture header virtual boundary existence flag, the SPS picture header virtual boundary existence flag, and information regarding the location of the virtual boundary.

[0131] In the embodiments described herein, information regarding the position of a virtual boundary may include information regarding the x-coordinate of a vertical virtual boundary and / or information regarding the y-coordinate of a horizontal virtual boundary. Specifically, information regarding the position of a virtual boundary may include information regarding the x-coordinate of a vertical virtual boundary and / or information regarding the y-coordinate of a horizontal virtual boundary in luma sample units. Furthermore, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the x-coordinate of a vertical virtual boundary present in the SPS. Furthermore, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the y-coordinate of a horizontal virtual boundary present in the SPS. Alternatively, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the x-coordinate of a vertical virtual boundary present in the picture header. Furthermore, information regarding the position of a virtual boundary may include information regarding the number of information (syntax elements) regarding the y-coordinate of a horizontal virtual boundary present in the picture header.

[0132] The following table shows exemplary syntax and semantics of SPS according to this embodiment.

[0133] [Table 1]

[0134] [Table 2]

[0135] The following table shows exemplary syntax and semantics of the PPS (picture parameter set) according to this embodiment.

[0136] [Table 3]

[0137] [Table 4]

[0138] The following table shows the exemplary syntax and semantics of the picture header according to this embodiment.

[0139] [Table 5-1]

[0140] [Table 5-2]

[0141] [Table 6-1]

[0142] [Table 6-2]

[0143] The following table shows exemplary syntax and semantics of a slice header according to this embodiment.

[0144] [Table 7]

[0145] [Table 8]

[0146] The following section describes the signaling of information for virtual boundaries that can be used in in-loop filtering.

[0147] In existing designs, to disable loop filters that cross virtual boundaries, i) the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is set to 0, and the PH virtual boundary presence flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag) exists for all picture headers and is set to 0, or ii) the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is set to 1, and information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_vertical_boudnaries) and information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_vertical_boudnaries) may both be set to 0.

[0148] In existing designs, the virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is set to 1 as described in ii) above, which can cause problems in the decoding procedure because the decoder anticipates signaling for the location of the virtual boundary.

[0149] The embodiments described in the following paragraphs may offer solutions to the aforementioned problems. The embodiments may be applied independently, or at least two or more embodiments may be applied in combination.

[0150] In one embodiment of this document, whether or not syntax elements for indicating virtual boundaries are included in the SPS can be controlled by flags. For example, there may be two such flags (e.g., an SPS virtual boundaries enabled flag and an SPS virtual boundaries present flag).

[0151] In one example according to this embodiment, the virtual boundary enable flag of the SPS may be referred to as sps_loop_filter_across_virtual_boundaries_disabled_flag (or sps_virtual_boundaries_enabled_flag). The virtual boundary enable flag of the SPS can indicate whether a feature for disabling the loop filter across a virtual boundary is enabled.

[0152] In one example according to this embodiment, the virtual boundary presence flag of the SPS may be referred to as sps_loop_filter_across_virtual_boundaries_disabled_present_flag (or sps_virtual_boundaries_present_flag). The virtual boundary presence flag of the SPS can indicate whether signaling information for virtual boundaries is included in the SPS or picture header (PH).

[0153] In one example according to this embodiment, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 0, then signaling information to disable loop filters crossing virtual boundaries may be included in the picture header.

[0154] In one example according to this embodiment, if information regarding the location of virtual boundaries (e.g., vertical virtual boundaries, horizontal virtual boundaries) is included in the SPS, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be restricted to be greater than 0.

[0155] In one example according to this embodiment, a variable can be derived that indicates whether the picture is currently filtered or disabled at virtual boundaries. For example, the variable may include VirtualBoundairesDisabledFlag.

[0156] In one example in this illustration, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 1, then VirtualBoundairesDisabledFlag may be 1.

[0157] In another example, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1, the virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 0, and the sum of the information regarding the number of vertical virtual boundaries (e.g., ph_num_ver_virtual_boundaries) and the information regarding the number of horizontal virtual boundaries (e.g., ph_num_hor_virtual_boundaries) is greater than 0, then VirtualBoundairesDisabledFlag may be 1.

[0158] In other cases in this example, VirtualBoundairesDisabledFlag may be 0.

[0159] The following table shows an exemplary syntax of SPS according to this embodiment.

[0160] [Table 9]

[0161] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0162] [Table 10]

[0163] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0164] [Table 11]

[0165] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0166] [Table 12]

[0167] In embodiments relating to Tables 9 to 12, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device to the decoding device may include a sequence parameter set (SPS) and a picture header (PH). The SPS may include a virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). The SPS may also include a virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) based on the virtual boundary availability flag.

[0168] For example, if the value of the virtual boundary availability flag is 1, the SPS may include a virtual boundary existence flag for the SPS. Based on the virtual boundary availability flag and the virtual boundary existence flag for the SPS, the SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]). For example, if the value of the virtual boundary availability flag is 1 and the value of the virtual boundary existence flag for the SPS is 1, the SPS may include information regarding the number of vertical virtual boundaries of the SPS, information regarding the position of the vertical virtual boundaries of the SPS, information regarding the number of horizontal virtual boundaries of the SPS, and information regarding the position of the horizontal virtual boundaries of the SPS.

[0169] In one example, the number of pieces of information regarding the position of the vertical virtual boundary of the SPS can be determined based on the number of pieces of information regarding the vertical virtual boundary of the SPS, and the number of pieces of information regarding the position of the horizontal virtual boundary of the SPS can be determined based on the number of pieces of information regarding the horizontal virtual boundary of the SPS. The picture header may include information regarding the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundary of the PH (ph_virtual_boundaries_pos_y[i]), based on the virtual boundary availability flag and the virtual boundary existence flag of the SPS.

[0170] For example, if the value of the virtual boundary availability flag is 1 and the value of the virtual boundary presence flag of the SPS is 0, the picture header may include information regarding the number of vertical virtual boundaries of the PH, information regarding the positions of the vertical virtual boundaries of the PH, information regarding the number of horizontal virtual boundaries of the PH, and information regarding the positions of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information regarding the positions of the vertical virtual boundaries of the PH can be determined based on the information regarding the number of vertical virtual boundaries of the PH, and the number of pieces of information regarding the positions of the horizontal virtual boundaries of the PH can be determined based on the information regarding the number of horizontal virtual boundaries of the PH.

[0171] In another embodiment of this document, each of the picture headers (picture headers) referencing an SPS may include a virtual boundary presence flag for the PH, ph_loop_filter_across_virtual_boundaries_disabled_present_flag (or ph_virtual_boundaries_present_flag). This embodiment can also be described together with the virtual boundary availability flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_flag) and the virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag), as in the previous embodiment.

[0172] In one example according to this embodiment, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 0, then each picture header information (picture header) that references the SPS may include the virtual boundary presence flag ph_loop_filter_across_virtual_boundaries_disalbed_present_flag (or ph_virtual_boundaries_present_flag) of the PH.

[0173] In one example according to this embodiment, if information regarding the location of virtual boundaries (e.g., vertical virtual boundaries, horizontal virtual boundaries) is included in the SPS, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be restricted to be greater than 0.

[0174] In one example according to this embodiment, a variable can be derived that indicates whether the filter is currently disabled at the virtual boundary for the picture. For example, the variable may include VirtualBoundairesDisabledFlag.

[0175] In one example in this illustration, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 1, then VirtualBoundairesDisabledFlag may be 1.

[0176] In another example, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and the virtual boundary presence flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the PH is 1, then VirtualBoundairesDisabledFlag may be 1.

[0177] In other cases in this example, VirtualBoundairesDisabledFlag may be 0.

[0178] The following table shows an exemplary syntax of SPS according to this embodiment.

[0179] [Table 13]

[0180] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0181] [Table 14]

[0182] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0183] [Table 15]

[0184] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0185] [Table 16-1]

[0186] [Table 16-2]

[0187] In embodiments relating to Tables 13 to 16, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device to the decoding device may include a sequence parameter set (SPS) and a picture header (PH). The SPS may include a virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary availability flag, the SPS may include a virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, if the value of the virtual boundary availability flag is 1, the SPS may include a virtual boundary presence flag. The SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]), based on the virtual boundary availability flag and the virtual boundary existence flag of the SPS.

[0188] For example, if the value of the virtual boundary availability flag is 1 and the value of the virtual boundary existence flag for the SPS is 1, the SPS may include information regarding the number of vertical virtual boundaries of the SPS, information regarding the positions of the vertical virtual boundaries of the SPS, information regarding the number of horizontal virtual boundaries of the SPS, and information regarding the positions of the horizontal virtual boundaries of the SPS. In one example, the number of pieces of information regarding the positions of the vertical virtual boundaries of the SPS can be determined based on the information regarding the number of vertical virtual boundaries of the SPS, and the number of pieces of information regarding the positions of the horizontal virtual boundaries of the SPS can be determined based on the information regarding the number of horizontal virtual boundaries of the SPS. The picture header may include a virtual boundary existence flag for the PH based on the virtual boundary availability flag and the virtual boundary existence flag for the SPS.

[0189] For example, if the value of the virtual boundary availability flag is 1 and the value of the virtual boundary existence flag for the SPS is 0, the picture header may include the virtual boundary existence flag for the PH. Based on the virtual boundary existence flag for the PH, the picture header may include information regarding the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the PH (ph_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the PH (ph_virtual_boundaries_pos_y[i]).

[0190] For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information regarding the number of vertical virtual boundaries of the PH, information regarding the positions of the vertical virtual boundaries of the PH, information regarding the number of horizontal virtual boundaries of the PH, and information regarding the positions of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information regarding the positions of the vertical virtual boundaries of the PH can be determined based on the information regarding the number of vertical virtual boundaries of the PH, and the number of pieces of information regarding the positions of the horizontal virtual boundaries of the PH can be determined based on the information regarding the number of horizontal virtual boundaries of the PH.

[0191] In another embodiment of this document, whether a syntax element for indicating a virtual boundary is included in the SPS can be controlled by a flag. For example, there may be two such flags (e.g., an SPS virtual boundaries present flag and an SPS PH virtual boundaries present flag).

[0192] In one example according to this embodiment, the virtual boundary presence flag of the SPS may be called sps_loop_filter_across_virtual_boundaries_disabled_present_flag (or sps_virtual_boundaries_present_flag). The virtual boundary presence flag of the SPS can indicate whether or not virtual boundary information is included in the SPS.

[0193] In one example according to this embodiment, the virtual boundary presence flag of the SPS PH may be called sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag. The virtual boundary presence flag of the SPS PH can indicate whether or not virtual boundary information is included in the picture header (PH).

[0194] In one example according to this embodiment, if the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 1, then the virtual boundary existence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS PH may not exist and may be restricted to being inferred to 0.

[0195] In one example according to this embodiment, if the virtual boundary presence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS PH is 1, signaling information to disable loop filters crossing virtual boundaries may be included in the picture header.

[0196] The following table shows an exemplary syntax of SPS according to this embodiment.

[0197] [Table 17]

[0198] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0199] [Table 18]

[0200] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0201] [Table 19]

[0202] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0203] [Table 20-1]

[0204] [Table 20-2]

[0205] In embodiments relating to Tables 17 to 20, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device to the decoding device may include a sequence parameter set (SPS) and a picture header (PH). The SPS may include a virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). Based on the virtual boundary presence flag for the SPS, the SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]).

[0206] For example, if the value of the virtual boundary existence flag of the SPS is 1, the SPS may include information regarding the number of vertical virtual boundaries of the SPS, information regarding the positions of the vertical virtual boundaries of the SPS, information regarding the number of horizontal virtual boundaries of the SPS, and information regarding the positions of the horizontal virtual boundaries of the SPS. In one example, the number of pieces of information regarding the positions of the vertical virtual boundaries of the SPS can be determined based on the information regarding the number of vertical virtual boundaries of the SPS, and the number of pieces of information regarding the positions of the horizontal virtual boundaries of the SPS can be determined based on the information regarding the number of horizontal virtual boundaries of the SPS. The SPS may include a virtual boundary existence flag for SPS PH based on the virtual boundary existence flag of the SPS.

[0207] For example, if the value of the virtual boundary existence flag of the SPS is 0, the SPS may include the virtual boundary existence flag of the SPS PH. The picture header may include the virtual boundary existence flag of the PH based on the virtual boundary existence flag of the SPS PH. For example, if the value of the virtual boundary existence flag of the SPS PH is 1, the picture header may include the virtual boundary existence flag of the PH. The picture header may include information about the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries), information about the position of the vertical virtual boundaries of the PH (ph_virtual_boundaries_pos_x[i]), information about the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries), and information about the position of the horizontal virtual boundaries of the PH (ph_virtual_boundaries_pos_y[i]) based on the virtual boundary existence flag of the PH.

[0208] For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information regarding the number of vertical virtual boundaries of the PH, information regarding the positions of the vertical virtual boundaries of the PH, information regarding the number of horizontal virtual boundaries of the PH, and information regarding the positions of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information regarding the positions of the vertical virtual boundaries of the PH can be determined based on the information regarding the number of vertical virtual boundaries of the PH, and the number of pieces of information regarding the positions of the horizontal virtual boundaries of the PH can be determined based on the information regarding the number of horizontal virtual boundaries of the PH.

[0209] In another embodiment of this document, if gradual decoding refresh (GDR) is available (i.e., if the value of gdr_enabled_flag is 1), the feature that the loop filter is disabled at a virtual boundary is enabled, and information about the virtual boundary may be signaled in the picture header (may be included in the picture header).

[0210] In another embodiment of this document, when the ability to disable loop filters across virtual boundaries is enabled, information regarding the signaling of the virtual boundary's location may be included in one or more parameter sets. For example, when the ability to disable loop filters across virtual boundaries is enabled, information regarding the signaling of the virtual boundary's location may be included in the SPS and picture header.

[0211] In this embodiment, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1 and signaling information regarding the location of the virtual boundary is included in one or more parameter sets, then the following applies:

[0212] a) Signaling information regarding the location of the virtual boundary may be included only in the SPS, only in the picture header, or in both the SPS and the picture header.

[0213] b) The derivation of VirtualBoundariesDisabledFlag for each picture is as follows:

[0214] - If sps_loop_filter_across_virtual_boundaries_disabled_flag is 0, then VirtualBoundariesDisabledFlag may be set to 0.

[0215] - In another case in this example, if information regarding the location of virtual boundaries is not signaled in all picture headers associated with the SPS or picture, VirtualBoundariesDisabledFlag may be set to 0.

[0216] - In other cases in this example (where the virtual boundary location is signaled only in the SPS, only in the picture header, or in both the SPS and the picture header), VirtualBoundariesDisabledFlag may be set to 1.

[0217] c) A virtual boundary applied to a picture may include a union of virtual boundaries signaled by parameter sets that the picture directly or indirectly references. For example, the virtual boundary may include (if any) a virtual boundary signaled by an SPS. For example, the virtual boundary may include (if any) a virtual boundary signaled by a picture header associated with the picture.

[0218] d) Restrictions may be applied to ensure that the maximum number of virtual boundaries per picture does not exceed a predefined value. For example, the predefined value may be 8.

[0219] e) Information regarding the location of virtual boundaries signaled in the picture header may be restricted (if any) so as not to be the same as information regarding the location of virtual boundaries included in other parameter sets (e.g., SPS or PPS).

[0220] - Alternatively, for a given virtual boundary currently applied to a picture, the location of the virtual boundary (e.g., the location of the same virtual boundary signaled to the picture header associated with the SPS and the picture) may be included in two different sets of parameters.

[0221] f) If the virtual boundary existence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS is 1, then the virtual boundary existence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) of the SPS PH does not exist and may be restricted to being inferred to 0.

[0222] The following table shows an exemplary syntax of SPS according to this embodiment.

[0223] [Table 21]

[0224] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0225] [Table 22]

[0226] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0227] [Table 23]

[0228] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0229] [Table 24-1]

[0230] [Table 24-2]

[0231] In embodiments relating to Tables 21 to 24, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device to the decoding device may include a sequence parameter set (SPS) and a picture header (PH). The SPS may include a virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary availability flag, the SPS may include a virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, if the value of the virtual boundary availability flag is 1, the SPS may include a virtual boundary presence flag. The SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]), based on the virtual boundary availability flag and the virtual boundary existence flag of the SPS.

[0232] For example, if the value of the virtual boundary availability flag is 1 and the value of the virtual boundary presence flag for the SPS is 1, the SPS may include information regarding the number of vertical virtual boundaries of the SPS, information regarding the positions of the vertical virtual boundaries of the SPS, information regarding the number of horizontal virtual boundaries of the SPS, and information regarding the positions of the horizontal virtual boundaries of the SPS. In one example, the number of pieces of information regarding the positions of the vertical virtual boundaries of the SPS can be determined based on the information regarding the number of vertical virtual boundaries of the SPS, and the number of pieces of information regarding the positions of the horizontal virtual boundaries of the SPS can be determined based on the information regarding the number of horizontal virtual boundaries of the SPS. The picture header may include the virtual boundary presence flag for the PH based on the virtual boundary availability flag.

[0233] For example, if the value of the virtual boundary availability flag is 1, the picture header may include the virtual boundary existence flag for the PH. Based on the virtual boundary existence flag for the PH, the picture header may include information regarding the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the PH (ph_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the PH (ph_virtual_boundaries_pos_y[i]). For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information regarding the number of vertical virtual boundaries of the PH, information regarding the position of the vertical virtual boundaries of the PH, information regarding the number of horizontal virtual boundaries of the PH, and information regarding the position of the horizontal virtual boundaries of the PH. In one example, the number of pieces of information relating to the position of the vertical virtual boundary of the PH can be determined based on the number of pieces of information relating to the vertical virtual boundary of the PH, and the number of pieces of information relating to the position of the horizontal virtual boundary of the PH can be determined based on the number of pieces of information relating to the horizontal virtual boundary of the PH.

[0234] In another embodiment of this document, loop filtering can be performed as in the embodiment described above, but without the restriction that the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries must be greater than 0.

[0235] In another embodiment of this document, virtual boundary information can be signaled in both SPS and PH. In one example of this embodiment, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1, then information regarding the number of vertical virtual boundaries, the number of horizontal virtual boundaries, and / or information regarding the locations of virtual boundaries may be included in the SPS. In addition, if the virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) of the SPS is 1, then information regarding the number of vertical virtual boundaries, the number of horizontal virtual boundaries, and / or information regarding the delta values ​​of the virtual boundary locations may be included in the picture header. The delta values ​​of the virtual boundary locations may represent the difference between the virtual boundary locations. The picture header may also include information regarding the sign of the virtual boundary locations.

[0236] According to one example of this embodiment, to derive the virtual boundary position for each picture, if the delta value of the virtual boundary position is not present in the picture header, the information about the virtual boundary position signaled by SPS may be used for loop filtering. If the delta value of the virtual boundary position is present in the picture header, the virtual boundary position may be derived based on the information about the virtual boundary position signaled by SPS and the sum of the associated delta values.

[0237] The following table shows an exemplary syntax of SPS according to this embodiment.

[0238] [Table 25]

[0239] The following table shows exemplary semantics for the syntax elements included in the syntax.

[0240] [Table 26]

[0241] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0242] [Table 27]

[0243] The following table shows exemplary semantics for the syntax elements included in the syntax.

[0244] [Table 28-1]

[0245] [Table 28-2]

[0246] In embodiments relating to Tables 25 to 28, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device to the decoding device may include a sequence parameter set (SPS) and a picture header (PH). The SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary enable flag, the SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]). For example, if the value of the virtual boundary availability flag is 1, the SPS may include information regarding the number of horizontal virtual boundaries of the SPS, information regarding the positions of the horizontal virtual boundaries of the SPS, information regarding the number of vertical virtual boundaries of the SPS, and information regarding the positions of the vertical virtual boundaries of the SPS.

[0247] In one example, the number of pieces of information regarding the position of the horizontal virtual boundary of the SPS can be determined based on the number of horizontal virtual boundaries of the SPS, and the number of pieces of information regarding the position of the vertical virtual boundary of the SPS can be determined based on the number of vertical virtual boundaries of the SPS. The picture header may include a virtual boundary existence flag for the PH based on the virtual boundary availability flag. For example, if the value of the virtual boundary availability flag is 1, the picture header may include the virtual boundary existence flag for the PH. The picture header may include information regarding the delta value of the position of the horizontal virtual boundary of the PH (ph_virtual_boundaries_pos_x_delta[i]), information regarding the sign of the position of the horizontal virtual boundary of the PH (ph_virtual_boundaries_pos_x_sign[i]), information regarding the delta value of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_y_delta[i]), and information regarding the sign of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_y_sign[i]), based on the virtual boundary existence flag for the PH.

[0248] For example, if the value of the virtual boundary existence flag for the PH is 1, the picture header may include information regarding the delta value of the position of the vertical virtual boundary of the PH, information regarding the sign of the position of the vertical virtual boundary of the PH, information regarding the delta value of the position of the horizontal virtual boundary of the PH, and information regarding the sign of the position of the horizontal virtual boundary of the PH. In one example, the number of pieces of information regarding the delta value of the position of the vertical virtual boundary of the PH and the number of pieces of information regarding the sign of the position of the vertical virtual boundary of the PH can be determined based on information regarding the number of vertical virtual boundaries of the SPS, and the number of pieces of information regarding the delta value of the position of the horizontal virtual boundary of the PH and the number of pieces of information regarding the sign of the position of the horizontal virtual boundary of the PH can be determined based on information regarding the number of horizontal virtual boundaries of the SPS.

[0249] Another embodiment of this document describes the signaling of information regarding the location of virtual boundaries for each picture. In one example, if information regarding the location of virtual boundaries is included in the SPS and information regarding the delta value of the virtual boundary location is not included in the picture header, the information regarding the location of virtual boundaries included in the SPS may be used for loop filtering. If information regarding the location of virtual boundaries is not included in the SPS but information regarding the delta value of the virtual boundary location is included in the picture header, the information regarding the location of virtual boundaries included in the picture header may be used for loop filtering. If information regarding the location of virtual boundaries is included in the SPS and information regarding the delta value of the virtual boundary location is included in the picture header, the location of the virtual boundary may be derived based on the sum of the information regarding the location of virtual boundaries signaled in the SPS and its associated delta value. If information regarding the location of virtual boundaries is not included in the SPS and information regarding the delta value of the virtual boundary location is not included in the picture header, a virtual boundary may not be applied to the picture.

[0250] The following table shows an exemplary syntax of SPS according to this embodiment.

[0251] [Table 29]

[0252] The following table shows exemplary semantics for the syntax elements included in the aforementioned syntax.

[0253] [Table 30]

[0254] The following table shows an exemplary syntax of the header information (picture header) according to this embodiment.

[0255] [Table 31]

[0256] The following table shows exemplary semantics regarding the syntax elements included in the syntax.

[0257] [Table 32-1]

[0258] [Table 32-2]

[0259] In embodiments related to Tables 29 to 32, the image information encoded by the encoding device and / or the image information obtained via the bitstream received from the encoding device by the decoding device may include a sequence parameter set (SPS) and a picture header (PH).

[0260] The SPS may include a virtual boundary availability flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary availability flag, the SPS may include a virtual boundary presence flag for the SPS (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, if the value of the virtual boundary availability flag is 1, the SPS may include a virtual boundary presence flag for the SPS. Based on the virtual boundary availability flag and the virtual boundary presence flag for the SPS, the SPS may include information regarding the number of vertical virtual boundaries of the SPS (sps_num_ver_virtual_boundaries), information regarding the position of the vertical virtual boundaries of the SPS (sps_virtual_boundaries_pos_x[i]), information regarding the number of horizontal virtual boundaries of the SPS (sps_num_hor_virtual_boundaries), and information regarding the position of the horizontal virtual boundaries of the SPS (sps_virtual_boundaries_pos_y[i]).

[0261] For example, if the value of the virtual boundary availability flag is 1 and the value of the virtual boundary existence flag of the SPS is 1, the SPS may include information regarding the number of horizontal virtual boundaries, information regarding the positions of the horizontal virtual boundaries, information regarding the number of vertical virtual boundaries, and information regarding the positions of the vertical virtual boundaries. In one example, the number of pieces of information regarding the positions of the horizontal virtual boundaries can be determined based on the information regarding the number of horizontal virtual boundaries, and the number of pieces of information regarding the positions of the vertical virtual boundaries can be determined based on the information regarding the number of vertical virtual boundaries. The picture header may include the virtual boundary existence flag of the PH based on the virtual boundary availability flag.

[0262] For example, if the value of the virtual boundary availability flag is 1, the picture header may include the virtual boundary existence flag for the PH. Based on the virtual boundary existence flag for the PH and the information regarding the number of vertical virtual boundaries of the SPS, the picture header may include information regarding the number of vertical virtual boundaries of the PH (ph_num_ver_virtual_boundaries). For example, if the value of the virtual boundary existence flag for the PH is 1 and the value of the information regarding the number of vertical virtual boundaries of the SPS is 0, the picture header may include information regarding the number of vertical virtual boundaries of the PH. In one example, the picture header may include information regarding the delta value of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_x_delta[i]) and information regarding the sign of the position of the vertical virtual boundary of the PH (ph_virtual_boundaries_pos_x_sign[i]) based on the information regarding the number of vertical virtual boundaries of the PH. In one example, based on information regarding the number of vertical virtual boundaries of the PH, the number of information regarding the delta values ​​of the positions of the vertical virtual boundaries of the PH and the number of information regarding the signs of the positions of the vertical virtual boundaries of the PH can be determined. The picture header may include information regarding the number of horizontal virtual boundaries of the PH (ph_num_hor_virtual_boundaries) based on the virtual boundary existence flag of the PH and information regarding the number of horizontal virtual boundaries of the SPS.

[0263] For example, if the value of the virtual boundary existence flag for the PH is 1 and the value of the information regarding the number of horizontal virtual boundaries for the SPS is 0, the picture header may include information regarding the number of horizontal virtual boundaries for the PH. In one example, the picture header may include information regarding the delta value of the position of the horizontal virtual boundaries of the PH (ph_virtual_boundaries_pos_y_delta[i]) and information regarding the sign of the position of the horizontal virtual boundaries of the PH (ph_virtual_boundaries_pos_y_sign[i]) based on the information regarding the number of horizontal virtual boundaries of the PH. In one example, the number of pieces of information regarding the delta value of the position of the horizontal virtual boundaries of the PH and the number of pieces of information regarding the sign of the position of the horizontal virtual boundaries of the PH can be determined based on the information regarding the number of horizontal virtual boundaries of the PH.

[0264] In conjunction with the aforementioned table, according to the embodiments of this document, the coding device can efficiently signal the information necessary to control in-loop filtering performed across virtual boundaries. In one example, information related to whether in-loop filtering is available across virtual boundaries can be signaled.

[0265] Figures 9 and 10 schematically illustrate an example of a video / image encoding method and related components according to the embodiments described herein.

[0266] The method disclosed in Figure 9 can be performed by the encoding device disclosed in Figure 2 or Figure 10. Specifically, for example, steps S900 to S920 in Figure 9 can be performed by the residual processing unit 230 of the encoding device in Figure 10, steps S930 and S940 in Figure 9 can be performed by the prediction unit 220 of the encoding device in Figure 10, step S950 in Figure 9 can be performed by the filtering unit 260 of the encoding device in Figure 10, and step S960 in Figure 9 can be performed by the entropy encoding unit 240 of the encoding device in Figure 10. The method disclosed in Figure 9 may include the embodiments described above.

[0267] Referring to Figure 9, the encoding device can derive a residual sample (S900). The encoding device can derive a residual sample for the current block, which can be derived based on the original sample and predicted sample of the current block. Specifically, the encoding device can derive a predicted sample of the current block based on a prediction mode. In this case, various prediction methods disclosed in this document, such as interpretation or intrapretation, can be applied. A residual sample can be derived based on the predicted sample and the original sample.

[0268] The encoding device can derive conversion coefficients (S910). The encoding device can derive conversion coefficients based on the conversion procedure for the residual sample. For example, the conversion procedure may include at least one of DCT, DST, GBT, or CNT.

[0269] The encoding device can derive quantized transformation coefficients. The encoding device can derive quantized transformation coefficients based on a quantization procedure for the transformation coefficients. The quantized transformation coefficients may take the form of a one-dimensional vector based on the coefficient scan order.

[0270] The encoding device can generate residual information (S920). The encoding device can generate residual information based on the conversion coefficients. The encoding device can generate residual information indicating the quantized conversion coefficients. Residual information can be generated through various encoding methods such as exponential Golomb, CAVLC, CABAC, etc.

[0271] The encoding device can derive predicted samples (S930). The encoding device can derive predicted samples for the current block based on the prediction mode. The encoding device can derive predicted samples for the current block based on the prediction mode. In this case, various prediction methods disclosed in this document, such as interpretation or intrapretation, can be applied.

[0272] The encoding device can generate prediction-related information (S940). The encoding device can generate prediction-related information based on prediction samples and / or the mode applied thereto. The prediction-related information may include information for various prediction modes (e.g., merge mode, MVP mode, etc.), MVD information, etc.

[0273] The encoding device can generate a reconstructed sample. The encoding device can generate a reconstructed sample based on the residual information. The reconstructed sample can be generated by adding the residual sample based on the residual information to the predicted sample. Specifically, the encoding device can perform a prediction (intra or inter prediction) for the current block and generate a reconstructed sample based on the predicted sample generated from the original sample and the prediction.

[0274] The restored sample may include a restored luma sample and a restored chroma sample. Specifically, the residual sample may include a residual luma sample and a residual chroma sample. The residual luma sample can be generated based on the original luma sample and a predicted luma sample. The residual chroma sample can be generated based on the original chroma sample and a predicted chroma sample. The encoding device can derive conversion coefficients (luma conversion coefficients) for the residual luma sample and / or conversion coefficients (chroma conversion coefficients) for the residual chroma sample. The quantized conversion coefficients may include quantized luma conversion coefficients and / or quantized chroma conversion coefficients.

[0275] The encoding device can now generate in-loop filtering-related information for the restored picture sample (S950). The encoding device can perform an in-loop filtering procedure on the restored sample and generate in-loop filtering-related information based on the in-loop filtering procedure. For example, the in-loop filtering-related information may include the virtual boundary information described in this document (such as the virtual boundary availability flag for the SPS, the virtual boundary availability flag for the picture header, the virtual boundary existence flag for the SPS, the virtual boundary existence flag for the picture header, and information regarding the location of the virtual boundary).

[0276] The encoding device can encode video / image information (S960). The encoding device can encode video / image information including the residual information, prediction-related information, and in-loop filtering-related information. The encoded video / image information can be output in the form of a bitstream. The bitstream can be transmitted to a decoding device via a network or storage medium.

[0277] The aforementioned image / video information may include a variety of information according to the embodiments described herein. For example, the image / video information may include information disclosed in at least one of Tables 1 to 32 described above.

[0278] In one embodiment, the image information may include an SPS and picture header information referencing the SPS. The SPS may include a virtual boundary availability flag (the virtual boundary availability flag of the SPS) related to whether signaling of information associated with a virtual boundary exists (or is available) in the SPS or the picture header information. The in-loop filtering procedure may be executed across the virtual boundary (or not executed across the virtual boundary) based on the virtual boundary availability flag. For example, the virtual boundary availability flag may indicate whether it is possible to disable an in-loop filtering procedure that crosses the virtual boundary.

[0279] In one embodiment, the SPS may include a virtual boundary existence flag for the SPS. For example, based on the virtual boundary existence flag for the SPS, it can be determined whether the SPS contains information regarding the location of the virtual boundary and information regarding the number of the virtual boundary.

[0280] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of vertical virtual boundaries.

[0281] In one embodiment, the SPS may include information regarding the location of a vertical virtual boundary. Furthermore, the number of pieces of information regarding the location of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0282] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of horizontal virtual boundaries.

[0283] In one embodiment, the SPS may include information regarding the location of a horizontal virtual boundary. Furthermore, the number of pieces of information regarding the location of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0284] In one embodiment, based on the value of the virtual boundary availability flag (SPS virtual boundary availability flag) being 1 and the value of the virtual boundary existence flag being 0, the picture header information may include the picture header's virtual boundary existence flag.

[0285] In one embodiment, based on the value of the virtual boundary presence flag in the picture header being 1, the picture header information may include information regarding the number of vertical virtual boundaries.

[0286] In one embodiment, the picture header information may include information about the position of a vertical virtual boundary. Furthermore, the number of pieces of information about the position of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0287] In one embodiment, based on the value of the virtual boundary presence flag in the picture header being 1, the picture header information may include information regarding the number of horizontal virtual boundaries.

[0288] In one embodiment, the picture header information may include information about the position of a horizontal virtual boundary. Furthermore, the number of pieces of information about the position of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0289] In one embodiment, based on the fact that the SPS includes information about the position of vertical virtual boundaries and information about the position of horizontal virtual boundaries, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be greater than 0.

[0290] In one embodiment, the image information (and / or in-loop filtering related information, virtual boundary related information) may further include a virtual boundary presence flag for the SPS, a virtual boundary presence flag for the picture header, and a gradual decoding refresh (GDR) enabled flag. For example, based on the value of the GDR enabled flag being 1, the value of the virtual boundary enabled flag (SPS virtual boundary enabled flag) may be 1, the value of the SPS virtual boundary presence flag may be 0, and the value of the picture header virtual boundary presence flag may be 1 (signaling of virtual boundary information may be present in the picture header).

[0291] Figures 11 and 12 schematically illustrate an example of a video / image decoding method and related components according to the embodiments described herein.

[0292] The method disclosed in Figure 11 can be performed by the decoding device disclosed in Figure 3 or Figure 12. Specifically, for example, S1100 in Figure 11 can be performed by the entropy decoding unit 310 of the decoding device, S1110 and S1120 can be performed by the residual processing unit 320 of the decoding device, S1130 can be performed by the prediction unit 330 of the decoding device, S1140 can be performed by the addition unit 340 of the decoding device, and S1150 can be performed by the filtering unit 350 of the decoding device. The method disclosed in Figure 11 may include the embodiments described above in this document.

[0293] Referring to Figure 11, the decoding device can receive / acquire video / image information (S1100). The video / image information may include residual information, prediction-related information, and / or in-loop filtering-related information (and / or virtual boundary-related information). The decoding device can receive / acquire the image / video information via a bitstream.

[0294] The aforementioned image / video information may include a variety of information according to the embodiments described herein. For example, the aforementioned image / video information may include information disclosed in at least one of Tables 1 to 32 described above.

[0295] The decoding device can derive quantized transformation coefficients. The decoding device can derive quantized transformation coefficients based on the residual information. The quantized transformation coefficients may take the form of a one-dimensional vector based on the coefficient scan order. The quantized transformation coefficients may include quantized luma transformation coefficients and / or quantized chroma transformation coefficients.

[0296] The decoding device can derive the conversion coefficients (S1110). The decoding device can derive the conversion coefficients based on an inverse quantization procedure for the quantized conversion coefficients. The decoding device can derive the Luma conversion coefficients via inverse quantization based on the quantized Luma conversion coefficients. The decoding device can derive the Chroma conversion coefficients via inverse quantization based on the quantized Chroma conversion coefficients.

[0297] The decoding device can generate / derive a residual sample (S1120). The decoding device can derive a residual sample based on an inverse transformation procedure for the transformation coefficients. The decoding device can derive a residual luma sample via an inverse transformation procedure based on luma transformation coefficients. The decoding device can derive a residual chroma sample via an inverse transformation procedure based on chroma transformation coefficients.

[0298] The decoding device can generate prediction samples (S1130). The decoding device can generate prediction samples for the current block based on the prediction-related information. The decoding device can perform predictions based on the image / video information and derive prediction samples for the current block. The prediction-related information may include prediction mode information. The decoding device can determine whether interpretation or intraprediction is applied to the current block based on the prediction mode information and can perform predictions based on this. The prediction samples may include prediction chroma samples and / or prediction chroma samples.

[0299] The decoding device can generate / derive a restored sample (S1140). For example, the decoding device can generate / derive a restored luminal sample and / or a restored chromatic sample. The decoding device can generate a restored luminal sample and / or a restored chromatic sample based on the residual information. The decoding device can generate a restored sample based on the residual information. The restored sample may include a restored luminal sample and / or a restored chromatic sample. The luminal component of the restored sample may correspond to the restored luminal sample, and the chromatic component of the restored sample may correspond to the restored chromatic sample. The decoding device can generate a predicted luminal sample and / or a predicted chromatic sample via a prediction procedure. The decoding device can generate a restored luminal sample based on the predicted luminal sample and the residual luminal sample. The decoding device can generate a restored chromatic sample based on the predicted chromatic sample and the residual chromatic sample.

[0300] The decoding device can generate a modified (filtered) reconstructed sample (S1150). The decoding device can generate a modified reconstructed sample by performing an in-loop filtering procedure on the reconstructed sample of the current picture. The decoding device can generate a modified reconstructed sample based on in-loop filtering related information (and / or virtual boundary related information). The decoding device can use a deblocking procedure, an SAO procedure, and / or an ALF procedure to generate a modified reconstructed sample.

[0301] In one example, step S1150 may include a step of determining whether the in-loop filtering procedure is performed across a virtual boundary. That is, the decoding device may determine whether the in-loop filtering procedure is performed across a virtual boundary. The decoding device may determine whether the in-loop filtering procedure is performed based on in-loop filtering-related information (and / or virtual boundary-related information).

[0302] In one embodiment, the image information may include an SPS and picture header information referencing the SPS. The SPS may include a virtual boundary availability flag (a virtual boundary presence flag for the SPS). Based on the virtual boundary availability flag, it can be determined whether signaling of information associated with the virtual boundary exists (or is available) in the SPS or the picture header information. Based on the determination, the step of generating the modified restored sample (S1150) may include a step of performing the in-loop filtering procedure across the virtual boundary (or a step of performing the in-loop filtering procedure without crossing the virtual boundary). For example, the virtual boundary availability flag may indicate whether it is possible to disable the in-loop filtering procedure across the virtual boundary.

[0303] In one embodiment, the SPS may include a virtual boundary existence flag for the SPS. For example, based on the virtual boundary existence flag for the SPS, it can be determined whether the SPS contains information regarding the location of the virtual boundary and information regarding the number of the virtual boundary.

[0304] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of vertical virtual boundaries.

[0305] In one embodiment, the SPS may include information regarding the location of a vertical virtual boundary. Furthermore, the number of pieces of information regarding the location of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0306] In one embodiment, based on the value of the virtual boundary existence flag of the SPS being 1, the SPS may include information regarding the number of horizontal virtual boundaries.

[0307] In one embodiment, the SPS may include information regarding the location of a horizontal virtual boundary. Furthermore, the number of pieces of information regarding the location of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0308] In one embodiment, the image information includes picture header information, and based on the value of the virtual boundary availability flag (SPS virtual boundary availability flag) being 1 and the value of the SPS virtual boundary existence flag being 0, the picture header information may include the picture header virtual boundary existence flag.

[0309] In one embodiment, based on the value of the virtual boundary presence flag in the picture header being 1, the picture header information may include information regarding the number of vertical virtual boundaries.

[0310] In one embodiment, the picture header information may include information about the position of a vertical virtual boundary. Furthermore, the number of pieces of information about the position of the vertical virtual boundary can be determined based on the number of vertical virtual boundaries.

[0311] In one embodiment, based on the value of the virtual boundary presence flag in the picture header being 1, the picture header information may include information regarding the number of horizontal virtual boundaries.

[0312] In one embodiment, the picture header information may include information about the position of a horizontal virtual boundary. Furthermore, the number of pieces of information about the position of the horizontal virtual boundary can be determined based on the number of horizontal virtual boundaries.

[0313] In one embodiment, based on the fact that the SPS includes information about the position of vertical virtual boundaries and information about the position of horizontal virtual boundaries, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries may be greater than 0.

[0314] In one embodiment, the image information (and / or in-loop filtering related information, virtual boundary related information) may further include a virtual boundary presence flag for the SPS, a virtual boundary presence flag for the picture header, and a gradual decoding refresh (GDR) enabled flag. For example, based on the value of the GDR enabled flag being 1, the value of the virtual boundary enabled flag (SPS virtual boundary enabled flag) may be 1, the value of the SPS virtual boundary presence flag may be 0, and the value of the picture header virtual boundary presence flag may be 1 (signaling of virtual boundary information may be present in the picture header).

[0315] The decoding device can receive information about the residual for the current block if a residual sample exists for the current block. The information about the residual may include transformation coefficients for the residual sample. Based on the residual information, the decoding device can derive a residual sample (or a residual sample array) for the current block. Specifically, the decoding device can derive quantized transformation coefficients based on the residual information. The quantized transformation coefficients may have a one-dimensional vector form based on the coefficient scan order. The decoding device can derive transformation coefficients based on an inverse quantization procedure for the quantized transformation coefficients. Based on the transformation coefficients, the decoding device can derive a residual sample.

[0316] The decoding device can generate a reconstructed sample based on an (intra) predicted sample and a residual sample, and derive a reconstructed block or reconstructed picture based on the reconstructed sample. Specifically, the decoding device can generate a reconstructed sample based on the sum of an (intra) predicted sample and a residual sample. Thereafter, as described above, the decoding device may apply deblocking filtering and / or in-loop filtering procedures such as the SAO procedure to the reconstructed picture to improve subjective / objective image quality as needed.

[0317] For example, a decoding device can decode a bitstream or encoded information to obtain image information that includes all or part of the aforementioned information (or syntax elements). Furthermore, the bitstream or encoded information can be stored on a computer-readable storage medium, which can trigger the aforementioned decoding method.

[0318] In the embodiments described above, the method is explained based on a flowchart as a series of steps or blocks, but the embodiments are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and that different steps may be included, or one or more steps in the flowchart may be omitted without affecting the scope of the embodiments described herein.

[0319] The methods according to the embodiments of this document described above can be implemented in software form, and the encoding and / or decoding devices related to this document may be included in, for example, image processing devices such as TVs, computers, smartphones, set-top boxes, and display devices.

[0320] In this document, when embodiments are implemented in software, the methods described above can be implemented by modules (processes, functions, etc.) that perform the functions described above. These modules are stored in memory and can be executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by various well-known means. The processor may include an ASIC (application-specific integrated circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (read-only memory), RAM (random access memory), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in this document may be implemented on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing may be implemented on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0321] Furthermore, the decoding and encoding devices to which the embodiments of this document apply may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communication, mobile streaming devices, storage media, camcorders, customized video (VoD) service providers, OTT video (Over the top video) devices, internet streaming service providers, 3D video devices, VR (virtual reality) devices, AR (argumente reality) devices, image phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video equipment, and may be used to process video signals or data signals. For example, OTT video (Over the top video) devices may include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0322] Furthermore, the processing methods to which the embodiments of this document apply can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having a data structure according to the embodiments of this document can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices that store data to be read by a computer. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium also includes media implemented in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method can be stored on a computer-readable recording medium or transmitted over a wired wireless network.

[0323] Furthermore, the embodiments described herein can be implemented as computer program products using program code, and the program code can be executed on a computer according to the embodiments described herein. The program code can be stored on a computer-readable carrier.

[0324] Figure 13 shows an example of a content streaming system to which the embodiments disclosed in this document may be applied.

[0325] Referring to Figure 13, the content streaming system to which the embodiments described in this document apply can broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0326] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then transmitting this bitstream to the streaming server. As an alternative, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server may be omitted.

[0327] The bitstream can be generated by an encoding method or a bitstream generation method to which an embodiment of this document applies, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0328] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0329] The streaming server can receive content from a media storage and / or encoding server. For example, if it starts receiving content from the encoding server, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0330] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, and HMDs), digital TVs, desktop computers, and digital signage.

[0331] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0332] The claims described herein can be combined in various ways. For example, the technical features of the method claims herein can be combined to realize an apparatus, and the technical features of the apparatus claims herein can be combined to realize a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize an apparatus, and the technical features of the method claims and the technical features of the apparatus claims herein can be combined to realize a method.

Claims

1. A decoding device for image decoding, Memory and The system comprises at least one processor connected to the memory, The aforementioned at least one processor is Image information, including residual information, prediction-related information, and in-loop filtering-related information, is acquired via a bitstream. Based on the aforementioned residual information, the conversion coefficient is derived. Based on the conversion coefficient, a residual sample is generated. Based on the aforementioned prediction-related information, prediction samples are generated. Based on the residual sample and the predicted sample, a reconstructed sample is generated. It is configured to generate a corrected restored sample based on the in-loop filtering related information, The aforementioned image information includes an SPS (sequence parameter set) and picture header information that references the SPS. The aforementioned SPS includes a virtual boundary availability flag and an SPS virtual boundary existence flag. The generation of the modified restored sample is Performing the in-loop filtering procedure on the restored sample, Based on the SPS virtual boundary existence flag, the process includes determining whether (i) information related to the virtual boundary exists in the SPS, or (ii) information related to the virtual boundary exists in the picture header information. Based on the virtual boundary availability flag, it is determined whether the in-loop filtering procedure that crosses the virtual boundary is enabled. Based on the SPS virtual boundary existence flag, the SPS includes information regarding the number of vertical virtual boundaries. Based on the information relating to the number of vertical virtual boundaries, the SPS is a decoding device that includes information relating to the location of the vertical virtual boundaries.

2. An encoding device for image encoding, Memory and The system comprises at least one processor connected to the memory, The aforementioned at least one processor is Currently, we derive a residual sample for the block, Based on the residual sample for the current block, the conversion coefficients are derived. Based on the aforementioned conversion coefficients, residual information is generated, Based on the residual sample, predictive samples are generated for the current block. Based on the aforementioned prediction samples, prediction-related information is generated. Generate in-loop filtering-related information for the in-loop filtering procedure of the restored sample. It is configured to encode image information including the residual information, the prediction-related information, and the in-loop filtering-related information, The aforementioned image information includes an SPS (sequence parameter set) and picture header information that references the SPS. The SPS includes a virtual boundary availability flag, and an SPS virtual boundary existence flag related to (i) whether information related to the virtual boundary exists in the SPS, or (ii) whether information related to the virtual boundary exists in the picture header information. The value of the virtual boundary availability flag is determined based on whether the in-loop filtering procedure is enabled across the virtual boundary. Based on the SPS virtual boundary existence flag, the SPS includes information regarding the number of vertical virtual boundaries. Based on the information relating to the number of vertical virtual boundaries, the SPS is an encoding device that includes information relating to the position of the vertical virtual boundaries.

3. In a device for transmitting image-related data, At least one processor configured to acquire the bitstream of the aforementioned image, wherein the bitstream is Currently, the goal is to derive a residual sample for the block, Based on the residual sample for the current block, the conversion coefficient is derived, Based on the aforementioned conversion coefficients, residual information is generated, Based on the residual sample, predictive samples are generated for the current block. Based on the aforementioned prediction samples, prediction-related information is generated, To generate in-loop filtering-related information for the in-loop filtering procedure of the restored sample, A processor generated based on encoding image information including the residual information, the prediction-related information and the in-loop filtering-related information, A transmitting unit configured to transmit the data including the bitstream, The aforementioned image information includes an SPS (sequence parameter set) and picture header information that references the SPS. The SPS includes a virtual boundary availability flag, and an SPS virtual boundary existence flag related to (i) whether information related to the virtual boundary exists in the SPS, or (ii) whether information related to the virtual boundary exists in the picture header information. The value of the virtual boundary availability flag is determined based on whether the in-loop filtering procedure is enabled across the virtual boundary. Based on the SPS virtual boundary existence flag, the SPS includes information regarding the number of vertical virtual boundaries. Based on the information relating to the number of the vertical virtual boundaries, the SPS is a device that includes information relating to the position of the vertical virtual boundaries.

Images