Image decoding methods, image encoding methods, and image data transmission methods

Abstract

This disclosure provides an image decoding method, an image encoding method, and an image data transmission method. According to embodiments of this document, information for performing in-loop filtering across virtual boundaries can be effectively communicated using signals. For example, in-loop filtering can be performed based on signaling related to whether in-loop filtering across virtual boundaries is permitted.

Description

[0001] This application is a divisional application of the original invention patent application No. 202080093245.7 (International Application No.: PCT / KR2020 / 016140, Application Date: November 17, 2020, Invention Title: Image Coding Apparatus and Method Based on Filtering). Technical Field

[0002] This document relates to a filter-based image coding device and method. Background Technology

[0003] Recently, the demand for high-resolution, high-quality images / videos, such as 4K, 8K, or even higher Ultra High Definition (UHD) images / videos, has increased across various fields. As image / video data becomes more high-resolution and of higher quality, the amount of information or bits to be transmitted increases relative to existing image / video data. Therefore, transmitting image data using media such as existing wired / wireless broadband lines or existing storage media, or storing image / video data using existing storage media, increases both transmission and storage costs.

[0004] In addition, interest and demand for immersive media such as virtual reality (VR) and artificial reality (AR) content or holograms have recently increased, and the broadcasting of images / videos with characteristics different from real images (e.g., game images) has increased.

[0005] Therefore, highly efficient image / video compression technologies are needed to effectively compress, transmit, store, and reproduce high-resolution, high-quality image / video information with the various characteristics described above.

[0006] Specifically, in-loop filtering is performed to improve subjective / objective visual quality, and schemes to increase the signaling efficiency of information used for in-loop filtering based on virtual boundaries are discussed. Summary of the Invention

[0007] Technical solution

[0008] According to the embodiments of this document, a method and apparatus for increasing image coding efficiency are provided.

[0009] According to the implementation method of this document, a high-efficiency filtering application method and device are provided.

[0010] According to the embodiments of this document, a method and apparatus for effectively applying deblocking, sample adaptive offset (SAO), and adaptive loop filtering (ALF) are provided.

[0011] According to the implementation method described in this document, in-loop filtering can be performed based on virtual boundaries.

[0012] According to the implementation of this document, the sequence parameter set (SPS) may include an SPS virtual boundary enable flag indicating whether in-loop filtering is performed across virtual boundaries.

[0013] According to the implementation method described in this document, in-loop filtering can be performed across virtual boundaries based on the SPS virtual boundary enable flag.

[0014] According to the embodiments of this document, an encoding device for performing video / image encoding is provided.

[0015] According to one embodiment of this document, a computer-readable digital storage medium is provided, wherein encoded video / image information generated according to the video / image encoding method disclosed in at least one embodiment of this document is stored.

[0016] According to embodiments of this document, a computer-readable digital storage medium is provided, which stores encoded information or encoded video / image information that causes a decoding device to perform the video / image decoding method disclosed in at least one embodiment of this document.

[0017] Beneficial effects

[0018] According to the implementation method described in this document, the overall image / video compression efficiency can be improved.

[0019] According to the implementation method described in this document, subjective / objective visual quality can be improved through efficient filtering.

[0020] The virtual boundary-based in-loop filtering process according to the implementation method in this document can save hardware resources.

[0021] According to the implementation method described in this document, loop-based filtering based on virtual boundaries can be effectively performed, and the filtering performance can be improved.

[0022] According to the implementation method described in this document, information for in-loop filtering based on virtual boundaries can be effectively communicated using signals. Attached Figure Description

[0023] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0024] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding apparatus to which embodiments of the present disclosure can be applied.

[0025] Figure 3 This is a schematic diagram illustrating the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0026] Figure 4An example of a layered architecture for encoding video / images is shown.

[0027] Figure 5 This is a flowchart illustrating a filtering-based encoding method in an encoding device.

[0028] Figure 6 This is a flowchart illustrating a filtering-based decoding method in a decoding device.

[0029] Figure 7 and Figure 8 Examples of video / image coding methods and related components according to embodiments of this document are illustrated schematically.

[0030] Figure 9 and Figure 10 Examples of image / video decoding methods and related components according to embodiments of this document are illustrated schematically.

[0031] Figure 11 Examples of content streaming systems to which the embodiments disclosed in this document can be applied are shown. Detailed Implementation

[0032] This disclosure may be modified in various forms, as will be described and illustrated in the accompanying drawings. However, these embodiments are not intended to limit this disclosure. The terminology used in the following description is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions, which shall be clearly distinguishable by different readings. Terms such as “comprising” and “having” are intended to indicate the presence of the features, quantities, steps, operations, elements, components or combinations thereof used in the following description, and therefore it should be understood that the possibility of having or adding one or more different features, quantities, steps, operations, elements, components or combinations thereof is not excluded.

[0033] Furthermore, the various configurations described in the accompanying drawings are independent examples illustrating functions that are distinct from each other, and are not intended to imply that the configurations are implemented by different hardware or different software. For example, two or more configurations may be combined to form one configuration, and a configuration may be divided into multiple configurations. Embodiments in which configurations are combined and / or separated are included within the scope of the claims without departing from the spirit of this document.

[0034] Hereinafter, examples of this embodiment will be described in detail with reference to the accompanying drawings. Furthermore, similar reference numerals are used throughout the drawings to indicate similar elements, and identical descriptions of similar elements will be omitted.

[0035] This document relates to video / image coding. For example, the methods / implementations disclosed in this document may relate to the Universal Video Coding (VVC) standard (ITU-T Rec.H.266), next-generation video / image coding standards after VVC, or other video coding-related standards (e.g., High Efficiency Video Coding (HEVC) standard (ITU-T Rec.H.265), Basic Video Coding (EVC) standard, AVS2 standard, etc.).

[0036] This document presents various implementations of video / image coding, and unless otherwise specified, the above implementations may also be combined with each other.

[0037] In this document, video can refer to a series of images over time. A frame typically refers to a unit representing an image within a specific time range, while a slice / piece refers to a unit that constitutes part of a frame in terms of encoding. A slice / piece may include one or more coding tree units (CTUs). A frame may consist of one or more slices / pieces. A frame may consist of one or more groups of pieces. A group of pieces may include one or more pieces.

[0038] A pixel or image unit can refer to the smallest unit that makes up a picture (or image). Additionally, the term "sample" can be used as the counterpart to a pixel. A sample can typically represent a pixel or pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0039] A unit can represent a basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region." In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M ​​columns and N rows. Alternatively, a sample may refer to a pixel value in the spatial domain, and when such pixel value is transformed to the frequency domain, it may refer to a transform coefficient in the frequency domain.

[0040] In this document, the terms “ / ” and “,” should be interpreted as indicating “and / or”. For example, the expression “A / B” can mean “A and / or B”. Furthermore, “A,B” can mean “A and / or B”. Additionally, “A / B / C” can mean “at least one of A, B, and / or C”. Also, “A / B / C” can mean “at least one of A, B, and / or C”.

[0041] Furthermore, in this document, the term "or" should be interpreted as indicating "and / or". For example, expressing "A or B" can include 1) only A, 2) only B, and / or 3) both A and B. In other words, the term "or" in this document should be interpreted as indicating "additionally or alternatively".

[0042] In this specification, "at least one of A and B" may mean "A only", "B only" or "both A and B". Furthermore, in this specification, the expressions "at least one of A or B" or "at least one of A and / or B" may be interpreted in the same way as "at least one of A and B".

[0043] Furthermore, in this specification, "at least one of A, B, and C" may mean "A only", "B only", "C only" or "any combination of A, B, and C". Additionally, "at least one of A, B, or C" or "at least one of A, B, and / or C" may mean "at least one of A, B, and C".

[0044] Furthermore, the parentheses used in this specification may mean "for example". Specifically, when expressing "prediction (intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction". In other words, the term "prediction" in this specification is not limited to "intra-frame prediction" and may indicate that "intra-frame prediction" is proposed as an example of "prediction". Moreover, even when expressing "prediction (i.e., intra-frame prediction)", it may indicate that "intra-frame prediction" is proposed as an example of "prediction".

[0045] In this specification, the technical features described separately in a single figure can be implemented individually or simultaneously.

[0046] Figure 1 Examples of publicly disclosed video / image coding systems applicable to this document are shown.

[0047] Reference Figure 1 A video / image encoding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or network.

[0048] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.

[0049] Video sources can acquire video / images through processes that capture, synthesize, or generate video / images. Video sources may include video / image capture devices and / or video / image generation devices. For example, a video / image capture device may include one or more cameras, a video / image archive containing previously captured video / images, etc. For example, a video / image generation device may include a computer, tablet computer, and smartphone, and may generate video / images (electronically). For example, virtual video / images may be generated via a computer, etc. In this case, the video / image capture process may be replaced by a process that generates related data.

[0050] Encoding devices can encode input video / images. For compression and encoding efficiency, encoding devices can perform a series of processes such as prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output as a bitstream.

[0051] The transmitter can send encoded image / image information or data, output as a bitstream, to a receiver of a receiving device in the form of a file or stream via a digital storage medium or network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0052] Decoding devices can decode video / images by performing a series of processes such as dequantization, inverse transform, and prediction, which correspond to the operations of encoding devices.

[0053] The renderer can render decoded video / images. The rendered video / images can be displayed on a monitor.

[0054] Figure 2 This is a schematic diagram illustrating the configuration of a video / image encoding apparatus to which this document can be applied. Hereinafter, the term "video encoding apparatus" may include image encoding apparatus.

[0055] Reference Figure 2The encoding device 200 may include and be configured with an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by one or more hardware components (e.g., an encoder chipset or processor). Additionally, the memory 270 may include a decoded picture buffer (DPB) and may also be configured by a digital storage medium. The hardware components may also include the memory 270 as an internal / external component.

[0056] Image segmenter 210 can divide an input image (or picture, frame) input to encoding device 200 into one or more processing units. As an example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure. For example, a coding unit may be divided into multiple deeper coding units based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. In this case, for example, a quadtree structure may be applied first, followed by a binary tree structure and / or a ternary tree structure. Alternatively, a binary tree structure may be applied first. Encoding processing according to this disclosure may be performed based on the final coding unit that is no longer divided. In this case, based on encoding efficiency such as image characteristics, the maximum coding unit may be directly used as the final coding unit, or, if necessary, the coding unit may be recursively divided into deeper coding units such that a coding unit with an optimal size can be used as the final coding unit. Here, encoding processing may include processes such as prediction, transformation, and reconstruction (described later). As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit may be split or divided from the aforementioned final encoding unit. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0057] In some cases, a unit can be used interchangeably with terms such as block or region. Typically, an M×N block can represent a sample or a set of transform coefficients consisting of M columns and N rows. A sample can typically represent a pixel or pixel value, or it can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. A sample can be used as a term corresponding to the pixels or picometers that configure a frame (or image).

[0058] Subtractor 231 generates a residual signal (residual block, residual sample, or residual sample array) by subtracting the prediction signal (prediction block, prediction sample, or prediction sample array) output from predictor 220 from the input image signal (original block, original sample, or original sample array), and the generated residual signal is sent to converter 232. Predictor 220 performs prediction on the processing target block (hereinafter referred to as "current block") and generates a prediction block including prediction samples of the current block. Predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction on the current block or on a CU-by-CU basis. As described later in the description of the various prediction modes, the predictor can generate various types of information related to the prediction (e.g., prediction mode information) and transmit the generated information to entropy encoder 240. The information about the prediction can be encoded in entropy encoder 240 and output as a bitstream.

[0059] Intra-predictor 222 can refer to samples within the current frame to predict the current block. Depending on the prediction mode, the referenced samples may be located near or far from the current block. The prediction modes in intra-prediction can include multiple non-directional modes and multiple directional modes. For example, non-directional modes can include DC modes or planar modes. For example, depending on the fineness of the prediction direction, directional modes can include 33 or 65 directional prediction modes. However, this is exemplary, and more or fewer directional prediction modes may be used depending on the settings. Intra-predictor 222 can also use prediction modes applied to neighboring blocks to determine the prediction mode applied to the current block.

[0060] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing within the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same as each other, or they may be different from each other. The temporally neighboring block may be referred to by names such as juxtaposed reference block, juxtaposed CU (colCU), etc., and the reference frame including the temporally neighboring block may also be referred to as a juxtaposed frame (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use motion information of neighboring blocks as motion information of the current block. In skip mode, unlike merge mode, residual signals may not be sent. Motion Vector Prediction (MVP) mode can use motion vectors of neighboring blocks as motion vector predictors and signal the motion vector difference to indicate the motion vector of the current block.

[0061] Predictor 220 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to predict a block, but also both intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Additionally, the predictor can perform intra-frame block copying (IBC) for predicting blocks. Intra-frame block copying can be used for content image / moving image coding in games, such as screen content coding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction, such that a reference block is derived in the current frame. That is, IBC can use at least one inter-frame prediction technique described in this document.

[0062] The predicted signals generated by the inter-frame predictor 221 and / or the intra-frame predictor 222 can be used to generate a reconstructed signal or a residual signal. The transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Graphical Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graphic when the relationship information between pixels is represented graphically. CNT refers to a transform obtained based on the predicted signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size or to blocks of variable size other than squares.

[0063] Quantizer 233 quantizes the transform coefficients and sends them to entropy encoder 240, which encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. This information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-type quantized transform coefficients into a one-dimensional vector based on the coefficient scan order, and generate information about the quantized transform coefficients based on this one-dimensional vector. Entropy encoder 240 can perform various encoding methods such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). Entropy encoder 240 can encode the information required for video / image reconstruction, either together with or separately from the quantized transform coefficients (e.g., values ​​of syntax elements). The encoded information (e.g., encoded video / image information) can be sent or stored in units of Network Abstraction Layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. In this document, the information and / or syntactic elements notified / transmitted by signals, as described later, can be encoded by the above-described encoding process and included in the bitstream. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits the signal output from the entropy encoder 240 and / or a storage unit (not shown) that stores the signal may be configured as an internal / external element of the encoding device 200, and alternatively, the transmitter may be included within the entropy encoder 240.

[0064] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transformer 235. Adder 250 adds the reconstructed residual signal to the prediction signal output from predictor 220 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample, or array of reconstructed samples). If the processing target block has no residual (e.g., in the case of applying skip mode), the prediction block can be used as a reconstructed block. As described below, the generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current frame and can be filtered for inter-frame prediction of the next frame.

[0065] In addition, luminance mapping with chroma scaling (LMCS) can be applied during screen encoding and / or reconstruction processing.

[0066] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 270 (specifically, the DPB of memory 270). Various filtering methods may include deblocking filtering, sample adaptive offset (SAO), adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various types of filtering-related information and transmit the generated information to entropy encoder 290, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 290 and output as a bitstream.

[0067] The modified reconstructed frame sent to memory 270 can be used as a reference frame in inter-frame predictor 221. When inter-frame prediction is applied by the encoding device, prediction mismatch between encoding device 200 and decoding device 300 can be avoided and encoding efficiency can be improved.

[0068] The DPB of memory 270 can store a modified reconstructed frame used as a reference frame in inter-frame predictor 221. Memory 270 can store motion information of blocks in the current frame that derive (or encode) motion information and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame predictor 221 as motion information for spatially or temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current frame and can transmit the reconstructed samples to intra-frame predictor 222.

[0069] Figure 3 This is a diagram used to schematically illustrate the configuration of the publicly disclosed video / image decoding device applicable to this document.

[0070] Reference Figure 3The decoding device 300 may include and be configured with an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 331 and an intra-frame predictor 332. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above may be configured by one or more hardware components (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware components may also include the memory 360 as an internal / external component.

[0071] When an input bitstream including video / image information is received, the decoding device 300 can respond to... Figure 2 The encoding device shown reconstructs an image by processing video / image information. For example, the decoding device 300 can deduce units / blocks based on block partitioning information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied to the encoding device. Thus, for example, the processing unit for decoding can be an encoding unit, and the encoding unit can be partitioned from encoding tree units or maximum encoding units according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding units. Furthermore, the reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.

[0072] Decoding device 300 can receive from Figure 2The encoding device outputs a signal in the form of a bitstream, and the received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals, as described later in this document, can be decoded and obtained from the bitstream by the decoding process. For example, the entropy decoder 310 decodes the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and outputs quantized values ​​of the syntax elements and transform coefficients of the residuals required for image reconstruction. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntactic element in the bitstream, determine a context model using information about the target syntactic element, decoding information of the target block, or information about symbols / bins decoded in a previous stage, and perform arithmetic decoding on the bins by predicting the probability of bin occurrence based on the determined context model, generating symbols corresponding to the values ​​of each syntactic element. In this case, the CABAC entropy decoding method can update the context model after determining the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. Information related to prediction from the information decoded by the entropy decoder 310 can be provided to the predictor 330, and information about the residuals from the entropy decoding performed in the entropy decoder 310 (i.e., quantized transform coefficients and related parameter information) can be input to the dequantizer 321. Additionally, information about filtering from the information decoded by the entropy decoder 310 can be provided to the filter 350. Furthermore, a receiver (not shown) for receiving signals output from the encoding device can be configured as an internal / external element of the decoding device 300, or the receiver can be a component of the entropy decoder 310. Furthermore, the decoding device according to this document may be referred to as a video / image / screen decoding device, and the decoding device may be classified as an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, a predictor 330, an adder 340, a filter 350, and a memory 360.

[0073] Dequantizer 321 can dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed by the encoding device. Dequantizer 321 can use quantization parameters (e.g., quantization step size information) to perform dequantization on the quantized transform coefficients and obtain the transform coefficients.

[0074] The inverse transformer 322 performs inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0075] Predictor 330 can perform prediction for the current block and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on information about the prediction output from entropy decoder 310 and determine a specific intra-frame / inter-frame prediction mode.

[0076] The predictor can generate a predicted signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to predict a block, but also both intra-frame and inter-frame prediction simultaneously. This can be referred to as combined intra-frame and inter-frame prediction (CIIP). Additionally, the predictor can perform intra-frame block copying (IBC) for predicting blocks. Intra-frame block copying can be used for content image / moving image coding in games, such as screen content coding (SCC). IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction, such that a reference block is derived in the current frame. That is, IBC can use at least one inter-frame prediction technique described in this document.

[0077] Intra-predictor 332 can refer to samples in the current frame to predict the current block. Depending on the prediction mode, the referenced samples may be located near or far from the current block. In intra-prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. Intra-predictor 332 can use prediction modes applied to neighboring blocks to determine the prediction mode to be applied to the current block.

[0078] Inter-frame predictor 331 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include information about the inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. For example, inter-frame predictor 331 can construct a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0079] Adder 340 generates a reconstruction signal (reconstructed image, reconstruction block, and reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from predictor 330. If the target block has no residual, such as when a skip mode is applied, the prediction block can be used as the reconstruction block.

[0080] Adder 340 may be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and as described later, it can also be output through filtering, or it can also be used for inter-frame prediction of the next frame.

[0081] In addition, Luminance Mapping with Chroma Scaling (LMCS) can also be applied in image decoding processing.

[0082] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 360 (specifically, the DPB of memory 360). For example, various filtering methods may include deblocking filtering, sample adaptive shifting, adaptive loop filtering, bilateral filtering, etc.

[0083] The (modified) reconstructed frame stored in the DPB of memory 360 can be used as a reference frame in inter-frame predictor 331. Memory 360 can store motion information of blocks in the current frame from which motion information is derived (or decoded) and / or motion information of already reconstructed blocks in the frame. The stored motion information can be transmitted to inter-frame predictor 331 to be used as motion information of spatially or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra-frame predictor 332.

[0084] In this specification, the embodiments described in the predictor 330, dequantizer 321, inverse transformer 322 and filter 350 of the decoding device 300 can also be applied in the same or corresponding manner as the predictor 220, dequantizer 234, inverse transformer 235 and filter 260 of the encoding device 200.

[0085] Furthermore, as described above, prediction is performed during video encoding to improve compression efficiency. This generates a prediction block that includes prediction samples of the current block, which is the block to be encoded (i.e., the target block). Here, the prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived in the same manner as in the encoding and decoding devices, and the encoding device can signal the decoding device information about the residual between the original block and the prediction block (residual information) instead of the original sample values ​​of the original block, thereby increasing image encoding efficiency. The decoding device can derive a residual block including residual samples based on the residual information, add the residual block and the prediction block to generate a reconstruction block including reconstructed samples, and generate a reconstructed image including the reconstruction block.

[0086] Residual information can be generated through transform and quantization processes. For example, the encoding device can derive a residual block between the original block and the prediction block, perform transform processing on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform quantization processing on the transform coefficients to derive quantized transform coefficients, and signal the relevant residual information (via bitstream) to the decoding device. Here, the residual information may include the value information, position information, transform technique, transform kernel, quantization parameters, etc., of the quantized transform coefficients. The decoding device can perform dequantization / inverse transform processing based on the residual information and derive residual samples (or residual blocks). The decoding device can generate a reconstructed frame based on the prediction block and the residual block. In addition, as a reference for inter-frame prediction of later frames, the encoding device can also perform dequantization / inverse transform on the quantized transform coefficients to derive residual blocks and generate a reconstructed frame based on them.

[0087] In this document, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the transform coefficients of the quantization may be referred to as transform coefficients. When transform / inverse transform is omitted, the transform coefficients may be referred to as coefficients or residual coefficients, or, for consistency of expression, may still be referred to as transform coefficients.

[0088] In this document, quantized transform coefficients and transform coefficients can be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, residual information may include information about the transform coefficients, and this information about the transform coefficients can be signaled via residual coding syntax. Transform coefficients can be derived based on residual information (or information about the transform coefficients), and scaled transform coefficients can be derived by the inverse transform (scaling) of the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaled transform coefficients. This can also be applied / expressed in other parts of this document.

[0089] The predictor of an encoding / decoding device can derive prediction samples by performing inter-frame prediction on a block-by-block basis. Inter-frame prediction can be derived in a way that depends on data elements (e.g., sample values ​​or motion information) of frames other than the current frame. When inter-frame prediction is applied to the current block, the prediction block (prediction sample array) of the current block can be derived based on the reference block (reference sample array) specified by the motion vector on the reference frame pointing to the reference frame index. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, the motion information of the current block can be predicted on a block, sub-block, or sample-by-sample basis based on the correlation between the motion information of neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction type (L0 prediction, L1 prediction, Bi prediction, etc.) information. When inter-frame prediction is applied, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. Temporally neighboring blocks can be referred to as collated reference blocks, collated CUs (colCU), etc., and reference frames including temporally neighboring blocks can be referred to as collated frames (colPic). For example, a candidate list of motion information can be constructed based on the neighboring blocks of the current block, and a signal can be used to indicate which candidate is selected (used) to derive the motion vector of the current block and / or the reference frame index, along with flags or index information. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the motion information of the current block can be the same as the motion information of the selected neighboring blocks. In skip mode, unlike merge mode, residual signals may not be sent. In the case of motion vector prediction (MVP) mode, the motion vectors of the selected neighboring blocks can be used as motion vector predictors, and the motion vector difference can be signaled. In this case, the motion vector of the current block can be derived using the sum of the motion vector predictor and the motion vector difference.

[0090] Depending on the inter-frame prediction type (L0 prediction, L1 prediction, Bi prediction, etc.), motion information may include L0 motion information and / or L1 motion information. Motion vectors in the L0 direction may be referred to as L0 motion vectors or MVL0, and motion vectors in the L1 direction may be referred to as L1 motion vectors or MVL1. Prediction based on L0 motion vectors may be called L0 prediction, prediction based on L1 motion vectors may be called L1 prediction, and prediction based on both L0 and L1 motion vectors may be called bi-prediction. Here, L0 motion vectors may indicate motion vectors associated with a reference frame list L0 (L0), and L1 motion vectors may indicate motion vectors associated with a reference frame list L1 (L1). The reference frame list L0 may include frames that are earlier than the current frame in the output order as reference frames, and the reference frame list L1 may include frames that are later than the current frame in the output order. Previous frames may be referred to as forward (reference) frames, and subsequent frames may be referred to as backward (reference) frames. The reference frame list L0 may also include frames that are later than the current frame in the output order as reference frames. In this scenario, in the reference screen list L0, previous screens can be indexed first, followed by subsequent screens. The reference screen list L1 may also include screens that precede the current screen in the output order as reference screens. In this case, subsequent screens can be indexed first in the reference screen list L1, followed by previous screens. Here, the output order may correspond to the screen order count (POC) order.

[0091] Figure 4 An example is shown of the layered structure of an encoded image / video.

[0092] Reference Figure 4 The encoded image / video is divided into a VCL (Video Coding Layer) that handles image / video decoding and its own processing, a subsystem that sends and stores encoded information, and a Network Abstraction Layer (NAL) that exists between the VCL and the subsystems and is responsible for network adaptation functions.

[0093] VCL can generate VCL data including compressed image data (slice data), or generate parameter sets including picture parameter sets (Picture Parameter Set: PSP), sequence parameter sets (Sequence Parameter Set: SPS), video parameter sets (VPS), etc., or additional supplemental enhancement information (SEI) messages required for image decoding processing.

[0094] In NAL, NAL cells are generated by adding header information (NAL cell header) to the raw byte sequence payload (RBSP) generated in VCL. In this case, RBSP refers to slice data, parameter sets, SEI messages, etc., generated in VCL. The NAL cell header may include NAL cell type information specified based on the RBSP data included in the corresponding NAL cell.

[0095] As shown in the figure, NAL units can be divided into VCL NAL units and non-VCL NAL units based on the RBSP generated in the VCL. VCL NAL units can refer to NAL units that include information about the image (slice data), while non-VCL NAL units can refer to NAL units that contain information (parameter set or SEI message) required for decoding the image.

[0096] The aforementioned VCL NAL units and non-VCL NAL units can be transmitted over a network by adding header information according to the data standard of the subsystem. For example, NAL units can be transformed into data forms of predetermined standards such as H.266 / VVC file format, Real-time Transport Protocol (RTP), Transport Stream (TS), etc., and transmitted over various networks.

[0097] As described above, the NAL cell type can be specified in the NAL cell according to the RBSP data structure included in the corresponding NAL cell, and information about the NAL cell type can be stored in the NAL cell header and notified by a signal.

[0098] For example, NAL units can be broadly classified into VCL NAL unit types and non-VCL NAL unit types based on whether the NAL unit includes information about the image (slice data). VCL NAL unit types can be classified based on the nature and type of the image included in the VCL NAL unit, and non-VCL NAL unit types can be classified based on the type of parameter set.

[0099] The following is an example of a NAL cell type specified based on the type of the parameter set included in a non-VCL NAL cell type.

[0100] -APS (Adaptive Parameter Set) NAL Unit: The type of NAL unit including APS.

[0101] -DPS (Decoding Parameter Set) NAL Unit: The type of NAL unit including DPS.

[0102] -VPS (Video Parameter Set) NAL Unit: Includes the type of NAL unit for the VPS.

[0103] -SPS (Sequence Parameter Set) NAL Unit: The type of NAL unit that includes SPS.

[0104] -PPS (Picture Parameter Set) NAL Unit: Includes the types of NAL units for PPS.

[0105] -PH (Header) NAL Unit: Includes the type of NAL unit for PH.

[0106] The aforementioned NAL unit type contains syntactic information for the NAL unit type, and this syntactic information can be stored in the NAL unit header and signaled. For example, the syntactic information can be nal_unit_type, and the NAL unit type can be specified by the nal_unit_type value.

[0107] Furthermore, as mentioned above, a frame can include multiple slices, and a slice can include a slice header and slice data. In this case, a frame header can be further added to multiple slices (slice headers and slice datasets) within a frame. The frame header (frame header syntax) can include information / parameters typically applicable to the frame. In this document, slices can be mixed with or replaced by tile groups. Additionally, in this document, slice headers can be mixed with tile group headers or replaced by type group headers.

[0108] A slice header (slice header syntax or slice header information) may include information / parameters typically applicable to a slice. An APS (APS syntax) or PPS (PPS syntax) may include information / parameters typically applicable to one or more slices or frames. An SPS (SPS syntax) may include information / parameters typically applicable to one or more sequences. A VPS (VPS syntax) may include information / parameters typically applicable to multiple layers. A DPS (DPS syntax) may include information / parameters typically applicable to the entire video. A DPS may include information / parameters related to the concatenation of encoded video sequences (CVS). In this document, High-Level Syntax (HLS) may include at least one of APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, frame header syntax, and slice header syntax.

[0109] In this document, the image / video information encoded in the encoding device and signaled to the decoding device in the form of a bitstream may include not only segmentation-related information, intra / inter-frame prediction information, residual information, and loop filtering information in the frame, but also information included in the slice header, the frame header, the APS, the PPS, the SPS, the VPS, and / or the DPS. Additionally, the image / video information may also include information from the NAL unit header.

[0110] Furthermore, to compensate for differences between the original and reconstructed images caused by errors in compression coding processes such as quantization, loop filtering can be performed on the reconstructed samples or reconstructed images as described above. As mentioned above, loop filtering can be performed by filters from the encoding and decoding devices, and deblocking filters, SAO, and / or adaptive loop filters (ALF) can be applied. For example, ALF processing can be performed after deblocking filtering and / or SAO processing is completed. However, even in this case, deblocking filtering and / or SAO processing can be omitted.

[0111] The following section describes frame reconstruction and filtering in detail. In image / video coding, reconstructed blocks can be generated in individual block units based on intra-frame prediction / inter-frame prediction, and a reconstructed frame including these blocks can be generated. When the current frame / slice is an I-frame / slice, blocks included in the current frame / slice can be reconstructed based solely on intra-frame prediction. Furthermore, when the current frame / slice is a P-frame or B-frame / slice, blocks included in the current frame / slice can be reconstructed based on either intra-frame prediction or inter-frame prediction. In this case, intra-frame prediction can be applied to some blocks in the current frame / slice, and inter-frame prediction can be applied to the remaining blocks.

[0112] Intra-frame prediction can represent a prediction of the current block's predicted samples based on reference samples in the frame to which the current block belongs (hereinafter, the current frame). When applying intra-frame prediction to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. The neighboring reference samples of the current block may include samples adjacent to the left boundary of the current block (size nW×nH), a total of 2×nH samples adjacent to the lower left, samples adjacent to the top boundary of the current block, a total of 2×nW samples adjacent to the upper right, and one sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighbor samples and multiple rows of left neighbor samples. Alternatively, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block (size nW×nH), a total of nH samples adjacent to the right boundary of the current block, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right of the current block.

[0113] However, some neighboring reference samples of the current block may not yet be decoded or may be unavailable. In this case, the decoder can configure the neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be configured by interpolation of available samples.

[0114] When deriving neighboring reference samples, there are two cases: (i) the predicted sample can be derived based on the average or interpolation of the neighboring reference samples of the current block, and (ii) the predicted sample can be derived based on reference samples among the neighboring reference samples of the current block that exist in a specific (prediction) direction of the predicted sample. Case (i) can be referred to as non-directional mode or non-angular mode, and case (ii) can be referred to as directional mode or angular mode. Alternatively, a predicted sample can be generated from the first and second neighboring samples among the neighboring reference samples, based on the predicted sample of the current block located in a direction opposite to the prediction direction of the intra-prediction mode of the current block. This case can be referred to as Linear Interpolation Intra-Prediction (LIP). Additionally, a chroma prediction sample can be generated based on a linear model using luminance samples. This case can be referred to as LM mode. Furthermore, a temporal prediction sample for the current block can be derived based on filtered neighboring reference samples. At least one reference sample derived according to the intra-prediction mode from existing neighboring reference samples (i.e., unfiltered neighboring reference samples) and a temporal prediction sample can be weighted and summed to derive the predicted sample for the current block. This case can be referred to as Position Dependent Intra-Prediction (PDPC). Alternatively, the reference sample line with the highest prediction accuracy among the neighboring reference sample lines of the current block can be selected to derive the predicted sample using reference samples on the corresponding line in the prediction direction. The reference sample line used in this paper can be indicated (by signaling) to the decoding device to perform intra-frame prediction coding. This situation can be referred to as multi-reference line (MRL) intra-frame prediction or MRL-based intra-frame prediction. Additionally, intra-frame prediction can be performed based on the same intra-frame prediction mode by dividing the current block into vertical or horizontal sub-partitions, and neighboring reference samples can be derived and used on a sub-partition basis. That is, in this case, the intra-frame prediction mode of the current block is applied equivalently to the sub-partitions, and in some cases, intra-frame prediction performance can be improved by deriving and using neighboring reference samples on a sub-partition basis. This prediction method can be referred to as intra-fractional (ISP) or ISP-based intra-frame prediction. The above intra-frame prediction methods can be referred to as intra-frame prediction types different from the intra-frame prediction modes in Section 1.2. Intra-frame prediction types can be referred to by various terms such as intra-frame prediction techniques or additional intra-frame prediction modes. For example, intra-prediction types (or additional intra-prediction modes, etc.) may include at least one of LIP, PDPC, MRL, and ISP as described above. General intra-prediction methods other than specific intra-prediction types such as LIP, PDPC, MRL, or ISP may be referred to as normal intra-prediction types. When a specific intra-prediction type is not applied, normal intra-prediction types are typically applied, and prediction can be performed based on the intra-prediction modes described above. Furthermore, optionally, post-processing filtering may be performed on the derived prediction samples.

[0115] Specifically, intra-frame prediction processing may include operations to determine the intra-frame prediction mode / type, operations to derive neighboring reference samples, and operations to derive predicted samples based on the intra-frame prediction mode / type. Optionally, post-processing filtering operations may be performed on the derived predicted samples.

[0116] The modified reconstructed frame can be generated through in-loop filtering, and this modified reconstructed frame can be output as a decoded frame in the decoding device, or stored in the decoded frame buffer or memory of the encoding / decoding device and used as a reference frame in inter-frame prediction processing at a later time when the frame is encoded / decoded. In-loop filtering may include deblocking filtering, Sample Adaptive Offset (SAO) processing, and / or Adaptive Loop Filter (ALF) processing as described above. In this case, one or more of the deblocking filtering, SAO, ALF, and bilateral filtering can be applied sequentially, or all of them can be applied sequentially. For example, SAO processing can be performed after deblocking filtering is applied to the reconstructed frame. Alternatively, for example, ALF processing can be performed after deblocking filtering is applied to the reconstructed frame. This can also be performed in the encoding device.

[0117] Deblocking filtering is a filtering technique that removes distortion at the boundaries between blocks in a reconstructed image. For example, deblocking filtering can derive the target boundary in the reconstructed image, determine the boundary strength (bS) of the target boundary, and perform deblocking filtering on the target boundary based on bS. bS can be determined based on the prediction modes of the two blocks adjacent to the target boundary, the difference in motion vectors, whether the reference image is the same, and whether there are non-zero effective coefficients, etc.

[0118] SAO (Sampling Offset Optimization) is a method based on samples to compensate for the offset difference between the reconstructed image and the original image. For example, SAO can be applied based on types such as band offset or edge offset. According to SAO, samples can be classified into different categories based on each SAO type, and the offset value can be added to each sample based on the category. SAO filtering information can include information about whether SAO is applied, SAO type information, SAO offset value information, etc. SAO can be applied to the reconstructed image after deblocking filtering has been applied.

[0119] ALF (Advanced Filtering Lamp) is a technique that filters reconstructed images based on filter shapes, filter coefficients, and samples. The encoding device can determine whether to apply ALF, the ALF shape, and / or ALF filter coefficients by comparing the reconstructed image with the original image, and can signal the determination result to the decoding device. That is, ALF filtering information can include information about whether ALF is applied, ALF filter shape information, ALF filter coefficient information, etc. ALF can be applied to the reconstructed image after deblocking filtering has been applied.

[0120] Figure 5 This is a flowchart illustrating a filtering-based encoding method in an encoding device. Figure 5 The method may include steps S500 to S530.

[0121] In step S500, the encoding device can generate a reconstructed image. Step S500 can be performed based on the above-mentioned reconstructed image (or reconstructed sample) generation process.

[0122] In step S510, the encoding device may determine whether to apply in-loop filtering (across virtual boundaries) based on in-loop filtering-related information. In this document, in-loop filtering may include at least one of the above-described deblocking filtering, SAO, and ALF.

[0123] In step S520, the encoding device may generate a modified reconstructed image (modified reconstructed sample) based on the determination in step S510. In this document, the modified reconstructed image (modified reconstructed sample) may be a filtered reconstructed image (filtered reconstructed sample).

[0124] In step S530, the encoding device can encode image / video information including information related to in-loop filtering based on in-loop filtering processing.

[0125] Figure 6 This is a flowchart illustrating a filtering-based decoding method in a decoding device. Figure 6 The method may include steps S600 to S630.

[0126] In step S600, the decoding device can obtain image / video information, including in-loop filtering information, from the bitstream. In this document, the bitstream may be based on encoded image / video information sent from the encoding device.

[0127] In step S610, the decoding device can generate a reconstructed image. Step S610 can be performed based on the reconstructed image (or reconstructed sample) described above.

[0128] In step S620, the decoding device may determine whether to apply in-loop filtering (across virtual boundaries) based on in-loop filtering-related information. In this document, in-loop filtering may include at least one of the above-described deblocking filtering, SAO, and ALF.

[0129] In step S630, the decoding device may generate a modified reconstructed image (modified reconstructed sample) based on the determination in step S620. In this document, the modified reconstructed image (modified reconstructed sample) may be a filtered reconstructed image (filtered reconstructed sample).

[0130] As described above, in-loop filtering can be applied to the reconstructed image. In this case, virtual boundaries can be defined to further improve the subjective / objective visual quality of the reconstructed image, and in-loop filtering can be applied across the virtual boundaries. For example, virtual boundaries can include discontinuous edges such as 360-degree images, VR images, boundaries, picture-in-picture (PIP), etc. For example, virtual boundaries can exist at predetermined locations, and their presence and / or location can be signaled. For example, a virtual boundary can be located on the fourth sample line above the CTU line (specifically, for example, above the fourth sample line above the CTU line). As another example, information about the presence and / or location of virtual boundaries can be signaled via HLS. HLS can include SPS, PPS, image header, slice header, etc., as described above.

[0131] Hereinafter, advanced syntactic signaling and semantics will be described according to embodiments of this disclosure.

[0132] The embodiments described in this document may include methods for controlling loop filters. This method for controlling loop filters can be applied to reconstruct the image. In-loop filters (loop filters) can be used for decoding the encoded bit rate. Loop filters may include the aforementioned deblocking, SAO, and ALF. The SPS may include flags associated with each of deblocking, SAO, and ALF. These flags may indicate whether the respective tools are available for encoding a Coding Layer Video Sequence (CLVS) or Coding Video Sequence (CVS) with reference to the SPS.

[0133] When a loop filter is available for CVS, its application can be controlled to avoid crossing specific boundaries. For example, it can be controlled whether the loop filter crosses sub-picture boundaries. Additionally, it can be controlled whether the loop filter crosses tile boundaries. Furthermore, it can be controlled whether the loop filter crosses virtual boundaries. In this paper, virtual boundaries can be defined on the CTU based on the availability of line buffers.

[0134] Regarding whether to perform in-loop filtering across virtual boundaries, in-loop filtering related information may include at least one of the following: SPS virtual boundary enable flag (virtual boundary enable flag in SPS), SPS virtual boundary presence flag, frame header virtual boundary presence flag, SPS frame header virtual boundary presence flag, and information about the location of the virtual boundary.

[0135] In the embodiments included in this document, information regarding the virtual boundary position may include information about the x-coordinate of the vertical virtual boundary and / or information about the y-coordinate of the horizontal virtual boundary. Specifically, information regarding the virtual boundary position may include information about the x-coordinate of the vertical virtual boundary and / or information about the y-axis of the horizontal virtual boundary in units of luminance samples. Additionally, information regarding the virtual boundary position may include the number of information (syntactic elements) regarding the x-coordinate of the vertical virtual boundary present in the SPS. Alternatively, information regarding the virtual boundary position may include the number of information (syntactic elements) regarding the x-coordinate of the vertical virtual boundary present in the header. Additionally, information regarding the virtual boundary position may include the number of information (syntactic elements) regarding the y-coordinate of the horizontal virtual boundary present in the header.

[0136] The following illustrates exemplary syntax and semantics of the SPS according to this embodiment.

[0137] [Table 1]

[0138]

[0139] [Table 2]

[0140]

[0141]

[0142] The following illustrates exemplary syntax and semantics of the Picture Parameter Set (PPS) according to this embodiment.

[0143] [Table 3]

[0144]

[0145] [Table 4]

[0146]

[0147] The following illustrates exemplary syntax and semantics of the header according to this embodiment.

[0148] [Table 5]

[0149]

[0150]

[0151] [Table 6]

[0152]

[0153]

[0154] The following illustrates exemplary syntax and semantics of the slice header according to this embodiment.

[0155] [Table 7]

[0156]

[0157] [Table 8]

[0158]

[0159]

[0160] The following describes the signaling information regarding the virtual boundaries that can be used in in-loop filtering.

[0161] In the existing design, there are two options to disable the loop filter across virtual boundaries: Option i) where the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) can be set to 0, and for each frame header, the PH virtual boundary presence flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag) can be present and set to 0; and Option ii) where the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) can be set to 1, and information regarding the number of SPS vertical virtual boundaries (sps_num_ver_vertical_boudnaries) and the number of SPS horizontal virtual boundaries (sps_num_hor_vertical_boudnaries) can be set to 0.

[0162] In the existing design, according to option ii), the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is set to 1, so the decoder expects signaling about the location of the virtual boundary, which may cause problems in the decoding process.

[0163] The embodiments described below offer solutions to the aforementioned problems. These embodiments can be applied independently. Alternatively, at least two embodiments can be applied in combination.

[0164] In the embodiments of this document, whether the SPS includes a syntactic element for indicating virtual boundaries can be controlled by flags. For example, the number of flags can be two (e.g., SPS virtual boundary enable flag, SPS virtual boundary exist flag).

[0165] In the example according to this embodiment, the SPS virtual boundary enable flag may be referred to as sps_loop_filter_across_virtual_boundaries_disabled_flag (or sps_virtual_boundaries_enabled_flag). The SPS virtual boundary enable flag can indicate whether the feature for disabling loop filters across virtual boundaries is enabled.

[0166] In the example according to this embodiment, the SPS virtual boundary presence flag may be referred to as sps_loop_filter_across_virtual_boundaries_disabled_present_flag (or sps_virtual_boundaries_present_flag). The SPS virtual boundary presence flag can indicate whether the signaling information used for the virtual boundary is included in the SPS or the picture header (PH).

[0167] In an example according to this embodiment, when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 0, signaling information for disabling loop filters across virtual boundaries can be included in the PH.

[0168] In an example according to this embodiment, when information about the location of virtual boundaries (e.g., vertical virtual boundaries, horizontal virtual boundaries) is included in the SPS, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries can be constrained to be greater than 0.

[0169] In the example according to this embodiment, a variable indicating whether the filter is disabled at the virtual boundaries of the current screen can be derived. For example, the variable may include VirtualBoundariesDisabledFlag.

[0170] As one example in this case, VirtualBoundariesDisabledFlag can be 1 when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and the SPS virtual boundary present flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 1.

[0171] As another case in this example, when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1, the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 0, and the sum of information about the number of vertical virtual boundaries (e.g., ph_num_ver_virtual_boundaries) and information about the number of horizontal virtual boundaries (e.g., ph_num_hor_virtual_boundaries) is greater than 0, the VirtualBoundariesDisabledFlag can be 1.

[0172] In other cases in this example, VirtualBoundariesDisabledFlag can be 0.

[0173] The following illustrates an exemplary syntax of the SPS according to this embodiment.

[0174] [Table 9]

[0175]

[0176] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0177] [Table 10]

[0178]

[0179] The following illustrates exemplary syntax for header information (screen header) according to this embodiment.

[0180] [Table 11]

[0181]

[0182] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0183] [Table 12]

[0184]

[0185] In the embodiments relating to Tables 9 to 12, the image information obtained by the encoding device and / or the image information obtained by receiving a bitstream from the encoding device to the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a Virtual Boundary Enable Flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the Virtual Boundary Enable Flag, the SPS may include an SPS Virtual Boundary Present Flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag).

[0186] For example, when the virtual boundary enable flag is 1, SPS may include an SPS virtual boundary presence flag. Based on the virtual boundary enable flag and the SPS virtual boundary presence flag, SPS may include information about the number of SPS vertical virtual boundaries (sps_num_ver_virtual_boundaries), information about the location of SPS vertical virtual boundaries (sps_virtual_boundaries_pos_x[i]), information about the number of SPS horizontal virtual boundaries (sps_num_hor_virtual_boundaries), and information about the location of SPS horizontal virtual boundaries (sps_virtual_boundaries_pos_y[i]). For example, when both the virtual boundary enable flag and the SPS virtual boundary presence flag are 1, SPS may include information about the number of SPS vertical virtual boundaries, information about the location of SPS vertical virtual boundaries, information about the number of SPS horizontal virtual boundaries, and information about the location of SPS horizontal virtual boundaries.

[0187] In the example, the number of information entries regarding the location of the SPS vertical virtual boundary can be determined based on the information regarding the number of SPS vertical virtual boundaries, and the number of information entries regarding the location of the SPS horizontal virtual boundary can be determined based on the information regarding the number of SPS horizontal virtual boundaries. Based on the virtual boundary enable flag and the SPS virtual boundary presence flag, the header may include information regarding the number of PH vertical virtual boundaries (ph_num_ver_virtual_boundaries), information regarding the location of PH vertical virtual boundaries (ph_virtual_boundaries_pos_x[i]), information regarding the number of PH horizontal virtual boundaries (ph_num_hor_virtual_boundaries), and information regarding the location of PH horizontal virtual boundaries (ph_virtual_boundaries_pos_y[i]).

[0188] For example, when the virtual boundary enable flag is set to 1 and the SPS virtual boundary presence flag is set to 0, the header may include information about the number of vertical virtual boundaries (PH), the location of vertical virtual boundaries (PH), the number of horizontal virtual boundaries (PH), and the location of horizontal virtual boundaries (PH). In this example, the number of entries regarding the location of vertical virtual boundaries (PH) can be determined based on the information about the number of vertical virtual boundaries (PH), and the number of entries regarding the location of horizontal virtual boundaries (PH) can also be determined based on the information about the number of horizontal virtual boundaries (PH).

[0189] In another embodiment of this document, the header information (frame header) of the frame referencing SPS may include a PH virtual boundary presence flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag or ph_virtual_boundaries_present_flag). This embodiment may also be described together with the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) and the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag).

[0190] In the example according to this embodiment, when the value of the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and the value of the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 0, each piece of information (frame header) of the screen referencing SPS may include the PH virtual boundary presence flag (ph_loop_filter_across_virtual_boundaries_disalbed_present_flag or ph_virtual_boundaries_present_flag).

[0191] In an example according to this embodiment, when information about the location of virtual boundaries (e.g., vertical virtual boundaries, horizontal virtual boundaries) is included in the SPS, the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries can be constrained to be greater than 0.

[0192] In an example according to this embodiment, a variable indicating whether the filter is disabled at the virtual boundaries can be derived for the current screen. For example, the variable may include VirtualBoundariesDisabledFlag.

[0193] As one example in this case, VirtualBoundariesDisabledFlag can be 1 when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and the SPS virtual boundary present flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 1.

[0194] As another case in this example, VirtualBoundariesDisabledFlag can be 1 when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and the PH virtual boundary present flag (ph_loop_filter_across_virtual_boundaries_disabled_present_flag) is 1.

[0195] In other cases in this example, VirtualBoundariesDisabledFlag can be 0.

[0196] The following illustrates an exemplary syntax of the SPS according to this embodiment.

[0197] [Table 13]

[0198]

[0199] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0200] [Table 14]

[0201]

[0202] The following illustrates exemplary syntax for header information (screen header) according to this embodiment.

[0203] [Table 15]

[0204]

[0205] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0206] [Table 16]

[0207]

[0208]

[0209] In the embodiments relating to Tables 13 to 16, the image information obtained by the encoding device and / or the image information obtained by receiving a bitstream from the encoding device to the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a Virtual Boundary Enable Flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the Virtual Boundary Enable Flag, the SPS may include an SPS Virtual Boundary Presence Flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, when the value of the Virtual Boundary Enable Flag is 1, the SPS may include the SPS Virtual Boundary Presence Flag. Based on the virtual boundary enable flag and the SPS virtual boundary existence flag, SPS can include information about the number of SPS vertical virtual boundaries (sps_num_ver_virtual_boundaries), information about the location of SPS vertical virtual boundaries (sps_virtual_boundaries_pos_x[i]), information about the number of SPS horizontal virtual boundaries (sps_num_hor_virtual_boundaries), and information about the location of SPS horizontal virtual boundaries (sps_virtual_boundaries_pos_y[i]).

[0210] For example, when the virtual boundary enable flag is 1 and the SPS virtual boundary presence flag is 1, the SPS may include information about the number of SPS vertical virtual boundaries, information about the location of SPS vertical virtual boundaries, information about the number of SPS horizontal virtual boundaries, and information about the location of SPS horizontal virtual boundaries. In this example, the number of entries for information about the location of SPS vertical virtual boundaries can be determined based on the information about the number of SPS vertical virtual boundaries, and the number of entries for information about the location of SPS horizontal virtual boundaries can be determined based on the information about the number of SPS horizontal virtual boundaries. Based on the virtual boundary enable flag and the SPS virtual boundary presence flag, the header may include a PH virtual boundary presence flag.

[0211] For example, when the virtual boundary enable flag is 1 and the SPS virtual boundary presence flag is 0, the header may include the PH virtual boundary presence flag. Based on the PH virtual boundary presence flag, the header may include information about the number of PH vertical virtual boundaries (ph_num_ver_virtual_boundaries), information about the location of PH vertical virtual boundaries (ph_virtual_boundaries_pos_x[i]), information about the number of PH horizontal virtual boundaries (ph_num_hor_virtual_boundaries), and information about the location of PH horizontal virtual boundaries (ph_virtual_boundaries_pos_y[i]).

[0212] For example, when the value of the PH virtual boundary presence flag is 1, the header may include information about the number of PH vertical virtual boundaries, information about the location of PH vertical virtual boundaries, information about the number of PH horizontal virtual boundaries, and information about the location of PH horizontal virtual boundaries. In this example, the number of entries for the location of PH vertical virtual boundaries can be determined based on the information about the number of PH vertical virtual boundaries, and the number of entries for the location of PH horizontal virtual boundaries can be determined based on the information about the number of PH horizontal virtual boundaries.

[0213] In another embodiment of this document, whether the syntactic element used to indicate the virtual boundary is included in the SPS can be controlled by flags. For example, the number of flags can be two (e.g., SPS virtual boundary presence flag, SPS PH virtual boundary presence flag).

[0214] In the example according to this embodiment, the SPS virtual boundary presence flag may be referred to as sps_loop_filter_across_virtual_boundaries_disabled_present_flag (or sps_virtual_boundaries_present_flag). The SPS virtual boundary presence flag indicates whether virtual boundary information is included in the SPS.

[0215] In the examples according to this disclosure, the SPS PH virtual boundary presence flag may be referred to as sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag. The SPS PH virtual boundary presence flag indicates whether virtual boundary information is included in the picture header (PH).

[0216] In the example according to this embodiment, it can be further constrained that when the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 1, the SPS PH virtual boundary presence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) does not exist and is inferred to be 0.

[0217] In an example according to this embodiment, when the SPS PH virtual boundary presence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) is 1, signaling information for disabling loop filters across virtual boundaries can be included in the PH.

[0218] The following illustrates an exemplary syntax of the SPS according to this embodiment.

[0219] [Table 17]

[0220]

[0221] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0222] [Table 18]

[0223]

[0224] The following illustrates exemplary syntax for header information (screen header) according to this embodiment.

[0225] [Table 19]

[0226]

[0227] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0228] [Table 20]

[0229]

[0230]

[0231] In the embodiments relating to Tables 17 to 20, the image information obtained by the encoding device and / or the image information obtained by receiving a bitstream from the encoding device to the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include an SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). Based on the SPS virtual boundary presence flag, the SPS may include information about the number of SPS vertical virtual boundaries (sps_num_ver_virtual_boundaries), information about the location of SPS vertical virtual boundaries (sps_virtual_boundaries_pos_x[i]), information about the number of SPS horizontal virtual boundaries (sps_num_hor_virtual_boundaries), and information about the location of SPS horizontal virtual boundaries (sps_virtual_boundaries_pos_y[i]).

[0232] For example, when the SPS virtual boundary presence flag is set to 1, the SPS may include information about the number of SPS vertical virtual boundaries, information about the location of SPS vertical virtual boundaries, information about the number of SPS horizontal virtual boundaries, and information about the location of SPS horizontal virtual boundaries. In this example, the number of entries regarding the location of SPS vertical virtual boundaries can be determined based on the information about SPS vertical virtual boundaries, and the number of entries regarding the location of SPS horizontal virtual boundaries can be determined based on the number of SPS horizontal virtual boundaries. Based on the SPS virtual boundary presence flag, the SPS may include an SPS PH virtual boundary presence flag.

[0233] For example, when the value of the SPS virtual boundary presence flag is 0, the SPS may include the SPS PH virtual boundary presence flag. Based on the SPS PH virtual boundary presence flag, the frame header may include the PH virtual boundary presence flag. For example, when the value of the SPS PH virtual boundary presence flag is 1, the frame header may include the PH virtual boundary presence flag. Based on the PH virtual boundary presence flag, the frame header may include information about the number of PH vertical virtual boundaries (ph_num_ver_virtual_boundaries), information about the position of PH vertical virtual boundaries (ph_virtual_boundaries_pos_x[i]), information about the number of PH horizontal virtual boundaries (ph_num_hor_virtual_boundaries), and information about the position of PH horizontal virtual boundaries (ph_virtual_boundaries_pos_y[i]).

[0234] For example, when the value of the PH virtual boundary presence flag is 1, the header may include information about the number of PH vertical virtual boundaries, information about the location of PH vertical virtual boundaries, information about the number of PH horizontal virtual boundaries, and information about the location of PH horizontal virtual boundaries. In this example, the number of entries for the location of PH vertical virtual boundaries can be determined based on the information about the number of PH vertical virtual boundaries, and the number of entries for the location of PH horizontal virtual boundaries can also be determined based on the information about the number of PH horizontal virtual boundaries.

[0235] In another embodiment of this document, when Progressive Decode Refresh (GDR) is available (i.e., the value of gdr_enabled_flag is 1), the feature of disabling the loop filter at the virtual boundary is enabled, and the virtual boundary information can be signaled in the frame header (which may include the frame header).

[0236] In another embodiment of this document, when the function to disable loop filters across virtual boundaries is enabled, signaling information about the location of the virtual boundary may be included in one or more parameter sets. For example, when the function to disable loop filters across virtual boundaries is enabled, information about the location of the virtual boundary may be included in the SPS and the header.

[0237] In this implementation, the following can be applied when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1 and signaling information about the location of the virtual boundary is included in one or more parameter sets.

[0238] a) Signaling information about the location of the virtual boundary may be included only in the SPS, only in the header, or in both the SPS and the header.

[0239] b) The VirtualBoundariesDisabledFlag for each screen can be derived as follows.

[0240] - When sps_loop_filter_across_virtual_boundaries_disabled_flag is 0, VirtualBoundariesDisabledFlag can be set to 0.

[0241] - In another case of this example, when neither the SPS associated with the screen nor the screen header signals information about the location of the virtual boundary, VirtualBoundariesDisabledFlag can be set to 0.

[0242] - In other cases in this example (when the location of the virtual boundary is signaled only in the SPS, only in the header, or in both the SPS and the header), VirtualBoundariesDisabledFlag can be set to 1.

[0243] c) Virtual boundaries applied to a frame may include the union of virtual boundaries signaled by a set of parameters directly or indirectly referenced by the frame. For example, virtual boundaries may include virtual boundaries signaled by a SPS (if present). For example, virtual boundaries may include virtual boundaries signaled by a frame associated with the frame (if present).

[0244] d) Constraints can be applied to ensure that the maximum number of virtual boundaries per frame does not exceed a predefined value. For example, the predefined value could be 8.

[0245] e) It may be further constrained that the information about the location of the virtual boundary (if present) signaled in the header should not be consistent with the information about the location of the virtual boundary included in another parameter set (e.g., SPS or PPS).

[0246] - Alternatively, for any virtual boundary location applied to the current frame, the virtual boundary location (e.g., the same virtual boundary location signaled by a signal in the SPS associated with the frame and the frame header) may be included in two different parameter sets.

[0247] f) Further constraints can be imposed: when the SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag) is 1, the SPS PH virtual boundary presence flag (sps_ph_loop_filter_across_virtual_boundaries_disabled_present_flag) does not exist and is inferred to be 0.

[0248] The following illustrates an exemplary syntax of the SPS according to this embodiment.

[0249] [Table 21]

[0250]

[0251] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0252] [Table 22]

[0253]

[0254] The following illustrates exemplary syntax for header information (screen header) according to this embodiment.

[0255] [Table 23]

[0256]

[0257] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0258] [Table 24]

[0259]

[0260]

[0261] In the embodiments relating to Tables 21 to 24, the image information obtained by the encoding device and / or the image information obtained by receiving a bitstream from the encoding device to the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a Virtual Boundary Enable Flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the Virtual Boundary Enable Flag, the SPS may include an SPS Virtual Boundary Presence Flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, when the value of the Virtual Boundary Enable Flag is 1, the SPS may include the SPS Virtual Boundary Presence Flag. Based on the virtual boundary enable flag and the SPS virtual boundary existence flag, SPS can include information about the number of SPS vertical virtual boundaries (sps_num_ver_virtual_boundaries), information about the location of SPS vertical virtual boundaries (sps_virtual_boundaries_pos_x[i]), information about the number of SPS horizontal virtual boundaries (sps_num_hor_virtual_boundaries), and information about the location of SPS horizontal virtual boundaries (sps_virtual_boundaries_pos_y[i]).

[0262] For example, when the virtual boundary enable flag is 1 and the SPS virtual boundary presence flag is 1, the SPS may include information about the number of SPS vertical virtual boundaries, information about the location of SPS vertical virtual boundaries, information about the number of SPS horizontal virtual boundaries, and information about the location of SPS horizontal virtual boundaries. In this example, the number of entries regarding the location of SPS vertical virtual boundaries can be determined based on the information about the number of SPS vertical virtual boundaries, and the number of entries regarding the location of SPS horizontal virtual boundaries can be determined based on the information about the number of SPS horizontal virtual boundaries. Based on the virtual boundary enable flag, the header may include a PH virtual boundary presence flag.

[0263] For example, when the virtual boundary enable flag is 1, the header may include a PH virtual boundary presence flag. Based on the PH virtual boundary presence flag, the header may include information about the number of PH vertical virtual boundaries (ph_num_ver_virtual_boundaries), information about the location of PH vertical virtual boundaries (ph_virtual_boundaries_pos_x[i]), information about the number of PH horizontal virtual boundaries (ph_num_hor_virtual_boundaries), and information about the location of PH horizontal virtual boundaries (ph_virtual_boundaries_pos_y[i]). For example, when the PH virtual boundary presence flag is 1, the header may include information about the number of PH vertical virtual boundaries, information about the location of PH vertical virtual boundaries, information about the number of PH horizontal virtual boundaries, and information about the location of PH horizontal virtual boundaries. In the example, the number of entries for information about the location of PH vertical virtual boundaries can be determined based on the information about the number of PH vertical virtual boundaries, and the number of entries for information about the location of PH horizontal virtual boundaries can be determined based on the information about the number of PH horizontal virtual boundaries.

[0264] In another embodiment of this document, loop filtering can be performed by limiting the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries to no greater than 0, according to the above embodiments.

[0265] In another embodiment of this document, information about virtual boundaries can be signaled in both the SPS and PH. In an example of this embodiment, when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1, information about the number of vertical virtual boundaries, the number of horizontal virtual boundaries, and / or the virtual boundary positions can be included in the SPS. Furthermore, when the SPS virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag) is 1, information about the number of vertical virtual boundaries, the number of horizontal virtual boundaries, and / or the virtual boundary position increment values ​​(incremental values ​​of virtual boundary positions) can be included in the header. The virtual boundary position increment value can refer to the difference between the positions of the virtual boundaries. Information about the sign of the virtual boundary positions can also be included in the header.

[0266] According to the example of this embodiment, in order to derive the virtual boundary position for each frame, if the incremental value of the virtual boundary position does not exist in the frame header, the information about the virtual boundary position notified by the signal in the SPS can be used for loop filtering. If the incremental value of the virtual boundary position exists in the frame header, the virtual boundary position can be derived based on the sum of the information about the virtual boundary position notified by the signal in the SPS and the associated incremental value.

[0267] The following illustrates an exemplary syntax of the SPS according to this embodiment.

[0268] [Table 25]

[0269]

[0270] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0271] [Table 26]

[0272]

[0273] The following illustrates exemplary syntax for header information (screen header) according to this embodiment.

[0274] [Table 27]

[0275]

[0276] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0277] [Table 28]

[0278]

[0279]

[0280] In the embodiments relating to Tables 25 to 28, the image information obtained by the encoding device and / or the image information obtained by receiving a bitstream from the encoding device to the decoding device may include a Sequence Parameter Set (SPS) and a Picture Header (PH). The SPS may include a Virtual Boundary Enable Flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the Virtual Boundary Enable Flag, the SPS may include information about the number of SPS vertical virtual boundaries (sps_num_ver_virtual_boundaries), information about the location of SPS vertical virtual boundaries (sps_virtual_boundaries_pos_x[i]), information about the number of SPS horizontal virtual boundaries (sps_num_hor_virtual_boundaries), and information about the location of SPS horizontal virtual boundaries (sps_virtual_boundaries_pos_y[i]). For example, when the Virtual Boundary Enable Flag is 1, the SPS may include information about the number of SPS horizontal virtual boundaries, information about the location of SPS horizontal virtual boundaries, information about the number of SPS vertical virtual boundaries, and information about the location of SPS vertical virtual boundaries.

[0281] In the example, the number of information entries regarding the location of the SPS horizontal virtual boundary can be determined based on the information regarding the number of SPS horizontal virtual boundaries, and the number of information entries regarding the location of the SPS vertical virtual boundary can be determined based on the information regarding the number of SPS vertical virtual boundaries. Based on the virtual boundary enable flag, the header may include the PH virtual boundary presence flag. For example, when the virtual boundary enable flag is 1, the header may include the PH virtual boundary presence flag. Based on the PH virtual boundary presence flag, the header may include information regarding the incremental value of the PH horizontal virtual boundary location (ph_virtual_boundaries_pos_x_delta[i]), the sign of the PH horizontal virtual boundary location (ph_virtual_boundaries_pos_x_sign[i]), the incremental value of the PH vertical virtual boundary location (ph_virtual_boundaries_pos_y_delta[i]), and the sign of the PH vertical virtual boundary location (ph_virtual_boundaries_pos_y_sign[i]).

[0282] For example, when the value of the PH virtual boundary presence flag is 1, the header may include information about the PH vertical virtual boundary position increment, information about the PH vertical virtual boundary position symbol, information about the PH horizontal virtual boundary position increment, and information about the PH horizontal virtual boundary position symbol. In this example, the number of entries for the PH vertical virtual boundary position increment and the number of entries for the PH vertical virtual boundary position symbol can be determined based on the number of SPS vertical virtual boundaries, and the number of entries for the PH horizontal virtual boundary position increment and the number of entries for the PH horizontal virtual boundary position symbol can be determined based on the number of SPS horizontal virtual boundaries.

[0283] In another embodiment of this document, signaling regarding information about the virtual boundary positions of each frame will be described. In the example, when information about the virtual boundary position is included in the SPS and information about the virtual boundary position increment value is not included in the frame header, the virtual boundary information included in the SPS can be used for loop filtering. When information about the virtual boundary position is not included in the SPS and information about the virtual boundary position increment value is included in the frame header, the virtual boundary information included in the frame header can be used for loop filtering. When information about the virtual boundary position is included in the SPS and information about the virtual boundary position increment value is included in the frame header, the virtual boundary position can be derived based on the sum of the information about the virtual boundary position signaled in the SPS and the associated increment value. When information about the virtual boundary position is not included in the SPS and information about the virtual boundary position increment value is not included in the frame header, virtual boundaries may not be applied to the frame.

[0284] The following illustrates an exemplary syntax of the SPS according to this embodiment.

[0285] [Table 29]

[0286]

[0287] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0288] [Table 30]

[0289]

[0290] The following illustrates exemplary syntax for header information (screen header) according to this embodiment.

[0291] [Table 31]

[0292]

[0293] The following illustrates exemplary semantics of syntactic elements included in the syntax.

[0294] [Table 32]

[0295]

[0296]

[0297]

[0298] In the embodiments relating to Tables 29 to 32, the image information obtained by the encoding device and / or the image information obtained by receiving a bitstream from the encoding device to the decoding device may include a sequence parameter set (SPS) and a picture header (PH).

[0299] The SPS may include a virtual boundary enable flag (sps_loop_filter_across_virtual_boundaries_disabled_flag). Based on the virtual boundary enable flag, the SPS may include an SPS virtual boundary presence flag (sps_loop_filter_across_virtual_boundaries_disabled_present_flag). For example, when the virtual boundary enable flag is 1, the SPS may include an SPS virtual boundary presence flag. Based on the virtual boundary enable flag and the SPS virtual boundary presence flag, the SPS may include information about the number of SPS vertical virtual boundaries (sps_num_ver_virtual_boundaries), information about the location of SPS vertical virtual boundaries (sps_virtual_boundaries_pos_x[i]), information about the number of SPS horizontal virtual boundaries (sps_num_hor_virtual_boundaries), and information about the location of SPS horizontal virtual boundaries (sps_virtual_boundaries_pos_y[i]).

[0300] For example, when the virtual boundary enable flag is set to 1 and the SPS virtual boundary presence flag is set to 1, the SPS may include information about the number of horizontal virtual boundaries, information about the location of horizontal virtual boundaries, information about the number of vertical virtual boundaries, and information about the location of vertical virtual boundaries. In this example, the number of entries regarding the location of horizontal virtual boundaries may be determined based on the information about the number of horizontal virtual boundaries, and the number of entries regarding the location of vertical virtual boundaries may be determined based on the information about the number of vertical virtual boundaries. Based on the virtual boundary enable flag, the header may include a PH virtual boundary presence flag.

[0301] For example, when the virtual boundary enable flag is 1, the header may include a PH virtual boundary presence flag. Based on the PH virtual boundary presence flag and information about the number of SPS vertical virtual boundaries, the header may include information about the number of PH vertical virtual boundaries (ph_num_ver_virtual_boundaries). For example, when the PH virtual boundary presence flag is 1 and the information about the number of SPS vertical virtual boundaries is 0, the header may include information about the number of PH vertical virtual boundaries. In the example, based on the information about the number of PH vertical virtual boundaries, the header may include information about the incremental value of the PH vertical virtual boundary position (ph_virtual_boundaries_pos_x_delta[i]) and information about the sign of the PH vertical virtual boundary position (ph_virtual_boundaries_pos_x_sign[i]). In the example, based on the information about the number of PH vertical virtual boundaries, the number of entries for the incremental value of the PH vertical virtual boundary position and the number of entries for the sign of the PH vertical virtual boundary position can be determined. Based on the PH virtual boundary presence flag and information about the number of SPS horizontal virtual boundaries, the header may include information about the number of PH horizontal virtual boundaries (ph_num_hor_virtual_boundaries).

[0302] For example, when the value of the PH virtual boundary presence flag is 1 and the value of the information regarding the number of SPS horizontal virtual boundaries is 0, the header may include information regarding the number of PH horizontal virtual boundaries. In this example, based on the information regarding the number of PH horizontal virtual boundaries, the header may include information about the incremental values ​​of the PH horizontal virtual boundary positions (ph_virtual_boundaries_pos_y_delta[i]) and information about the signs of the PH horizontal virtual boundary positions (ph_virtual_boundaries_pos_y_sign[i]). In this example, based on the information regarding the number of PH horizontal virtual boundaries, the number of entries for the incremental values ​​of the PH horizontal virtual boundary positions and the number of entries for the signs of the PH horizontal virtual boundary positions can be determined.

[0303] According to the implementation methods in this document and the table above, information for performing in-loop filtering across virtual boundaries can be effectively communicated using signals. For example, in-loop filtering can be performed based on signaling related to whether in-loop filtering is permitted across virtual boundaries.

[0304] Figure 7 and Figure 8 Examples of video / image coding methods and related components according to embodiments of this document are illustrated schematically.

[0305] Figure 7 The method disclosed in the article can be derived from Figure 2 or Figure 8 The encoding device disclosed in the document executes the code. Specifically, for example, Figure 7 The S700 and S710 can be supplied by Figure 8 The residual processor 230 of the encoding device executes, Figure 7 The S720 can be supplied by Figure 8 The filter 260 of the encoding device is executed. Figure 7 The S730 can be supplied by Figure 8 The entropy encoder 240 of the encoding device performs this. Additionally, although... Figure 7 Not shown in the image, the predicted sample or prediction-related information can be obtained from... Figure 7 The predictor 220 of the encoding device is derived, and the bit stream can be generated from residual information or prediction-related information by the entropy encoder 240 of the encoding device. Figure 7 The methods disclosed herein may include the embodiments described above.

[0306] Reference Figure 7 The encoding device can derive residual samples (S700). The encoding device can derive residual samples for the current block, and the residual samples for the current block can be derived based on the original samples and predicted samples of the current block. Specifically, the encoding device can derive predicted samples for the current block based on the prediction mode. In this case, various prediction methods disclosed in this document (e.g., inter-frame prediction or intra-frame prediction) can be applied. Residual samples can be derived based on predicted samples and original samples.

[0307] The encoding device can derive the transform coefficients. The encoding device can derive the transform coefficients based on a transform processing of the residual samples. For example, the transform processing can include at least one of the discrete cosine transform (DCT), discrete sine transform (DST), graph-based transform (GBT), and conditional nonlinear transform (CNT).

[0308] The encoding device can derive the quantized transform coefficients. The encoding device can derive the quantized transform coefficients based on the quantization process of the transform coefficients. Based on the coefficient scan order, the quantized transform coefficients can have a one-dimensional vector form.

[0309] The encoding device can generate residual information (S710). The encoding device can generate residual information based on the transform coefficients. The encoding device can generate residual information indicating the quantized transform coefficients. The residual information can be generated using various encoding methods such as exponential Golomb, CAVLC, CABAC, etc.

[0310] The encoding device can generate reconstructed samples. The encoding device can generate reconstructed samples based on residual information. Reconstructed samples can be generated by adding predicted samples and residual samples based on residual information. Specifically, the encoding device can perform prediction (intra-frame or inter-frame prediction) on the current block and can generate reconstructed samples based on the original samples and the predicted samples generated from the prediction.

[0311] The reconstructed samples may include reconstructed luminance samples and reconstructed chrominance samples. Specifically, the residual samples may include residual luminance samples and residual chrominance samples. The residual luminance samples can be generated based on the original luminance samples and the predicted luminance samples. The residual chrominance samples can be generated based on the original chrominance samples and the predicted chrominance samples. The encoding device can derive the transform coefficients (luminance transform coefficients) of the residual luminance samples and / or the transform coefficients (chrominance transform coefficients) of the residual chrominance samples. The quantized transform coefficients may include quantized luminance transform coefficients and / or quantized chrominance transform coefficients.

[0312] The encoding device can generate information related to in-loop filtering of the reconstructed sample (S720). The encoding device can perform in-loop filtering on the reconstructed sample and can generate information related to in-loop filtering based on the in-loop filtering process. For example, the information related to in-loop filtering may include the aforementioned information about virtual boundaries (SPS virtual boundary enable flag, frame header virtual boundary enable flag, SPS virtual boundary existence flag, frame header virtual boundary existence flag, information about the position of the virtual boundary, etc.).

[0313] The encoding device can encode video / image information (S730). The image information may include residual information, prediction-related information, and / or in-loop filtering-related information. The encoded video / image information can be output as a bitstream. The bitstream can be transmitted to the decoding device via a network or storage medium.

[0314] Image / video information may include various information according to embodiments of this document. For example, images / videos may include information disclosed in at least one of Tables 1 to 32 above.

[0315] In this implementation, in-loop filtering information may include an SPS virtual boundary enable flag related to whether in-loop filtering is performed across virtual boundaries. For example, signaling in the SPS or header information regarding the presence of information related to virtual boundaries can be determined based on the SPS virtual boundary enable flag. The SPS virtual boundary enable flag can indicate whether in-loop filtering can be disabled across virtual boundaries.

[0316] In this implementation, the image information may include a Sequence Parameter Set (SPS). The SPS may include an SPS virtual boundary enable flag and an SPS virtual boundary presence flag. Furthermore, whether information regarding the location of virtual boundaries and the number of virtual boundaries is included in the SPS can be determined based on the SPS virtual boundary presence flag.

[0317] In this implementation, based on the value of the SPS virtual boundary presence flag being 1, the SPS may include information about the number of vertical virtual boundaries.

[0318] In an implementation, the SPS may include information about the location of the vertical virtual boundaries. Furthermore, the number of entries regarding the location of the vertical virtual boundaries may be determined based on information about the number of vertical virtual boundaries.

[0319] In this implementation, based on the value of the SPS virtual boundary presence flag being 1, the SPS may include information about the number of horizontal virtual boundaries.

[0320] In an implementation, the SPS may include information about the location of the horizontal virtual boundaries. Furthermore, the number of entries regarding the location of the horizontal virtual boundaries may be determined based on information about the number of horizontal virtual boundaries.

[0321] In this implementation, the image information includes header information. Additionally, based on the SPS virtual boundary enable flag being 1 and the SPS virtual boundary presence flag being 0, the header information may include the header virtual boundary presence flag.

[0322] In this implementation, based on the value of the virtual boundary presence flag being 1, the header information may include information about the number of vertical virtual boundaries.

[0323] In this implementation, the header information may include information about the location of the vertical virtual boundary. Furthermore, the number of entries regarding the location of the vertical virtual boundary may be determined based on information about the number of vertical virtual boundaries.

[0324] In this implementation, based on the value of the virtual boundary presence flag being 1, the header information may include information about the number of horizontal virtual boundaries.

[0325] In this implementation, the header information may include information about the location of the horizontal virtual boundary. Furthermore, the number of entries regarding the location of the horizontal virtual boundary may be determined based on information about the number of horizontal virtual boundaries.

[0326] In the implementation, the SPS includes information about the location of vertical virtual boundaries and information about the location of horizontal virtual boundaries, and the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries can be greater than 0.

[0327] In implementations, information related to in-loop filtering (and / or information related to virtual boundaries) may further include an SPS virtual boundary presence flag, a frame header virtual boundary presence flag, and a Progressive Decoding Refresh (GDR) enable flag. For example, based on a GDR enable flag value of 1, the SPS virtual boundary enable flag (virtual boundary enable flag) value may be 1, the SPS virtual boundary presence flag value may be 0, and the frame header virtual boundary presence flag value may be 1 (signaling of virtual boundary information may exist in the frame header).

[0328] Figure 9 and Figure 10 Examples of video / image decoding methods and related components according to embodiments of this document are illustrated schematically.

[0329] Figure 9 The method disclosed in the article can be derived from Figure 3 or Figure 10 The decoding device disclosed in the document performs the operation. Specifically, for example, Figure 9 S900 can be executed by the entropy decoder 310 of the decoding device, S910 can be executed by the residual processor 320 and / or adder 340 of the decoding device, and S920 can be executed by the filter 350 of the decoding device. Figure 9 The methods disclosed herein may include the embodiments described above.

[0330] Reference Figure 9 The decoding device can receive / acquire video / image information (S900). The video / image information may include residual information, prediction-related information, and / or in-loop filtering-related information. The decoding device can receive / acquire image / video information via a bitstream.

[0331] Image / video information may include various information according to embodiments of this document. For example, images / videos may include information disclosed in at least one of Tables 1 to 32 above.

[0332] The decoding device can derive quantized transform coefficients. The decoding device can derive quantized transform coefficients based on residual information. Based on the coefficient scanning order, the quantized transform coefficients can have a one-dimensional vector form. The quantized transform coefficients may include quantized luminance transform coefficients and / or quantized chrominance transform coefficients.

[0333] The decoding device can derive the transform coefficients. It can derive the transform coefficients based on the dequantization process of the quantized transform coefficients. It can derive the luminance transform coefficients by dequantizing the quantized luminance transform coefficients. It can derive the chrominance transform coefficients by dequantizing the quantized chrominance transform coefficients.

[0334] The decoding device can generate / derive residual samples. The decoding device can derive residual samples based on the inverse transform of the transform coefficients. The decoding device can derive residual luminance samples based on the luminance transform coefficients through inverse transform. The decoding device can derive residual chrominance samples based on the chrominance transform coefficients through inverse transform.

[0335] The decoding device can generate / derive reconstructed samples (S910). For example, the decoding device can generate / derive reconstructed luminance samples and / or reconstructed chrominance samples. The decoding device can generate reconstructed luminance samples and / or reconstructed chrominance samples based on residual information. The decoding device can generate reconstructed samples based on residual information. The reconstructed samples may include reconstructed luminance samples and / or reconstructed chrominance samples. The luminance component of the reconstructed sample may correspond to the reconstructed luminance sample, and the chrominance component of the reconstructed sample may correspond to the reconstructed chrominance sample. The decoding device can generate predicted luminance samples and / or predicted chrominance samples through prediction processing. The decoding device can generate reconstructed luminance samples based on predicted luminance samples and residual luminance samples. The decoding device can generate reconstructed chrominance samples based on predicted chrominance samples and residual chrominance samples.

[0336] The decoding device can generate modified (filtered) reconstruction samples (S920). The decoding device can generate modified reconstruction samples by performing in-loop filtering processing on the reconstruction samples of the current frame. The decoding device can generate modified reconstruction samples based on in-loop filtering related information (and / or virtual boundary related information). The decoding device can use deblocking processing, SAO processing, and / or ALF processing to generate modified reconstruction samples.

[0337] In this implementation, the image information may include an SPS (Simultaneous Partition Screen). The SPS may include an SPS virtual boundary enable flag. Additionally, in-loop filtering may be performed across virtual boundaries (or may not) based on the SPS virtual boundary enable flag. For example, signaling in the SPS or header information regarding the presence of information related to virtual boundaries may be determined based on the SPS virtual boundary enable flag. The SPS virtual boundary enable flag may indicate whether in-loop filtering can be disabled across virtual boundaries.

[0338] In an implementation, the SPS may also include an SPS virtual boundary presence flag. Whether information about the location of virtual boundaries and the number of virtual boundaries are included in the SPS can be determined based on the SPS virtual boundary presence flag.

[0339] In this implementation, based on the value of the SPS virtual boundary presence flag being 1, the SPS may include information about the number of vertical virtual boundaries.

[0340] In an implementation, the SPS may include information about the location of the vertical virtual boundaries. Furthermore, the number of entries regarding the location of the vertical virtual boundaries may be determined based on information about the number of vertical virtual boundaries.

[0341] In this implementation, based on the value of the SPS virtual boundary presence flag being 1, the SPS may include information about the number of horizontal virtual boundaries.

[0342] In an implementation, the SPS may include information about the location of the horizontal virtual boundaries. Furthermore, the number of entries regarding the location of the horizontal virtual boundaries may be determined based on information about the number of horizontal virtual boundaries.

[0343] In this implementation, the image information includes header information. Additionally, based on the SPS virtual boundary enable flag being 1 and the SPS virtual boundary presence flag being 0, the header information may include the header virtual boundary presence flag.

[0344] In this implementation, based on the value of the virtual boundary presence flag being 1, the header information may include information about the number of vertical virtual boundaries.

[0345] In this implementation, the header information may include information about the location of the vertical virtual boundary. Furthermore, the number of entries regarding the location of the vertical virtual boundary may be determined based on information about the number of vertical virtual boundaries.

[0346] In this implementation, based on the value of the virtual boundary presence flag being 1, the header information may include information about the number of horizontal virtual boundaries.

[0347] In this implementation, the header information may include information about the location of the horizontal virtual boundary. Furthermore, the number of entries regarding the location of the horizontal virtual boundary may be determined based on information about the number of horizontal virtual boundaries.

[0348] In the implementation, the SPS includes information about the location of vertical virtual boundaries and information about the location of horizontal virtual boundaries, and the sum of the number of vertical virtual boundaries and the number of horizontal virtual boundaries can be greater than 0.

[0349] In implementations, information related to in-loop filtering (and / or information related to virtual boundaries) may further include an SPS virtual boundary presence flag, a frame header virtual boundary presence flag, and a Progressive Decoding Refresh (GDR) enable flag. For example, based on a GDR enable flag value of 1, the SPS virtual boundary enable flag (virtual boundary enable flag) value may be 1, the SPS virtual boundary presence flag value may be 0, and the frame header virtual boundary presence flag value may be 1 (signaling of virtual boundary information may exist in the frame header).

[0350] Given residual samples for the current block, the decoding device can receive residual information for the current block. The residual information may include transform coefficients of the residual samples. The decoding device can derive the residual samples (or an array of residual samples) for the current block based on the residual information. Specifically, the decoding device can derive the quantized transform coefficients based on the residual information. Based on the coefficient scan order, the quantized transform coefficients may have a one-dimensional vector form. The decoding device can derive the transform coefficients based on the dequantization process of the quantized transform coefficients. The decoding device can derive the residual samples based on the transform coefficients.

[0351] The decoding device can generate reconstructed samples based on (intra-frame) predicted samples and residual samples, and can deduce reconstructed blocks or reconstructed frames based on the reconstructed samples. Specifically, the decoding device can generate reconstructed samples based on the sum between (intra-frame) predicted samples and residual samples. Subsequently, as described above, the decoding device can optionally apply in-loop filtering processing, such as deblocking filtering and / or SAO processing, to the reconstructed frames to improve subjective / objective image quality.

[0352] For example, a decoding device can obtain all or part of the image information (or syntactic elements) mentioned above by decoding the bitstream or encoded information. Furthermore, the bitstream or encoded information can be stored in a computer-readable storage medium, enabling the aforementioned decoding method to be executed.

[0353] Although the method is described in the above embodiments based on a flowchart listing steps or blocks in sequence, the steps in this document are not limited to a specific order. Specific steps may be performed in different steps or in different orders or simultaneously relative to the above content. Furthermore, those skilled in the art will understand that the steps in the flowchart are not exclusive, and another step may be included, or one or more steps in the flowchart may be deleted, without affecting the scope of this disclosure.

[0354] The methods described above according to this disclosure may be in the form of software, and the encoding and / or decoding devices according to this disclosure may be included in an apparatus for image processing (e.g., TV, computer, smartphone, set-top box, display device, etc.).

[0355] When the embodiments of this disclosure are implemented in software, the above methods can be implemented by modules (processes or functions) that perform the above functions. Modules can be stored in memory and executed by a processor. Memory can be installed inside or outside the processor and can be connected to the processor via various well-known means. The processor may include application-specific integrated circuits (ASICs), other chipsets, logic circuits, and / or data processing devices. Memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. In other words, embodiments of this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in the various figures can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information about the implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.

[0356] Furthermore, the decoding and encoding devices employing the embodiments of this document may be included in multimedia broadcast transceivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices and real-time communication devices (e.g., video communication), mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, internet streaming service providers, 3D video devices, virtual reality (VR) devices, augmented reality (AR) devices, image telephony video devices, vehicle terminals (e.g., vehicle (including autonomous vehicle) terminals, aircraft terminals, or ship terminals), and medical video devices; and can be used to process image signals or data. For example, OTT video devices may include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0357] Furthermore, the processing method applying the embodiments of this document can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data having the data structure according to the embodiments of this document can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices for storing computer-readable data. For example, computer-readable recording media may include Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Computer-readable recording media also include media embodied in the form of carrier waves (e.g., transmission via the Internet). Additionally, the bitstream generated by this encoding method can be stored in a computer-readable recording medium or transmitted via wired or wireless communication networks.

[0358] Furthermore, the embodiments described in this document can be specifically implemented as computer program products based on program code, and the program code can be executed on a computer according to the embodiments described in this document. The program code can be stored on a computer-readable medium.

[0359] Figure 11 Examples of content streaming systems to which the implementation methods of this document can be applied are shown.

[0360] Reference Figure 11 Content streaming systems that utilize the embodiments described in this document typically include encoding servers, streaming servers, network servers, media storage devices, user devices, and multimedia input devices.

[0361] An encoding server is used to compress content input from multimedia input devices (e.g., smartphones, cameras, camcorders, etc.) into digital data to generate a bitstream and send it to a streaming server. As another example, if the multimedia input device (e.g., smartphone, camera, camcorder, etc.) generates the bitstream directly, the encoding server can be omitted.

[0362] The bitstream can be generated using the encoding method or bitstream generation method described in this document. Furthermore, the stream server can temporarily store the bitstream during transmission or reception.

[0363] A streaming server sends multimedia data to a user's device based on a user's request via a web server, which acts as an instrument to inform the user of available services. When a user requests a desired service, the web server forwards the request to the streaming server, which then sends the multimedia data to the user. In this respect, the content streaming system may include a separate control server, which in this case controls the commands / responses between the various devices in the content streaming system.

[0364] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, the streaming server can store the bitstream for a predetermined period of time to smoothly provide streaming services.

[0365] For example, user equipment may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet computers, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smartwatches), glasses-type terminals (smart glasses), head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0366] In a content streaming system, each server can operate as a distributed server, and in this case, the data received by each server can be processed in a distributed manner.

[0367] The claims in this specification can be combined in various ways. For example, technical features in the method claims can be combined to implement or perform in a device, and technical features in the device claims can be combined to implement or perform in a method. Furthermore, technical features in both the method and device claims can be combined to implement or perform in a device.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: Image information, including residual information, is obtained through bitstream; Based on the residual information, a reconstructed sample of the current image is generated; and Modified reconstructed samples are generated based on in-loop filtering of the reconstructed samples. The image information includes the sequence parameter set (SPS) and the image header information. The SPS includes an SPS virtual boundary enable flag. The SPS also includes an SPS virtual boundary presence flag, based on the value of the SPS virtual boundary enable flag. Specifically, the decision to perform the in-loop filtering process across virtual boundaries is determined based on the SPS virtual boundary enable flag. Specifically, the value of the SPS virtual boundary presence flag determines whether information regarding the location of the virtual boundary and information regarding the number of virtual boundaries are included in the SPS. Wherein, based on the value of the SPS virtual boundary enable flag being 1 and the value of the SPS virtual boundary presence flag being 0, the frame header information includes the frame header virtual boundary presence flag, and Specifically, whether information about the location of the virtual boundary and information about the number of virtual boundaries, based on the value of the virtual boundary presence flag in the viewfinder, are included in the viewfinder information.

2. An image encoding method performed by an encoding device, the image encoding method comprising the following steps: Generate residual samples for the current block; Residual information is generated based on the residual samples used for the current block; Generate in-loop filtering-related information for reconstructing samples of the current frame; as well as The image information, including the residual information and the filtering-related information within the loop, is encoded. The image information includes the sequence parameter set (SPS) and the image header information. The SPS includes an SPS virtual boundary enable flag. The SPS also includes an SPS virtual boundary presence flag, based on the value of the SPS virtual boundary enable flag. The in-loop filtering related information includes the SPS virtual boundary enable flag, which relates to whether in-loop filtering is performed across virtual boundaries. Specifically, the value of the SPS virtual boundary presence flag determines whether information regarding the location of the virtual boundary and information regarding the number of virtual boundaries are included in the SPS. Wherein, based on the value of the SPS virtual boundary enable flag being 1 and the value of the SPS virtual boundary presence flag being 0, the frame header information includes the frame header virtual boundary presence flag, and Specifically, whether information about the location of the virtual boundary and information about the number of virtual boundaries, based on the value of the virtual boundary presence flag in the viewfinder, are included in the viewfinder information.

3. A method for transmitting data for an image, the method comprising the following steps: Obtaining a bitstream for the image, wherein the bitstream is generated based on the following steps: generating residual samples for the current block, generating residual information based on the residual samples for the current block, generating in-loop filtering related information for reconstructed samples of the current frame, and encoding image information including the residual information and the in-loop filtering related information; and Send the data including the bit stream. The image information includes the sequence parameter set (SPS) and the image header information. The SPS includes an SPS virtual boundary enable flag. The SPS also includes an SPS virtual boundary presence flag, based on the value of the SPS virtual boundary enable flag. The in-loop filtering related information includes the SPS virtual boundary enable flag, which relates to whether in-loop filtering is performed across virtual boundaries. Specifically, the value of the SPS virtual boundary presence flag determines whether information regarding the location of the virtual boundary and information regarding the number of virtual boundaries are included in the SPS. Wherein, based on the value of the SPS virtual boundary enable flag being 1 and the value of the SPS virtual boundary presence flag being 0, the frame header information includes the frame header virtual boundary presence flag, and Specifically, whether information about the location of the virtual boundary and information about the number of virtual boundaries, based on the value of the virtual boundary presence flag in the viewfinder, are included in the viewfinder information.

Images