Image decoding method, image encoding method, computer-readable storage medium, and method for transmitting image data

Abstract

An image decoding method performed by a decoding apparatus, according to the present disclosure, may comprise the steps of: obtaining, from a bitstream, a display rectangles (DR)-related supplemental enhancement information (SEI) message; and obtaining a thumbnail picture from an original picture on the basis of DR-related information included in the DR-related SEI message.

Description

Video decoding method, video encoding method, computer-readable storage medium and method for transmitting data for video

[0001] The present disclosure relates to a method for decoding / encoding image information, a computer-readable storage medium for storing image information, and a method for transmitting image information.

[0002] Recently, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), has been increasing across various fields. As video data becomes higher in resolution and quality, the relative amount of information or bits transmitted increases compared to conventional video data. This increase in transmitted information or bits leads to higher transmission and storage costs.

[0003] Accordingly, high-efficiency video compression technology is required to effectively transmit, store, and play back high-resolution, high-quality video information.

[0004] The present disclosure aims to improve the reliability of a coding system including an encoding device and a decoding device.

[0005] The present disclosure aims to improve the coding efficiency of a coding system including an encoding device and a decoding device.

[0006] The present disclosure aims to improve the data transmission efficiency of a coding system including an encoding device and a decoding device.

[0007] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0008] According to one aspect of the present disclosure, an image decoding method may include the step of obtaining a Display Rectangles (DR) related Supplemental Enhancement Information (SEI) message from a bitstream; and the step of obtaining a thumbnail picture from an original picture based on DR-related information included in the DR-related SEI message.

[0009] According to one aspect of the present disclosure, an apparatus for decoding image information may be characterized by acquiring a Display Rectangles (DR) related Supplemental Enhancement Information (SEI) message from a bitstream and acquiring a thumbnail picture from an original picture based on the DR related information included in the DR related SEI message.

[0010] In a method or device for decoding the above image information, the DR-related SEI message may be characterized by including at least one of identifier information identifying the DR-related SEI message, purpose information indicating the purpose of the DR-related SEI message, persistence information indicating the persistence of the DR-related SEI message, and rectangle-based thumbnail existence information indicating definition information of the thumbnail picture.

[0011] In a method or device for decoding the above image information, the target information may be characterized in that the DR-related SEI message indicates thumbnail information of the original picture or indicates spatial area information of the original picture.

[0012] In a method or device for decoding the above image information, based on the fact that the target information indicates that the DR-related SEI message indicates thumbnail information of the original picture, the DR-related SEI message may be characterized by including one display rectangle and not including information on the existence of a filling method for the display rectangle.

[0013] In a method or device for decoding the above image information, the thumbnail picture may be characterized in that it is determined by rectangular area information included in the DR-related SEI message based on the rectangular-based thumbnail existence information, or is determined as the entire picture to which the DR-related SEI message is applied.

[0014] In a method or device for decoding the above image information, based on the fact that the rectangle-based thumbnail existence information indicates that the thumbnail picture is determined by the rectangle area information, the DR-related SEI message may be characterized by including rectangle coordinate information, rectangle size information, and rectangle fill information.

[0015] In a method or device for decoding the above image information, the persistence information may be characterized by indicating whether the DR-related SEI message is applied only to the current picture, or whether it is applied to the current picture and subsequent pictures in the output order based on satisfying a predefined termination condition.

[0016] In a method or device for decoding the above image information, the DR-related SEI message further includes square information for each of a plurality of display squares, and the square information may be characterized by including, for each of the plurality of display squares, at least one of square identifier information identifying the display square, square purpose information indicating the purpose of the display square, and square-based thumbnail existence information indicating definition information of the thumbnail picture.

[0017] In a method or device for decoding the above image information, the square target information corresponding to each of the plurality of display squares may be characterized in that the DR-related SEI message indicates thumbnail information of the original picture and spatial area information of the original picture.

[0018] In a method or device for decoding the above image information, the rectangle target information corresponding to each of the plurality of display rectangles indicates that the DR-related SEI message indicates thumbnail information of the original picture, and the DR-related SEI message may be characterized in that it does not include information on the existence of a filling method for the display rectangles.

[0019] In a method or device for decoding the above image information, the thumbnail picture may be characterized in that it is determined by the rectangular area information included in the DR-related SEI message based on the rectangular-based thumbnail existence information for each of the plurality of display rectangles, or is determined as the entire picture to which the DR-related SEI message is applied.

[0020] According to one aspect of the present disclosure, an image encoding method comprises: a step of generating a DR-related SEI (Supplemental Enhancement Information) message including DR (Display Rectangles) related information; and a step of encoding image information including the DR-related SEI message, wherein a thumbnail picture is generated from an original picture based on the DR-related information.

[0021] According to one aspect of the present disclosure, an apparatus for encoding image information may be characterized by generating a DR-related SEI (Supplemental Enhancement Information) message containing DR (Display Rectangles) related information, encoding image information containing said DR-related SEI message, and generating a thumbnail picture from an original picture based on said DR-related information.

[0022] In a method or device for encoding the above-mentioned video information, the DR-related SEI message may be characterized by including at least one of identifier information identifying the DR-related SEI message, purpose information indicating the purpose of the DR-related SEI message, persistence information indicating the persistence of the DR-related SEI message, and rectangle-based thumbnail existence information indicating definition information of the thumbnail picture.

[0023] In a method or device for encoding the above-mentioned image information, the above-mentioned target information may be characterized in that the above-mentioned DR-related SEI message indicates thumbnail information of the above-mentioned original picture or indicates spatial area information of the above-mentioned original picture.

[0024] In a method or device for encoding the above-mentioned image information, the thumbnail picture may be characterized in that it is determined by rectangular area information included in the DR-related SEI message based on the rectangular-based thumbnail existence information, or is determined as the entire picture to which the DR-related SEI message is applied.

[0025] In a method or device for encoding the above-mentioned image information, the DR-related SEI message further includes square information for each of a plurality of display squares, and the square information may be characterized by including, for each of the plurality of display squares, at least one of square identifier information identifying the display square, square purpose information indicating the purpose of the display square, and square-based thumbnail existence information indicating definition information of the thumbnail picture.

[0026] In a method or device for encoding the above-mentioned image information, the square target information corresponding to each of the plurality of display squares may be characterized in that the DR-related SEI message indicates thumbnail information of the original picture and indicates spatial area information of the original picture.

[0027] In a method or device for encoding the above image information, the rectangle-based thumbnail existence information may be characterized by being generated based on the rectangle area information included in the DR-related SEI message, or determined as the entire picture to which the DR-related SEI message is applied.

[0028] In a method or device for encoding the above-mentioned image information, the DR-related SEI message further includes square information for each of a plurality of display squares, and the square information may be characterized by including, for each of the plurality of display squares, at least one of square identifier information identifying the display square, square purpose information indicating the purpose of the display square, and square-based thumbnail existence information indicating definition information of the thumbnail picture.

[0029] In a method or device for encoding the above-mentioned image information, the square target information corresponding to each of the plurality of display squares may be characterized in that the DR-related SEI message indicates thumbnail information of the original picture and indicates spatial area information of the original picture.

[0030] In a method or device for encoding the above-mentioned image information, the rectangle-based thumbnail existence information for each of the plurality of display rectangles may be characterized by being generated based on the thumbnail picture being determined by the rectangle area information included in the DR-related SEI message, or being determined as the entire picture to which the DR-related SEI message is applied.

[0031] According to one aspect of the present disclosure, a bitstream generated by a video encoding method is stored in a computer-readable storage medium. The video encoding method may include the step of generating a DR-related SEI (Supplemental Enhancement Information) message containing DR-related information; and the step of encoding video information containing the DR-related SEI message, wherein a thumbnail picture may be generated from an original picture based on the DR-related information.

[0032] According to one aspect of the present disclosure, a method for transmitting data for an image comprises: a step of acquiring image information, wherein the image information includes a DR-related SEI (Supplemental Enhancement Information) message containing DR-related information; and a step of transmitting the data containing the image information, wherein a thumbnail picture is generated from an original picture based on the DR-related information.

[0033] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0034] According to the present disclosure, the reliability of a coding system including an encoding device and a decoding device can be improved.

[0035] According to the present disclosure, the coding efficiency of a coding system including an encoding device and a decoding device can be improved.

[0036] According to the present disclosure, the data transmission efficiency of a coding system including an encoding device and a decoding device can be improved.

[0037] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0038] FIG. 1 is a schematic diagram illustrating a video coding system to which an embodiment according to the present disclosure can be applied.

[0039] FIG. 2 is a schematic diagram showing an encoding device to which an embodiment according to the present disclosure can be applied.

[0040] FIG. 3 is a schematic diagram showing a decoding device to which an embodiment according to the present disclosure can be applied.

[0041] Figure 4 illustrates an exemplary hierarchical structure for a coded video / image.

[0042] FIG. 5 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0043] FIG. 6 is a diagram illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0044] FIG. 7 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0045] Hereinafter, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0046] In describing the embodiments of the present disclosure, detailed descriptions of known configurations or functions are omitted if it is determined that such descriptions could obscure the essence of the present disclosure. Additionally, parts of the drawings unrelated to the description of the present disclosure have been omitted, and similar parts are denoted by similar reference numerals.

[0047] In the present disclosure, when a component is described as being "connected," "combined," or "joined" with another component, this may include not only a direct connection but also an indirect connection in which another component exists in between. Furthermore, when a component is described as "comprising" or "having" another component, this means that, unless specifically stated otherwise, it does not exclude the other component but may include an additional component.

[0048] In the present disclosure, terms such as first, second, etc. are used solely for the purpose of distinguishing one component from another and do not limit the order or importance of the components unless specifically stated otherwise. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and likewise, a second component in one embodiment may be referred to as a first component in another embodiment.

[0049] In this disclosure, distinct components are intended to clearly describe their respective features and do not imply that the components are separate. That is, multiple components may be integrated to form a single hardware or software unit, or a single component may be distributed to form multiple hardware or software units. Accordingly, such integrated or distributed embodiments are included within the scope of this disclosure, unless otherwise noted.

[0050] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Furthermore, embodiments including additional components in addition to the components described in various embodiments are also included within the scope of the present disclosure.

[0051] The present disclosure relates to the encoding and decoding of images. For example, the methods and embodiments disclosed in this document may be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard) or next-generation video / image coding standards (e.g., H.267 or H.268).

[0052] The present disclosure presents various embodiments relating to video / image coding, and unless otherwise stated, said embodiments may be performed in combination with one another.

[0053] Unless newly defined in this disclosure, the terms used herein may have the ordinary meanings commonly used in the technical field to which this disclosure belongs.

[0054] In this disclosure, "video" may refer to a set of images over time. In this disclosure, "picture" generally refers to a unit representing a single image at a specific time, and a slice / tile is a unit that constitutes a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular area of ​​rows of CTUs within a tile in a picture. In this document, tile groups and slices may be used interchangeably. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.

[0055] In the present disclosure, "pixel" or "pel" may refer to the smallest unit constituting a picture (or image). Additionally, "sample" may be used as a term corresponding to pixel. A sample may generally represent a pixel or a pixel value, may represent only the pixel / pixel value of the luminance component, or may represent only the pixel / pixel value of the chroma component.

[0056] In this disclosure, "unit" may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to that area. A unit may include one luminance block and two chroma (e.g., cb, cr) blocks. Depending on the case, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include samples (or sample arrays) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0057] In the present disclosure, "current block" may mean one of "current coding block," "current coding unit," "block to be encoded," "block to be decoded," or "block to be processed." When prediction is performed, "current block" may mean "current prediction block" or "block to be predicted." When transformation (inverse transformation) / quantization (inverse quantization) is performed, "current block" may mean "current transformation block" or "block to be transformed." When filtering is performed, "current block" may mean "block to be filtered."

[0058] In the present disclosure, "current block" may mean a block comprising both a luminous component block and a chroma component block, or "luma block of the current block," unless explicitly stated as a chroma block. The luminous component block of the current block may be expressed by including an explicit description of a luminous component block, such as "luma block" or "current luminous block." Additionally, the chroma component block of the current block may be expressed by including an explicit description of a chroma component block, such as "chroma block" or "current chroma block."

[0059] In the present disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Additionally, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0060] In the present disclosure, "or" may be interpreted as "and / or". For example, "A or B" may mean 1) "A" only, 2) "B" only, or 3) "A and B". Alternatively, in the present disclosure, "or" may mean "additionally or alternatively".

[0061] FIG. 1 is a schematic diagram illustrating a video / image coding system to which an embodiment according to the present disclosure can be applied.

[0062] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image or data in the form of a file or streaming to the receiving device via a digital storage medium or a network.

[0063] The source device may include a video source, an encoding device, and a transmission unit. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0064] A video source may acquire video / images through processes such as video / image capture, synthesis, or generation. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and may generate video / images (electronically). For example, virtual video / images may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0065] The encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0066] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device in the form of a file or streaming via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0067] The decoding device can decode video / images by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operation of the encoding device.

[0068] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0069] FIG. 2 is a schematic diagram illustrating an encoding device to which an embodiment according to the present disclosure can be applied.

[0070] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor (221) and an intra-predictor (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit (250) may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to the embodiment. Additionally, the memory (270) may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0071] The image segmentation unit (210) can divide an input image (or picture, frame) input to an encoding device (200) into one or more processing units. For example, the processing unit may be called a coding unit (CU). A coding unit may be recursively divided into a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, a single coding unit may be divided into multiple coding units of a deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. For example, a quad-tree structure may be applied first, and a binary-tree structure and / or a ternary-tree structure may be applied later. Alternatively, a binary-tree structure may be applied first. A coding procedure according to the present disclosure may be performed based on the final coding unit that is no longer divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be used directly as the final coding unit, or, if necessary, the maximum coding unit may be recursively divided into lower-depth coding units so that a coding unit of the optimal size is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may each be divided or partitioned from the final coding unit.The above prediction unit may be a unit of sample prediction, and the above transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0072] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. A sample may be used to refer to a single picture (or image) as a term corresponding to a pixel or pel.

[0073] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, the unit that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231). The prediction unit (220) can perform a prediction for a block to be processed (hereinafter, current block) and generate a predicted block (predicted block) containing prediction samples for said current block. The prediction unit (220) can determine whether intra prediction is applied or inter prediction is applied in units of the current block or CU. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0074] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0075] The inter prediction unit (221) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different from each other. The temporal neighboring blocks may be referred to by names such as collocated reference block, collocated CU (colCU), etc. A reference picture containing the aforementioned temporal surrounding blocks may be called a collocated picture (colPic). For example, the inter prediction unit (221) may construct a list of motion information candidates based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of surrounding blocks as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0076] The prediction unit (220) may generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit (220) may apply intra prediction or inter prediction for the prediction of the current block, as well as apply intra prediction and inter prediction simultaneously. A prediction method that applies intra prediction and inter prediction simultaneously for the prediction of the current block may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit (220) may be based on an intra block copy (IBC) prediction mode or a palette mode for the prediction of the block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. Palette mode can be viewed as an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within a picture can be signaled based on information regarding palette tables and palette indices.

[0077] The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or to generate a residual signal. The subtraction unit (231) can generate a residual signal (residual signal, residual block, residual sample array) by subtracting the prediction signal (predicted block, prediction sample array) output from the prediction unit (220) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit (232).

[0078] The transformation unit (232) can generate transform coefficients by applying a transformation technique to a residual signal. For example, the transformation technique may include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transformation obtained based on a prediction signal generated using all previously reconstructed pixels. The transformation process may be applied to a block of pixels of the same size in a square, or to a block of variable size that is not square.

[0079] The quantization unit (233) can quantize the transformation coefficients and transmit them to the entropy encoding unit (240). The entropy encoding unit (240) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (233) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients.

[0080] The entropy encoding unit (240) can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit (190) may encode information required for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The signaling information, transmitted information, and / or syntax elements mentioned in the present disclosure may be included in the video / image information. The video / image information may be encoded through the encoding procedure described above and included in the bitstream.

[0081] The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmitting a signal output from the entropy encoding unit (240) and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the encoding device (200), or the transmission unit may be provided as a component of the entropy encoding unit (240).

[0082] The quantized transformation coefficients output from the quantization unit (233) can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) can be restored by applying inverse quantization and inverse transformation to the quantized transformation coefficients through the inverse quantization unit (234) and the inverse transformation unit (235).

[0083] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0084] The adder (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The adder (250) may be called a reconstructed unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0085] The filtering unit (260) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (260) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (270), specifically in the DPB of memory (170). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240), as described below in the description of each filtering method. The information regarding filtering can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0086] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter-prediction unit (221). Through this, the encoding device (200) can avoid prediction mismatches between the encoding device (200) and the decoding device when inter-prediction is applied, and can also improve encoding efficiency.

[0087] The DPB in memory (270) can store a modified restored picture to be used as a reference picture in the inter prediction unit (221). Memory (270) can store motion information of blocks from which motion information is derived (or encoded) in the current picture and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter prediction unit (221) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory (270) can store restoration samples of restored blocks in the current picture and transmit them to the intra prediction unit (222).

[0088] FIG. 3 is a schematic diagram illustrating a decoding device to which an embodiment according to the present disclosure can be applied.

[0089] As illustrated in FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor (332) and an intra-predictor (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The aforementioned entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) may be configured by a single hardware component (e.g., a decoder chipset or a processor) according to an embodiment. Additionally, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0090] When a bitstream containing video / image information is input, the decoding device (300) can restore the image by performing a process corresponding to the process performed by the encoding device (200) of FIG. 2. For example, the decoding device (300) can perform decoding using a processing unit applied in the encoding device (200). Thus, the processing unit for decoding may be, for example, a coding unit. The coding unit may be a coding tree unit, or a maximum coding unit may be obtained by dividing it according to a quad tree structure, a binary tree structure, and / or a binary tree structure. And, the restored image signal decoded and output through the decoding device (300) can be played back through a playback device (not shown).

[0091] The decoding device (300) can receive a signal output from the encoding device (200) of FIG. 2 in the form of a bitstream. The received signal can be decoded through an entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The decoding device (300) can decode the picture based on the information regarding the parameter sets and / or the general constraint information. The signaling / received information and / or syntax elements described below can be obtained from the bitstream by decoding through the decoding procedure. For example, the entropy decoding unit (310) can decode information within the bitstream based on coding methods such as exponential coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transformation coefficients regarding residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information on the syntax element to be decoded and decoding information of surrounding and decoding target blocks or information on symbols / bins decoded in the previous step, predict the probability of occurrence of the bin according to the determined context model, and perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin after determining the context model. Among the information decoded in the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (330), and residual values ​​for which entropy decoding was performed in the entropy decoding unit (310), i.e., quantized transformation coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive residual signals (residual blocks, residual samples, residual sample array). Additionally, among the information decoded in the entropy decoding unit (310), information regarding filtering can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), inverse transform unit (322), adder (340), filtering unit (350), memory (360), inter prediction unit (332), and intra prediction unit (331).

[0092] In the inverse quantization unit (321), the quantized transformation coefficients can be inversely quantized to output transformation coefficients. The inverse quantization unit (321) can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device (200). The inverse quantization unit (321) can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain transformation coefficients.

[0093] In the inverse conversion unit (322), the conversion coefficients can be inversely converted to obtain a residual signal (residual block, residual sample array).

[0094] The prediction unit (330) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and palette index can be included in the above video / image information and signaled.

[0095] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The description of the intra prediction unit (222) may be applied equally to the intra prediction unit (331). The referenced samples may be located in the neighborhood of the current block or located away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit (331) may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0096] The inter prediction unit (332) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-prediction can be performed based on various prediction modes (techniques), and information regarding the prediction may include information indicating the mode (technique) of inter-prediction for the current block.

[0097] The adder (340) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (330) (including the inter prediction unit (332) and / or intra prediction unit (331)). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block. The description of the adder (250) can be applied equally to the adder (340). The adder (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0098] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0099] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (360), specifically in the DPB of memory (360). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0100] The (modified) restored picture stored in the DPB of the memory (360) can be used as a reference picture in the inter-prediction unit (332). The memory (360) can store motion information of blocks from which motion information within the current picture has been derived (or decoded) and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (332) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (360) can store restoration samples of blocks restored within the current picture and transmit them to the intra-prediction unit (331).

[0101] In this specification, the embodiments described in the filtering unit (260), inter prediction unit (221), and intra prediction unit (222) of the encoding device (200) may be applied to the filtering unit (350), inter prediction unit (332), and intra prediction unit (331) of the decoding device (300) in the same or corresponding manner.

[0102] Figure 4 illustrates an exemplary hierarchical structure for a coded video / image.

[0103] Referring to Figure 4, the coded image is divided into a VCL (video coding layer) that handles the decoding processing of the image and the image itself, a subsystem that transmits and stores the encoded information, and a NAL (network abstraction layer) that exists between the VCL and the subsystem and is responsible for network adaptation functions.

[0104] In VCL, VCL data containing compressed image data (slice data) can be generated, or parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages that are additionally required in the decoding process of the image can be generated.

[0105] In NAL, a NAL unit can be created by adding header information (NAL unit header) to the Raw Byte Sequence Payload (RBSP) generated in VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated in VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the NAL unit.

[0106] As shown in FIG. 4, NAL units can be classified into VCL NAL units and Non-VCL NAL units depending on the RBSP generated in VCL. A VCL NAL unit may refer to a NAL unit containing information about an image (slice data), and a Non-VCL NAL unit may refer to a NAL unit containing information necessary to decode an image (parameter set or SEI message).

[0107] The aforementioned VCL NAL unit and Non-VCL NAL unit can be transmitted over a network by attaching header information according to the data specifications of the underlying system. For example, the NAL unit can be transformed into a data format of a specified specification, such as H.266 / VVC file format, RTP (Real-time Transport Protocol), TS (Transport Stream), etc., and transmitted over various networks.

[0108] As described above, the NAL unit type can be determined according to the RBSP data structure included in the NAL unit, and information about this NAL unit type can be stored in the NAL unit header and signaled.

[0109] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether they contain information about the image (slice data). VCL NAL unit types can be classified according to the properties and types of the picture included in the VCL NAL unit, while Non-VCL NAL unit types can be classified according to the types of parameter sets.

[0110] The following is an example of a NAL unit type specified according to the type of parameter set included in the Non-VCL NAL unit type.

[0111] - APS (Adaptation Parameter Set) NAL unit: Type for the NAL unit containing the APS

[0112] - DPS (Decoding Parameter Set) NAL unit: Type for the NAL unit containing the DPS

[0113] - VPS (Video Parameter Set) NAL unit: Type for the NAL unit containing the VPS

[0114] - SPS (Sequence Parameter Set) NAL unit: Type for the NAL unit containing the SPS

[0115] - PPS(Picture Parameter Set) NAL unit: Type for the NAL unit containing the PPS

[0116] The above-described NAL unit types have syntax information for the NAL unit type, and said syntax information can be stored in the NAL unit header and signaled. For example, said syntax information may be nal_unit_type, and NAL unit types may be specified by the nal_unit_type value.

[0117] The slice header (slice header syntax) may include information / parameters that can be commonly applied to the slice. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be commonly applied to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that can be commonly applied to one or more sequences. The VPS (VPS syntax) may include information / parameters that can be commonly applied to multiple layers. The DPS (DPS syntax) may include information / parameters that can be commonly applied to the entire video. The DPS may include information / parameters related to the concatenation of the CVS (coded video sequence). In this document, the term High level syntax (HLS) may include at least one of the above APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[0118] In the present disclosure, image / video information encoded from an encoding device to a decoding device and signaled in the form of a bitstream includes not only information related to partitioning within a picture, intra / inter prediction information, residual information, in-loop filtering information, etc., but may also include information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, and / or information included in the VPS.

[0119] In the present disclosure, the following descriptors may define the parsing process for each syntax element.

[0120] - ae(v) is a context-adaptive arithmetic entropy-coded syntax element.

[0121] - b(8) is a byte with an arbitrary bit string pattern of length 8 bits. The parsing process of this descriptor is defined by the return value of the function read_bits(8).

[0122] - f(n) is an n-bit fixed-pattern bit string where the left bits are written first. The parsing process of this descriptor is defined by the return value of the function read_bits(n).

[0123] - i(n) is a signed integer using n bits, where n is denoted as "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process of this descriptor is defined by interpreting the return value of the function read_bits(n) into a two's complement integer representation where the most significant bit is written first.

[0124] - se(v) is a 0th-order Exp-Golomb encoded signed integer syntax element, written starting from the left bit. The parsing process of this descriptor is defined with order k set to 0.

[0125] - st(v) is a null-terminated string encoded in UCS (Universal Coded Character Set) transmission format 8 (UTF-8) characters as defined in ISO / IEC 10646. The parsing process is defined as follows.

[0126] st(v) starts at a byte-aligned position within the bitstream and reads and returns a sequence of consecutive bytes from the bitstream, starting from the current position and excluding the next byte-aligned byte with a value of 0x00. The bitstream pointer then advances by (stringLength + 1) * 8 bit positions, where stringLength is equal to the number of returned bytes.

[0127] For reference, the st(v) syntax descriptor is used in this disclosure only when the current position in the bitstream is a byte-aligned position.

[0128] - tu(v) is a truncated unary encoding scheme that uses up to the number of bits defined by the maximum value (maxVal), and maxVal is defined in the semantics of the corresponding syntax element.

[0129] - u(n) is an unsigned integer using n bits. If n is marked as "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process of this descriptor is defined by interpreting the return value of the function read_bits(n) as a binary representation of an unsigned integer where the most significant bit is written first.

[0130] - ue(v) is a 0th-order Exp-Golomb encoded unsigned integer syntax element, written starting from the left bit. The parsing process of this descriptor is defined with order k set to 0.

[0131] The SEI message related to the present disclosure will be described below.

[0132] Table 1 shows an example of a Display Rectangles SEI message syntax element according to one embodiment.

[0133] [Table 1]

[0134]

[0135] An example of the semantics of a Display Rectangles SEI message according to one embodiment is described.

[0136] The display rectangle (DR) SEI message describes one or more display rectangles that can be selected to be used to form a recommended display picture from a cropped decoded picture.

[0137] The use of this SEI message requires the definition of the following variables:

[0138] - The width and height of the cropped decoded output picture in units of luma samples, referred to herein as CroppedWidth and CroppedHeight, respectively.

[0139] - As a chroma format indicator, it is referred to as ChromaFormatIdc in this specification.

[0140] - The cropped decoded picture array DecodedPicture[ cIdx ][ x ][ y ], where cIdx = 0..(ChromaFormatIdc = = 0 ) ? 0 : 2, x = 0..( cIdx = = 0 ) ? CroppedWidth : CroppedWidth / SubWidthC - 1, y = 0..( cIdx = = 0 ) ? CroppedHeight : CroppedHeight / SubHeightC - 1.

[0141] - Bit depth for a sample of a luma component, referred to as BitDepthY in this specification, and, where ChromaFormatIdc is not 0, bit depth for a sample of two associated chroma components, referred to as BitDepthC in this specification.

[0142] Variables SubWidthC and SubHeightC are derived from ChromaFormatIdc.

[0143] A dr_cancel_flag equal to 1 indicates that the SEI message cancels the persistence of any previous DR SEI message. A dr_cancel_flag equal to 0 indicates that display rectangle information follows.

[0144] The DR SEI message applies to the current picture and persists to all subsequent pictures in output order until one or more of the following conditions become true:

[0145] - New CLVS started.

[0146] - Bitstream terminated.

[0147] - A picture with a DR SEI message is output from the current layer, and that picture is positioned after the current picture in the output order.

[0148] A dr_pos_updates_flag equal to 0 indicates that the SEI message provides descriptions of one or more display rectangles used to form a display picture from a cropped decoded picture. A dr_pos_updates_flag equal to 1 indicates that the SEI message contains updates for one or more display rectangles. For the first display rectangle SEI existing within the CLVS, the value of dr_pos_updates_flag must be equal to 0.

[0149] dr_log2_unit_size specifies the size of the display rectangle and the granularity of the position.

[0150] DrUnitSize = 1 << dr_log2_unit_size

[0151] As a requirement for bitstream conformity, DrUnitSize % SubWidthC and DrUnitSize % SubHeightC must be equal to 0.

[0152] If dr_pos_updates_flag is equal to 0, the variable DrUnitSizeInit is set to equal to DrUnitSize.

[0153] The value of dr_rect_size_len_minus1 plus 1 specifies the lengths of the syntax elements dr_rect_top_left_x[ i ], dr_rect_top_left_y[ i ], dr_rect_width[ i ], dr_rect_height[ i ], dr_fill_top_left_x[ i ], dr_fill_top_left_y[ i ], dr_fill_width[ i ] and dr_fill_height[ i ].

[0154] The value obtained by adding 1 to dr_num_rect_minus1 specifies the number of display rectangles described in the SEI message.

[0155] dr_display_aspect_ratio[ i ] represents the display aspect ratio of the i-th display rectangle as defined in Table 2 below.

[0156] [Table 2]

[0157]

[0158] dr_rect_width[ i ] specifies the width of the i-th rectangle in units. If it does not exist, the value of dr_rect_width[ i ] is inferred to be the same as the value of the corresponding syntax element in the previous DR SEI message in terms of output order.

[0159] The variable DrRectWidth is derived as follows.

[0160] DrRectWidth[ i ] = DrUnitSizeInit * dr_rect_width[ i ]

[0161] dr_rect_height[ i ] specifies the height of the i-th rectangle in DrUnitSizeInit units, if present. If it does not exist and dr_display_aspect_ratio[ i ] is equal to 0, the value of dr_rect_height[ i ] is inferred to be the same as the value of the corresponding syntax element in the previous DR SEI message in terms of output order.

[0162] The variable DrRectHeight is derived as follows.

[0163] if ( dr_display_aspect_ratio[ i ] == 0 )

[0164] DrRectHeight[ i ] = DrUnitSizeInit * dr_rect_height[ i ]

[0165] else

[0166] DrRectHeight[ i ] = ( DrRectWidth[ i ] * DrDarHeight[ i ] ) / DrDarWidth[ i ]

[0167] prs_fill_method_present_flag[ i ] equal to 1 indicates that the dr_fill_method_idc[ i ] syntax element exists. prs_fill_method_present_flag[ i ] equal to 0 indicates that the dr_fill_method_idc[ i ] syntax element does not exist.

[0168] dr_fill_method_idc[ i ], if present, represents the filling method applied to the i-th rectangle as defined in Table 3 below. If not present, the value of dr_fill_method_idc[ i ] is inferred to be equal to 0. The value of dr_fill_method_idc[ i ] must be within the range of 0 to 2 (inclusive) in a bitstream suitable for this version of the disclosure. Values ​​in the range of 3 to 8 (inclusive) for dr_fill_method_idc[ i ] are reserved for future use by ITU-T | ISO / IEC. While this version of the disclosure requires that the value of dr_fill_method_idc[ i ] be within the range of 0 to 2 (inclusive), the decoder must allow values ​​in the range of 3 to 7 (inclusive) for dr_fill_method_idc[ i ] to appear in the syntax, in which case the filling method is not defined.

[0169] [Table 3]

[0170]

[0171] dr_three_colour_comp_flag[ i ], equivalent to 1, specifies that the dr_target_init_sample[ i ] syntax element will be signaled for the three color components of the i-th square. dr_three_colour_comp_flag, equivalent to 0, specifies that the dr_fill_value[ i ] syntax element will be signaled for one color component.

[0172] The value of dr_fill_bit_depth_minus8[ i ] plus 8 specifies the bit depth of the fill value for the i-th display rectangle. dr_fill_bit_depth_minus8[ i ] must be in the range of 0 to 8 (inclusive).

[0173] The variable initBitDepth[ i ] is set to be equal to dr_fill_bit_depth_minus8[ i ] + 8.

[0174] dr_fill_value[ i ][ j ] represents the fill value for the j-th color component of the i-th rectangle. The length of the above syntax element is initBitDepth[ i ] bits.

[0175] If not present, the value of dr_fill_value

[0000] is inferred to be equal to 0, and the values ​​of dr_fill_value

[0001] and dr_fill_value

[0002] are inferred to be equal to 1 << (BitDepth - 1).

[0176] dr_fill_blur_sigma[ i ] specifies the sigma parameter of the Gaussian blur to be applied to the fill of the i-th display rectangle, if present.

[0177] dr_fill_content_width[ i ] specifies the width of the content area used to fill the i-th display rectangle in units of DrUnitSizeInit. If it does not exist, the value of dr_fill_content_width[ i ] is set to the value of the corresponding syntax element in the previous DR SEI message in the output order.

[0178] The variable DrFillWidth is derived as follows.

[0179] DrFillWidth[ i ] = DrUnitSizeInit * dr_fill_content_width[ i ]

[0180] The variable DrFillHeight is derived as follows.

[0181] DrFillHeight[ i ] = ( DrFillWidth[ i ] * DrRectWidth[ i ] ) / DrRectHeight[ i ]

[0182] dr_rect_top_left_x[ i ] and dr_rect_top_left_y[ i ] specify the top-left x and y coordinates of the i-th rectangle in DrUnitSize units, respectively.

[0183] dr_rect_top_left_x_sign[ i ] and dr_rect_top_left_y_sign[ i ] specify the sign to be applied to dr_rect_top_left_x[ i ] and dr_rect_top_left_y[ i ], respectively.

[0184] dr_fill_content_region_top_left_x[ i ] and dr_fill_content_region_top_left_y[ i ] respectively specify the top-left x and y coordinates of the area used for filling the i-th display rectangle in units of DrUnitSize.

[0185] Extrapolate(OutPic, InPic, x0, y0, w0, h0) is defined as a function that extracts a rectangle of position (x0, y0) and luminance size w0Xh0 from the input picture InPic and extrapolates it to the output picture OutPic. If OutPic has larger dimensions than the area extracted from InPic, an upscaling operation is performed.

[0186] Blur(OutPic, InPic, x0, y0, w0, h0, sigma0) is defined as a function that extracts a rectangle of position (x0, y0) and luminance size w0Xh0 from the input picture InPic, scales it to the resolution of the output picture OutPic, and applies a Gaussian blur with a standard deviation of sigma0. If OutPic has a larger dimension than the area extracted from InPic, an upscaling operation is performed.

[0187] The selected display rectangle index selIdx can be determined by external means by comparing the aspect ratio and display resolution of the intended display with the values ​​of the syntax elements dr_display_aspect_ratio[ i ], dr_rect_width[ i ], and dr_rect_height[ i ] for each display rectangle.

[0188] When the value of selIdx is selected, it is recommended to form the picture intended for display DrDisplay[ selIdx ] to have a luminance resolution DrRectWidth[ selIdx ] x DrRectHeight[ selIdx ] and a chroma format ChromaFormatIdc, which is derived as shown in Table 4 below.

[0189] [Table 4]

[0190]

[0191] Table 5 below shows an example of a thumbnail source information (TSI) SEI message syntax element according to one embodiment.

[0192] [Table 5]

[0193]

[0194] An example of the semantics of a thumbnail source information (TSI) SEI message according to one embodiment is described.

[0195] Thumbnail Source Information (TSI) SEI messages provide information about the source of a thumbnail image or video for associated video content.

[0196] If a TSI SEI message exists in any AU of a CVS, a TSI SEI message must exist in the first AU of that CVS. All TSI SEI messages within a CVS must have the same content.

[0197] tsi_thumbnail_id contains an identification number that can be used to uniquely identify a thumbnail image or video within a bitstream.

[0198] tsi_source_type indicates the source of the thumbnail image or video as specified in Table 6 below.

[0199] [Table 6]

[0200]

[0201] If tsi_layer_id exists, it specifies the layer identifier of the layer from which the thumbnail image or video is obtained.

[0202] If present, tsi_uri identifies the source of a thumbnail image or video and includes a URI having the syntax and semantics specified by IETF Internet Standard 66.

[0203] A tsi_region_info_present_flag equal to 1 indicates that the tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y syntax elements exist in the thumbnail extraction information SEI message associated with the current TSI SEI message. A tsi_region_info_present_flag equal to 0 indicates that the tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y syntax elements do not exist in the thumbnail extraction information SEI message associated with the current TSI SEI message.

[0204] Table 7 shows an example of a thumbnail extraction information (TEI) SEI message syntax element according to one embodiment.

[0205] [Table 7]

[0206]

[0207] An example of the semantics of a thumbnail extraction information (TEI) message according to one embodiment is described.

[0208] A thumbnail extraction information SEI message indicates that one or more pictures to which the SEI message applies, or a specific area of ​​the picture(s), is a thumbnail image or video for the associated video content.

[0209] If there is a TSI SEI message in CLVS where tsi_source_type is equal to 0 and tsi_thumbnail_id is equal to curThumbId, then there must be at least one TEI SEI message in CLVS where tei_thumbnail_id is equal to curThumbId. If there is a TEI SEI message in CLVS where tei_thumbnail_id is equal to curThumbId, then there must be a TSI SEI message in CVS where tsi_source_type is less than 2 and tsi_thumbnail_id is equal to curThumbId.

[0210] tei_thumbnail_id includes an identification number that can be used to uniquely identify a thumbnail image or video within a bitstream.

[0211] A tei_cancel_flag equal to 1 indicates that the TEI SEI message cancels the persistence of the previous TEI SEI message in the output order applied to the current layer. A tei_cancel_flag equal to 0 indicates that thumbnail extraction information follows.

[0212] tei_persistence_flag specifies the persistence of thumbnail extraction information provided by this SEI message. tei_persistence_flag equal to 0 specifies that thumbnail extraction information is applied only to the current picture. tei_persistence_flag equal to 1 specifies that thumbnail extraction information is applied to the current picture and all subsequent pictures of the current layer in the output order, provided that it is applied until one or more of the following conditions are true.

[0213] - New CLVS of the current layer have started.

[0214] - Bitstream terminated.

[0215] - The picture of the current layer associated with the TEI SEI message is output, and that picture is positioned after the current picture in the output order.

[0216] If tei_width_minus1 and tei_height_minus1 exist, tei_width_minus1 + 1 and tei_height_minus1 + 1 specify the width and height of the corresponding area in units of luma samples, respectively.

[0217] As a requirement for bitstream conformity, (tei_width_minus1 + 1) % SubWidthC must be equal to 0, and (tei_height_minus1 + 1) % SubHeightC must be equal to 0.

[0218] If tei_top_left_x and tei_top_left_y exist, tei_top_left_x and tei_top_left_y specify the horizontal and vertical positions of the top-left sample of the corresponding area, respectively.

[0219] As a requirement for bitstream conformity, tei_top_left_x % SubWidthC must be equal to 0, and tei_top_left_y % SubHeightC must be equal to 0.

[0220] The region(s) indicated by tei_width_minus1, tei_height_minus1, tei_top_left_x and tei_top_left_y are, if present, thumbnail images or videos. If tei_width_minus1, tei_height_minus1, tei_top_left_x and tei_top_left_y do not exist, the picture(s) to which the TEI SEI message applies are thumbnail images or videos.

[0221] Thumbnails serve as a means to facilitate previewing and navigation of video content. They are typically created by extracting and editing specific portions of the original video, or provided as separately encoded video content. Depending on the implementation, they may be presented as full-screen preview videos or as small icons to support efficient content browsing.

[0222] We argue that since thumbnails can be generated by extracting and editing specific parts of original video content, it is useful to include an SEI message indicating the source of the thumbnail and the method used to extract and edit the original video content to obtain the thumbnail. It is argued that this approach provides common signaling across different systems and devices while ensuring a clear association between the original content and the corresponding thumbnail. Furthermore, we argue that integrating thumbnail signaling within the video stream can improve storage efficiency and facilitate synchronization between thumbnails and video playback.

[0223] Accordingly, the present invention proposes SEI messages for indicating the source and extraction information of a thumbnail video of an image.

[0224] Thumbnails are representative images or videos that provide a quick preview of content, helping users identify and navigate videos more easily. They can be determined by creators or content providers to reflect their intentions, and nowadays, AI is also increasingly being used to automatically generate thumbnails based on content analysis.

[0225] Thumbnails can be generated by extracting and editing temporal and / or spatial portions of the original video content. They may also be provided as separately encoded video content. The encoded video content of the thumbnail may be a specific layer of the current bitstream, or it may be a bitstream separate from the bitstream of the original video content.

[0226] In Content Provider (CP) services, thumbnails are frequently used as full-screen preview videos to enhance the browsing experience. These previews allow users to quickly grasp the essence of the content before playback, making it easier to choose what to watch.

[0227] On user devices, thumbnails are typically generated by capturing a portion of a video and displaying it as an icon. These icons help users efficiently navigate videos they have saved or recently played, and provide a convenient way to browse their content library.

[0228] Each item according to one embodiment may be applied individually or in combination.

[0229] 1. Instead of the TEI SEI message, modify the display rectangle SEI message so that the SEI message provides thumbnail extraction information.

[0230] 2. Alternatively, instead of the TEI SEI message, the display rectangle SEI message is modified so that the display rectangle within the SEI message provides thumbnail extraction information.

[0231] One embodiment relates to Item 1 described above. One embodiment is based on the VSEI and VVC specifications. The syntax elements of a Display Rectangles SEI message according to one embodiment are as shown in Table 8 below.

[0232] [Table 8]

[0233]

[0234] An example of semantics of a Display Rectangles SEI message according to one embodiment is described. Semantics different from the semantics of the DR SEI message described above are described, and any description of semantics identical to the semantics of the DR SEI message described above is replaced by the description of the semantics of the DR SEI message described above.

[0235] dr_id contains an identification number that can be used to identify DR SEI messages. The value of nnpfc_id must be in the range of 0 to 16,383 (inclusive).

[0236] dr_persistence_flag specifies the persistence of DR SEI messages. dr_persistence_flag equal to 0 specifies that DR SEI messages apply only to the current picture. dr_persistence_flag equal to 1 specifies that DR SEI messages apply to the current picture and all pictures following the current layer in output order, and persist until one or more of the following conditions are true.

[0237] - New CLVS of the current layer have started.

[0238] - Bitstream terminated.

[0239] - The picture associated with the DR SEI message is output from the current layer, and that picture is positioned after the current picture in the output order.

[0240] dr_purpose represents the purpose of the DR SEI as defined in Table 9 below. The value of dr_purpose must be within the range of 0 to 1 (inclusive). Values ​​for dr_purpose in the range of 2 to 255 (inclusive) are reserved for future use by ITU-T | ISO / IEC and must not exist in the bitstream suitable for this version of the disclosure. A decoder suitable for this version of the disclosure must allow any value of dr_purpose in the range of 0 to 255 (inclusive).

[0241] [Table 9]

[0242]

[0243] dr_rect_info_present_flag equal to 1 includes dr_pos_updates_flag, dr_log2_unit_size, dr_rect_size_len_minus1, dr_num_rect_minus1, dr_rect_top_left_x[ i ], dr_rect_top_left_x_sign[ i ], dr_rect_top_left_y[ i ], and dr_rect_top_left_y_sign[ i ] indicates the presence of syntax elements, dr_display_aspect_ratio[ i ], dr_rect_width[ i ], dr_rect_height[ i ], dr_fill_method_present_flag[ i ], dr_fill_method_idc[ i ], dr_fill_bit_depth_minus8[ i ], dr_fill_value[ i ][ j ], dr_fill_blur_sigma[ i ], The syntax elements dr_fill_content_width[ i ], dr_fill_content_region_top_left_x[ i ], and dr_fill_content_region_top_left_y[ i ] indicate that they may exist.dr_rect_info_present_flag equal to 0 is dr_pos_updates_flag, dr_log2_unit_size, dr_rect_size_len_minus1, dr_num_rect_minus1, dr_rect_top_left_x[ i ], dr_rect_top_left_x_sign[ i ], dr_rect_top_left_y[ i ], dr_display_aspect_ratio[ i ], dr_rect_width[ i ], dr_rect_height[ i ], dr_fill_method_present_flag[ i ], dr_fill_method_idc[ i ], dr_fill_bit_depth_minus8[ i ], dr_fill_value[ i ][ j ], dr_fill_blur_sigma[ i ], dr_fill_content_width[ i ], dr_fill_content_region_top_left_x[ i ], and dr_fill_content_region_top_left_y[ i ] indicates that syntax elements do not exist.

[0244] If dr_purpose is equal to 0, dr_num_rect_minus1 must be equal to 0 and dr_fill_method_present_flag[ i ] must be equal to 0.

[0245] If there is a TSI SEI message in CLVS where tsi_source_type is 0 and tsi_thumbnail_id is equal to curThumbId, then there must be at least one DR SEI message in CLVS where dr_id is equal to curThumbId and dr_purpose is equal to 0. If there is a DR SEI message in CLVS where dr_id is equal to curThumbId and dr_purpose is equal to 0, then there must be a TSI SEI message in CVS where tsi_source_type is less than 2 and tsi_thumbnail_id is equal to curThumbId.

[0246] If dr_rect_info_present_flag is equal to 1, the display rectangle(s) indicated by DrRectWidth[i], DrRectHeight[i], DrRectTopLeftX[i], and DrRectTopLeftX[i] are thumbnail images or videos. If dr_rect_info_present_flag is equal to 0, the picture(s) to which the DR SEI message applies are thumbnail images or videos.

[0247] One embodiment relates to Item 2 described above. One embodiment is based on the VSEI and VVC specifications. The Thumbnail Source Information (TSI) SEI message syntax according to one embodiment is as shown in Table 10 below.

[0248] [Table 10]

[0249]

[0250] An example of semantics of a Display Rectangles SEI message according to one embodiment is described. Semantics different from those of the Display Rectangles (DR) SEI message described above are described, and any description of semantics identical to those of the DR SEI message described above is replaced by the description of the DR SEI message semantics described above.

[0251] dr_persistence_flag specifies the persistence of DR SEI messages. A dr_persistence_flag equal to 0 specifies that DR SEI messages apply only to the current picture. A dr_persistence_flag equal to 1 specifies that DR SEI messages apply to the current picture and all pictures following the current layer in output order, and persists until one or more of the following conditions become true:

[0252] - New CLVS of the current layer have started.

[0253] - Bitstream terminated.

[0254] - The picture associated with the DR SEI message is output from the current layer, and that picture is positioned after the current picture in the output order.

[0255] dr_rect_id[ i ] contains an identification number that can be used to identify the i-th display rectangle. The value of nnpfc_id must be in the range of 0 to 16,383 (inclusive).

[0256] dr_rect_purpose[ i ] represents the purpose of the i-th display rectangle as defined in Table 11 below. The value of dr_rect_purpose[ i ] must be in the range of 0 to 1 (inclusive). Values ​​for dr_purpose in the range of 2 to 255 (inclusive) are reserved for future use by ITU-T | ISO / IEC and must not exist in the bitstream suitable for this version of the disclosure. A decoder suitable for this version of the disclosure must allow any value of dr_purpose in the range of 0 to 255 (inclusive).

[0257] [Table 11]

[0258]

[0259] If dr_rect_purpose[ i ] is equal to 0, dr_fill_method_present_flag[ i ] must be equal to 0.

[0260] If there exists a TSI SEI message in CLVS where tsi_source_type is 0 and tsi_thumbnail_id is equal to curThumbId, then there must be at least one DR SEI message in CLVS where dr_rect_id[i] is equal to curThumbId and dr_rect_purpose[i] is equal to 0. If there exists a DR SEI message in CLVS where dr_rect_id[i] is equal to curThumbId and dr_rect_purpose[i] is equal to 0, then there must be a TSI SEI message in CVS where tsi_source_type is less than 2 and tsi_thumbnail_id is equal to curThumbId.

[0261] If dr_rect_info_present_flag[ i ] is equal to 1, the display rectangle(s) indicated by DrRectWidth[ i ], DrRectHeight[ i ], DrRectTopLeftX[ i ] and DrRectTopLeftX[ i ] are thumbnail images or videos. If dr_rect_info_present_flag[ i ] is equal to 0, the picture(s) to which the DR SEI message applies are thumbnail images or videos.

[0262] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms, etc. described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0263] The operations described below do not constitute an essential component of one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of one embodiment, and previously described operations may be added. Moreover, unless they contradict previously described operations, the operations described below form one embodiment integrally with previously described operations and do not form a separate embodiment distinct from previously described operations.

[0264] FIG. 5 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0265] The decoding method (S500) may include the operations described below.

[0266] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0267] The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the decoding method according to one embodiment, and the previously described operations may be added.

[0268] The operations described below form a single embodiment integrally with the configurations and / or operations described above, unless they conflict with the configurations and / or operations described above, and do not form a separate embodiment distinct from the configurations and / or operations described above.

[0269] The decoding method (S500) can be executed by a decoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0270] A decoding device can acquire image information. For example, a processor of a decoding device can acquire image information. The image information may include a picture to be decoded.

[0271] For example, the image information may include additional information related to at least one picture, such as information regarding one or more display rectangles used to construct a display picture from a cropped decoded picture (e.g., information on the number of display rectangles, size information, filling information, ratio information, location information, identifier information, persistence information, cancellation information, and / or purpose information). Additionally, the image information may further include rectangle-based thumbnail presence information indicating whether the thumbnail picture is determined by a display rectangle area or by the entire picture to which a display rectangle-related SEI message is applied.

[0272] The decoding device can acquire a SEI (supplemental enhancement information) message (S510).

[0273] SEI messages can convey specific types of information that assist in processes related to the decoding, display, or other purposes of image information. Here, SEI messages may not be necessary for the decoding process to determine the sample values ​​of the decoded picture.

[0274] For example, SEI messages may include SEI messages related to display rectangles.

[0275] A SEI message related to a display rectangle may indicate one or more display rectangles that can be used to form a display picture from a cropped decoding picture, or may include information regarding such rectangles.

[0276] The SEI message related to the display rectangle may have various names such as display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, DR SEI message, DR related message, DR related information, DR message, DR information, display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, etc., and the names are not limited.

[0277] SEI messages related to display rectangles can take various forms. For example, SEI messages related to DR can be syntax elements or syntax structures containing one or more syntax elements. Additionally, SEI messages related to display rectangles can be raw byte sequence payloads (RBSP) containing one or more syntax elements or syntax structures. For example, SEI messages related to display rectangles can be expressed as display_rectangles ( payloadSize ), but are not limited thereto.

[0278] For example, the decoding device can obtain SEI messages related to the display rectangle (DR) from the bitstream.

[0279] For example, a decoding device can parse a bitstream received from an encoding device to obtain a DR-related SEI message. The DR-related SEI message may include DR-related information regarding at least one display rectangle that can be used to form a display picture from a cropped decoding picture.

[0280] A DR-related SEI message may include information on the number of display rectangles, information on the ratio of display rectangles, information on the fill of display rectangles, information on the size of display rectangles, information on the position of display rectangles, information on canceling display rectangles, information on the unit of display rectangles, information on the length of display rectangles, and / or information on updating display rectangles. Additionally, a DR-related SEI message may include at least one of identifier information identifying the DR-related SEI message, purpose information indicating the purpose of the DR-related SEI message, persistence information indicating the persistence of the DR-related SEI message, and rectangle-based thumbnail existence information indicating definition information of a thumbnail picture.

[0281] The above identifier information may include an identification number that can be used to identify DR-related SEI messages. For example, the value of the identifier information may be within the range of 0 to 16,383 (inclusive).

[0282] Identifier information may take various forms and may be represented by various names. For example, identifier information may be a syntax element or a syntax structure containing one or more syntax elements. For example, identifier information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, identifier information may include dr_id, but is not limited thereto.

[0283] The above purpose information may indicate the purpose of a DR-related SEI message. For example, the value of the purpose information may be in the range of 0 to 1 (inclusive). Additionally, values ​​for the purpose information in the range of 2 to 255 (inclusive) may be reserved for future use.

[0284] The above-mentioned purpose information may take various forms and may be expressed by various names. For example, the purpose information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the purpose information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the purpose information may include dr_purpose, but is not limited thereto.

[0285] According to one example, the purpose information may be configured so that the DR-related SEI message indicates thumbnail information of the original picture or indicates spatial area information of the original picture. For example, if the value of the purpose information is 0, the DR-related SEI message may be interpreted as indicating a thumbnail of the original video content. Accordingly, the decoding device may generate or obtain a thumbnail picture from the original picture based on rectangle-related information (e.g., location, size, etc.) included in the DR-related SEI message. On the other hand, if the value of the purpose information is 1, the DR-related SEI message may be interpreted as indicating a spatial portion of the original video content, and the decoding device may utilize the DR-related SEI message to identify a specific spatial area of ​​the original picture or to construct a display picture according to the purpose.

[0286] Meanwhile, values ​​of the target information in the range of 2 to 255 (inclusive) may be reserved for future use and may not be included in the bitstream conforming to the present disclosure. However, considering future specification extensibility or interoperability, a decoding device conforming to the present disclosure may allow the target information to have any value within the range of 0 to 255 (inclusive). For example, the decoding device may not stop processing the bitstream even when a reserved value is received, and may omit processing for the corresponding target value or treat it as undefined.

[0287] For example, based on the objective information indicating that the DR-related SEI message indicates thumbnail information of the original picture, the DR-related SEI message may include a single display rectangle. That is, when the objective information indicates that the DR-related SEI message indicates thumbnail information of the original picture, the DR-related SEI message may include only information regarding a single display rectangle. This may indicate that the number of display rectangles included in the DR-related SEI message is set to 1 by the display rectangle count information. Since the thumbnail picture can be defined as a single representative area or a single representative picture in this way, information necessary for thumbnail acquisition can be concisely signaled by providing location information and size information of the thumbnail through a single display rectangle rather than describing multiple display rectangles.

[0288] Here, the display rectangle count information may represent the number of display rectangles described within the SEI message.

[0289] The display rectangle count information may take various forms and may be represented by various names. For example, the display rectangle count information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle count information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, the display rectangle count information may include dr_num_rect_minus1, but is not limited thereto.

[0290] For example, if the information on the number of display rectangles is expressed as dr_num_rect_minus1, the value obtained by adding 1 to dr_num_rect_minus1 may represent the number of display rectangles described within the SEI message. Thus, if the destination information indicates that the DR-related SEI message refers to thumbnail information of the original picture, dr_num_rect_minus1 may be 0.

[0291] In addition, if the above-mentioned target information indicates thumbnail information, the DR-related SEI message may not include information regarding filling for the display rectangle, such as information on the existence of a filling method. This can contribute to alleviating implementation complexity by excluding information for display correction that is not essential for defining and acquiring the thumbnail picture, thereby reducing the amount of additional information included in the bitstream and preventing the decoding device from performing additional operations and parameter interpretation for filling processing.

[0292] Fill-related information for a display rectangle may include information on the existence of a display rectangle fill method, information on the display rectangle fill method, information on the display rectangle fill color, information on the display rectangle fill depth, information on the display rectangle fill value, information on the display rectangle fill blur, and / or information on the width of the display rectangle fill content.

[0293] Information regarding the existence of a display rectangle fill method may indicate whether a DR-related SEI message includes information regarding a display rectangle fill method. For example, information regarding the existence of a display rectangle fill method with a value equal to 1 may indicate that a DR-related SEI message includes syntax elements for the rectangle fill method information. Additionally, information regarding the existence of a display rectangle fill method with a value equal to 0 may indicate that a DR-related SEI message does not include syntax elements for the display rectangle fill method information. However, this is not limited thereto, and what is indicated by information regarding the existence of a display rectangle fill method with a value equal to 1 may be interchangeable with what is indicated by information regarding the existence of a display rectangle fill method with a value equal to 0.

[0294] Information regarding the existence of a display rectangle fill method may take various forms and may be represented by various names. For example, information regarding the existence of a display rectangle fill method may be a syntax element or a syntax structure containing one or more syntax elements. For example, information regarding the existence of a display rectangle fill method may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, information regarding the existence of a display rectangle fill method may include, but is not limited to, dr_fill_method_present_flag[ i ], dr_fill_method_present_idc[ i ], prs_fill_method_present_flag[ i ], or prs_fill_method_present_idc[ i ].

[0295] For example, if the information on the existence of a display rectangle fill method is represented by dr_fill_method_present_flag[ i ], a dr_fill_method_present_flag[ i ] equal to 1 may indicate that the dr_fill_method_idc[ i ] syntax element exists. Also, a dr_fill_method_present_flag[ i ] equal to 0 may indicate that the dr_fill_method_idc[ i ] syntax element does not exist.

[0296] For example, if the information on the existence of a display rectangle fill method is represented by prs_fill_method_present_flag[ i ], prs_fill_method_present_flag[ i ] equal to 1 may indicate that the dr_fill_method_idc[ i ] syntax element exists. Also, prs_fill_method_present_flag[ i ] equal to 0 may indicate that the dr_fill_method_idc[ i ] syntax element does not exist.

[0297] The display rectangle filling method information may indicate a filling method applied to the display rectangle, if present. For example, during the process of the display rectangle forming the display picture, a blank area may occur due to a difference in size and / or shape between the output area and the cropped decoding picture area, and the display rectangle filling method information may include a method for how to fill the blank area. For example, the display rectangle filling method information may indicate a filling method applied to the rectangle as defined in Table 3 above.

[0298] The display rectangle fill method information may take various forms and may be represented by various names. For example, the display rectangle fill method information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle fill method information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the display rectangle fill method information may include dr_fill_method_idc[ i ], but is not limited thereto.

[0299] For example, if the display rectangle fill method information is represented by dr_fill_method_idc[ i ], and the value does not exist, the value of dr_fill_method_idc[ i ] can be inferred to be equal to 0. That is, if the decoding device does not obtain a separate fill method from the DR-related SEI message, the decoding device may apply a constant value-based fill method to the blank area by default. Additionally, dr_fill_method_idc[ i ] equal to 1 may represent a spatial extrapolation fill method, and dr_fill_method_idc[ i ] equal to 2 may represent a replication with scaling and blurring fill method.

[0300] Additionally, in a bitstream suitable for the present disclosure, the value of dr_fill_method_idc[ i ] may be a value within the range of 0 to 2 (inclusive). Meanwhile, among the values ​​of dr_fill_method_idc[ i ], 3 to 8 (inclusive) may be reserved values ​​for future use. In the present disclosure, it may be required that the value of dr_fill_method_idc[ i ] be within the range of 0 to 2, but the decoding device may allow the value of dr_fill_method_idc[ i ] to be obtained as a value within the range of 3 to 7 (inclusive), in which case the corresponding fill method may be treated as undefined. That is, the decoding device may determine a fill method for the i-th display rectangle according to the value of dr_fill_method_idc[ i ] and construct a display picture based on the determined fill method.

[0301] Display rectangle fill color information may indicate whether the fill value for the display rectangle is obtained based on multiple color components. For example, depending on the display rectangle fill color information, the sample value applied in the fill process for the display rectangle may be a single color component value or a value for multiple color components (e.g., three color components).

[0302] For example, display rectangle fill color information with a value equal to 1 may indicate that the dr_target_init_sample[ i ] syntax element is included for three color components of the rectangle within a DR-related SEI message. Additionally, display rectangle fill color information with a value equal to 0 may indicate that display rectangle fill value information for one color component is included within a DR-related SEI message. However, this is not limited to the above, and what is indicated by display rectangle fill color information with a value equal to 1 may be interchangeable with what is indicated by display rectangle fill color information with a value equal to 0.

[0303] Display rectangle fill color information may take various forms and may be represented by various names. For example, display rectangle fill color information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill color information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable length bit. For example, display rectangle fill color information may include dr_three_colour_comp_flag[ i ], but is not limited thereto.

[0304] For example, if the display rectangle fill color information is represented by dr_three_colour_comp_flag[ i ], dr_three_colour_comp_flag[ i ], which is equal to 1, may indicate that the dr_target_init_sample[ i ] syntax element will be signaled for each of the three color components of the i-th rectangle. Also, dr_three_colour_comp_flag, which is equal to 0, may indicate that the dr_fill_value[ i ] syntax element will be signaled for one color component.

[0305] Display rectangle fill depth information can represent the bit depth of the fill value for the display rectangle.

[0306] Display rectangle fill depth information may take various forms and may be represented by various names. For example, display rectangle fill depth information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill depth information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, display rectangle fill depth information may include dr_fill_bit_depth_minus8[ i ], but is not limited thereto.

[0307] For example, if the display rectangle fill depth information is represented as dr_fill_bit_depth_minus8[ i ], the value obtained by adding 8 to dr_fill_bit_depth_minus8[ i ] may represent the bit depth of the fill value for the i-th display rectangle. For example, dr_fill_bit_depth_minus8[ i ] may be in the range of 0 to 8 (inclusive), but this is merely illustrative and is not limited to values ​​within the above range.

[0308] The display rectangle fill value information can represent the fill value for the color component of the rectangle.

[0309] The display rectangle fill value information may take various forms and may be represented by various names. For example, the display rectangle fill value information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle fill value information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the display rectangle fill value information may include dr_fill_value[ i ][ j ], but is not limited thereto.

[0310] For example, if the display rectangle fill depth information is represented by dr_fill_value[ i ][ j ], dr_fill_value[ i ][ j ] may represent the fill value for the j-th color component of the i-th rectangle. For example, the length of the dr_fill_value[ i ][ j ] syntax element may be initBitDepth[ i ] bits. Here, the variable initBitDepth[ i ] may be set to equal to dr_fill_bit_depth_minus8[ i ] + 8 based on the display rectangle fill depth information. For example, if the display rectangle fill depth information does not exist, the value of dr_fill_value

[0000] is inferred to be equal to 0, and the values ​​of dr_fill_value

[0001] and dr_fill_value

[0002] may be inferred to be equal to 1 << (BitDepth - 1 ).

[0311] Display rectangle fill blur information can represent the sigma parameter of the Gaussian blur to be applied to the fill of the display rectangle.

[0312] Display rectangle fill blur information may take various forms and may be represented by various names. For example, display rectangle fill blur information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill blur information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, display rectangle fill blur information may include dr_fill_blur_sigma[ i ], but is not limited thereto.

[0313] For example, if the display rectangle fill blur information is represented as dr_fill_blur_sigma[ i ], dr_fill_blur_sigma[ i ] may represent the sigma parameter of the Gaussian blur to be applied to the fill of the i-th display rectangle.

[0314] Display rectangle fill content width information can indicate the width of the content area used to fill the display rectangle.

[0315] Display rectangle fill content width information may take various forms and may be represented by various names. For example, display rectangle fill content width information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill content width information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, display rectangle fill content width information may include dr_fill_content_width[ i ], but is not limited thereto.

[0316] For example, if the display rectangle fill content width information is represented as dr_fill_content_width[ i ], dr_fill_content_width[ i ] may represent the width of the content area used to fill the i-th display rectangle in units of DrUnitSizeInit. Here, if dr_pos_updates_flag described later is equal to 0, the variable DrUnitSizeInit may be set to be equal to DrUnitSize. If such information does not exist, the value of dr_fill_content_width[ i ] may be set to the value of the corresponding syntax element within the previous DR-related SEI message in the output order.

[0317] The display rectangle location information may include the display rectangle top-left x-coordinate information, the display rectangle top-left x-coordinate sign information, the display rectangle top-left y-coordinate information, the display rectangle top-left y-coordinate sign information, the display rectangle fill area top-left x-coordinate information, and / or the display rectangle fill area top-left y-coordinate information.

[0318] The display rectangle top-left x-coordinate information can represent the top-left x-coordinate of the rectangle.

[0319] The display rectangle top-left x-coordinate information may take various forms and be represented by various names. For example, the display rectangle top-left x-coordinate information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle top-left x-coordinate information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the display rectangle top-left x-coordinate information may include dr_rect_top_left_x[ i ], but is not limited thereto.

[0320] For example, if the top-left x-coordinate information of a display rectangle is represented as dr_rect_top_left_x[ i ], dr_rect_top_left_x[ i ] can represent the top-left x-coordinate of the i-th rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0321] The y-coordinate information of the top-left corner of the display rectangle can represent the top-left x-coordinate of the rectangle.

[0322] The y-coordinate information of the top-left corner of the display rectangle may take various forms and be represented by various names. For example, the y-coordinate information of the top-left corner of the display rectangle may be a syntax element or a syntax structure containing one or more syntax elements. For example, the y-coordinate information of the top-left corner of the display rectangle may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the y-coordinate information of the top-left corner of the display rectangle may include dr_rect_top_left_y[ i ], but is not limited thereto.

[0323] For example, if the y-coordinate information of the top-left corner of a display rectangle is represented as dr_rect_top_left_y[ i ], dr_rect_top_left_y[ i ] can represent the y-coordinate of the top-left corner of the i-th rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0324] The sign information for the top-left x-coordinate of the display rectangle can indicate the sign to be applied to the top-left x-coordinate of the display rectangle.

[0325] The sign information for the top-left x-coordinate of a display rectangle can take various forms and be represented by various names. For example, the sign information for the top-left x-coordinate of a display rectangle may be a syntax element or a syntax structure containing one or more syntax elements. For example, the sign information for the top-left x-coordinate of a display rectangle may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an integer without a sign consisting of two or more bits or a variable-length bit. For example, the sign information for the top-left x-coordinate of a display rectangle may include dr_rect_top_left_x_sign[ i ], but is not limited thereto.

[0326] For example, if the sign information for the top-left x-coordinate of a display rectangle is represented as dr_rect_top_left_x_sign[ i ], dr_rect_top_left_x_sign[ i ] can represent the sign to be applied to the top-left x-coordinate of the i-th rectangle.

[0327] The sign information of the top-left y-coordinate of the display rectangle can indicate the sign to be applied to the top-left y-coordinate of the display rectangle.

[0328] The sign information for the top-left y-coordinate of the display rectangle may take various forms and be represented by various names. For example, the sign information for the top-left y-coordinate of the display rectangle may be a syntax element or a syntax structure containing one or more syntax elements. For example, the sign information for the top-left y-coordinate of the display rectangle may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an integer without a sign consisting of two or more bits or a variable-length bit. For example, the sign information for the top-left y-coordinate of the display rectangle may include dr_rect_top_left_y_sign[ i ], but is not limited thereto.

[0329] For example, if the sign information for the top-left y-coordinate of a display rectangle is represented as dr_rect_top_left_y_sign[ i ], dr_rect_top_left_y_sign[ i ] can represent the sign to be applied to the top-left y-coordinate of the i-th rectangle.

[0330] The top-left x-coordinate information of the display rectangle fill area can represent the top-left x-coordinate of the area used for filling the display rectangle.

[0331] The x-coordinate information of the top-left corner of the display rectangle fill area may take various forms and may be represented by various names. For example, the x-coordinate information of the top-left corner of the display rectangle fill area may be a syntax element or a syntax structure containing one or more syntax elements. For example, the x-coordinate information of the top-left corner of the display rectangle fill area may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the x-coordinate information of the top-left corner of the display rectangle fill area may include dr_fill_content_region_top_left_x[ i ], but is not limited thereto.

[0332] For example, if the top-left x-coordinate information of the display rectangle fill area is expressed as dr_fill_content_region_top_left_x[ i ], dr_fill_content_region_top_left_x[ i ] may represent the top-left x-coordinate of the area used for filling the i-th display rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0333] The top-left y-coordinate information of the display rectangle filling area can represent the top-left y-coordinate of the area used for filling the display rectangle.

[0334] The y-coordinate information of the top-left corner of the display rectangle fill area may take various forms and may be represented by various names. For example, the y-coordinate information of the top-left corner of the display rectangle fill area may be a syntax element or a syntax structure containing one or more syntax elements. For example, the y-coordinate information of the top-left corner of the display rectangle fill area may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the y-coordinate information of the top-left corner of the display rectangle fill area may include dr_fill_content_region_top_left_y[ i ], but is not limited thereto.

[0335] For example, if the top-left y-coordinate information of the display rectangle filling area is expressed as dr_fill_content_region_top_left_y[ i ], dr_fill_content_region_top_left_y[ i ] may represent the top-left y-coordinate of the area used for filling the i-th display rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0336] Meanwhile, the DR-related SEI message may include rectangle-based thumbnail existence information to indicate the definition method of the thumbnail picture. Based on the rectangle-based thumbnail existence information, the decoding device may determine whether the thumbnail picture is determined by the rectangle area information included in the DR-related SEI message or by the entire picture to which the DR-related SEI message is applied. That is, based on the rectangle-based thumbnail existence information, the thumbnail picture may be determined by the rectangle area information included in the DR-related SEI message or by the entire picture to which the DR-related SEI message is applied.

[0337] For example, based on the fact that the rectangle-based thumbnail existence information indicates that the thumbnail picture is determined by the rectangle area information, the DR-related SEI message may include rectangle coordinate information, rectangle size information, and rectangle fill information. That is, when the rectangle-based thumbnail existence information indicates that the thumbnail picture is determined by the rectangle area information, the DR-related SEI message may be configured to include parameters for obtaining the thumbnail picture from a specific spatial area of ​​the original picture.

[0338] Here, the rectangle-based thumbnail presence information can indicate whether syntax elements for settings related to the position of the display rectangle exist in the DR-related SEI message. That is, the decoding device can determine whether rectangle area information is included in the DR-related SEI message based on the rectangle-based thumbnail presence information. For example, rectangle-based thumbnail presence information with a value equal to 1 can indicate that syntax elements regarding display rectangle size information, display rectangle position information, display rectangle unit information, display rectangle length information, and / or display rectangle update information may be included in the DR-related SEI message. Additionally, rectangle-based thumbnail presence information with a value equal to 1 can indicate that syntax elements regarding display rectangle fill information may optionally be included in the DR-related SEI message. Additionally, square-based thumbnail existence information with a value equal to 0 may indicate that syntax elements regarding display square size information, display square position information, display square unit information, display square length information, display square update information and / or display square fill information are not included in the DR-related SEI message.

[0339] Rectangle-based thumbnail presence information may take various forms and may be represented by various names. For example, rectangle-based thumbnail presence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, rectangle-based thumbnail presence information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, rectangle-based thumbnail presence information may include dr_rect_info_present_flag or dr_rect_info_present_flag[ i ], but is not limited thereto.

[0340] Specifically, the above DR-related SEI message may include square coordinate information, for example, display square top-left x-coordinate information and display square top-left y-coordinate information indicating the coordinates of the top-left corner of the square, and display square top-left x-coordinate sign information and display square top-left y-coordinate sign information indicating the sign of each coordinate. Additionally, the above DR-related SEI message may include square size information, for example, display square width information and / or display square height information (or a height value derived according to aspect ratio information) indicating the width and height of the square. Accordingly, the decoding device can extract a square area defined by the coordinate information and size information from the original picture and generate a thumbnail picture from the extracted area.

[0341] For example, display rectangle size information may include display rectangle width information and / or display rectangle height information.

[0342] Display rectangle width information can be expressed in units of rectangle width. If such information is not present, the value of the display rectangle width information can be inferred to be the same as the value of the corresponding syntax element within the previous display rectangle-related SEI message in terms of output order.

[0343] Display rectangle width information may take various forms and may be represented by various names. For example, display rectangle width information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle width information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, display rectangle width information may include dr_rect_width[ i ], but is not limited thereto.

[0344] For example, if the display rectangle width information is represented as dr_rect_width[ i ], dr_rect_width[ i ] can represent the width of the i-th display rectangle in units.

[0345] The display rectangle height information can represent the height of the rectangle if such information exists. If such information does not exist, the value of the display rectangle height information can be inferred to be the same as the value of the corresponding syntax element within the previous display rectangle-related SEI message in terms of output order.

[0346] Display rectangle height information may take various forms and may be represented by various names. For example, display rectangle height information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle height information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, display rectangle height information may include dr_rect_height[ i ], but is not limited thereto.

[0347] For example, if the display rectangle height information is represented by dr_rect_height[ i ], and such information exists, dr_rect_height[ i ] can represent the height of the i-th rectangle in units of the variable DrUnitSizeInit. Here, the variable DrUnitSizeInit can be set to be equal to DrUnitSize, which is determined according to the above, when dr_pos_updates_flag described later is equal to 0. If such information does not exist and the display rectangle aspect ratio is equal to 0, the value of dr_rect_height[ i ] can be inferred to be equal to the value of the corresponding syntax element in the previous display rectangle-related SEI message in terms of output order.

[0348] Additionally, the display rectangle ratio information may represent the display aspect ratio of the display rectangle. For example, the display rectangle ratio information may be specified as one of a plurality of predefined aspect ratios, and the horizontal and vertical components of the aspect ratio corresponding to each display rectangle may be determined according to the value of the display rectangle ratio information.

[0349] Display aspect ratio information may take various forms and may be represented by various names. For example, display aspect ratio information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display aspect ratio information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or variable-length bits. For example, display aspect ratio information may include dr_display_aspect_ratio[ i ], but is not limited thereto.

[0350] For example, if the display rectangle ratio information is represented as dr_display_aspect_ratio[ i ], dr_display_aspect_ratio[ i ] may represent the display aspect ratio of the i-th display rectangle. Meanwhile, the value of dr_display_aspect_ratio[ i ] may be interpreted as specified in Table 2 above. For example, if the value of dr_display_aspect_ratio[ i ] is 1, the aspect ratio of the i-th display rectangle may be set to 16:9; if the value of dr_display_aspect_ratio[ i ] is 2, the aspect ratio may be set to 5:4; and if the value of dr_display_aspect_ratio[ i ] is 5, the aspect ratio may be set to 1:1. Additionally, if the value of dr_display_aspect_ratio[ i ] is 0, the aspect ratio may be interpreted as undefined. If the value of dr_display_aspect_ratio[ i ] is in the range of 10 to 15, such values ​​may be reserved values ​​that are not used in the bitstream of the present disclosure or are reserved for future expansion.

[0351] Display square unit information can indicate the granularity of the display square size and position.

[0352] Display square unit information may take various forms and may be represented by various names. For example, display square unit information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display square unit information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable length bit. For example, display square unit information may include dr_log2_unit_size, etc., but is not limited thereto.

[0353] Display rectangle length information can represent the lengths of syntax elements of display rectangle position information and / or display rectangle size information.

[0354] Display rectangle length information may take various forms and may be represented by various names. For example, display rectangle length information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle length information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable length bit, or an unsigned integer consisting of two or more bits or a variable length bit. For example, display rectangle length information may include dr_rect_size_len_minus1, but is not limited thereto.

[0355] For example, if the display rectangle length information is represented by dr_rect_size_len_minus1, the value obtained by adding 1 to dr_rect_size_len_minus1 can represent the lengths of the syntax elements dr_rect_top_left_x[ i ], dr_rect_top_left_y[ i ], dr_rect_width[ i ], dr_rect_height[ i ], dr_fill_top_left_x[ i ], dr_fill_top_left_y[ i ], dr_fill_width[ i ] and dr_fill_height[ i ].

[0356] Additionally, display rectangle update information may indicate whether the DR-related SEI message is a message that provides an explanation of the display rectangle information, or a message that includes an update to the existing display rectangle.

[0357] For example, display rectangle update information with a value equal to 1 may indicate that a DR-related SEI message provides a description of one or more display rectangles used to form a display picture from a cropped decoded picture. Additionally, display rectangle update information with a value equal to 0 may indicate that a DR-related SEI message includes updates for one or more display rectangles. However, this is not limited to these, and what is indicated by display rectangle update information with a value equal to 1 may be interchangeable with what is indicated by display rectangle update information with a value equal to 0.

[0358] Display rectangle update information may take various forms and may be represented by various names. For example, display rectangle update information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle update information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, display rectangle update information may include dr_pos_updates_flag or dr_pos_updates_idc, but is not limited thereto.

[0359] Furthermore, the above DR-related SEI message may include, as necessary, information on the existence of a display rectangle fill method, information on the display rectangle fill method, information on the color of the display rectangle fill, information on the depth of the display rectangle fill, information on the value of the display rectangle fill, information on the blur of the display rectangle fill, and / or information on the width of the content of the display rectangle fill. The above display rectangle fill information may be utilized, for example, in the process of expanding a rectangular area extracted from an original picture to match the display resolution (upscaling), or in the process of correcting the outer edges or empty areas of the extracted area.

[0360] Meanwhile, the DR-related SEI message may include persistence information, and said persistence information may indicate to which range of pictures the DR-related information signaled by the DR-related SEI message applies. That is, based on the value of the persistence information, the decoding device may determine whether to limit the scope of application of the DR-related SEI message to the current picture or to extend it to subsequent pictures in the output order. The said persistence information may indicate whether the DR-related SEI message applies only to the current picture, or to the current picture and subsequent pictures in the output order based on satisfying a predefined termination condition. For example, persistence information with a value equal to 1 may indicate that the DR-related SEI message applies to the current picture and all pictures after the current layer in the output order. Additionally, persistence information with a value equal to 0 may indicate that the DR-related SEI message applies only to the current picture.

[0361] The above persistence information may take various forms and may be represented by various names. For example, the persistence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the persistence information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, the persistence information may include dr_persistence_flag, but is not limited thereto.

[0362] For example, if persistence information is represented by dr_persistence_flag and dr_persistence_flag is equal to 0, the decoding device can determine that the DR-related SEI message applies only to the current picture. In this case, the DR-related information included in the DR-related SEI message can be configured to apply only to the current picture. Accordingly, it is possible to provide a thumbnail definition or a display rectangle definition for a specific picture on a one-time basis.

[0363] Additionally, if dr_persistence_flag is equal to 1, the decoding device may determine that the DR-related SEI message is applied to the current picture and subsequent pictures of the current layer in the output order. In this case, the DR-related information may be configured to be continuously applied to subsequent pictures as well. At this time, the application may be maintained until at least one of a predefined termination condition is satisfied. For example, the termination condition may include at least one of the following: when a new encoded layered image sequence (CLVS) starts in the current layer, when the bitstream ends, or when a picture associated with the DR-related SEI message is output in the current layer, and said picture corresponds to a subsequent picture of the current picture in the output order.

[0364] By providing such persistence information, the repeated signaling of the same DR-related information for multiple consecutive pictures can be reduced, and the decoding device can obtain thumbnail pictures or apply display rectangles to subsequent pictures in the same way by referring to the DR-related SEI message.

[0365] The information regarding the display rectangle described above can be obtained based on the display rectangle cancellation information. That is, the decoding device can check the display rectangle cancellation information during the process of parsing the DR-related SEI message from the bitstream, and determine whether to obtain and the scope of application of the display rectangle-related information according to the value of the cancellation information.

[0366] Here, the display square cancellation information may indicate whether to apply persistence to DR-related SEI messages. For example, a display square cancellation information with a value equal to 1 may indicate that the persistence of any previous DR-related SEI messages is canceled. Additionally, a display square cancellation information with a value equal to 0 may indicate that display square information follows. However, this is not limited to this, and what a display square cancellation information with a value equal to 1 indicates may be different from what a display square cancellation information with a value equal to 0 indicates.

[0367] Display rectangle cancellation information may take various forms and may be represented by various names. For example, display rectangle cancellation information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle cancellation information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, display rectangle cancellation information may include dr_cancel_flag, dr_rect_cancel_flag[ i ], or dr_cancel_idc, but is not limited thereto.

[0368] According to one embodiment, a DR-related SEI message may further include DR-related information for each of a plurality of display rectangles to provide definitions of display rectangles that can be applied to one picture or a plurality of pictures. Here, since the DR-related information can be provided independently for each display rectangle, it is possible to describe a plurality of display rectangles simultaneously within a single DR-related SEI message while assigning different operations and meanings to each display rectangle.

[0369] Specifically, the DR-related information may include rectangle identifier information for distinguishing each display rectangle. Based on the rectangle identifier information, the decoding device may track the display rectangle corresponding to the same identifier within a DR-related SEI message, or identify the same display rectangle between different DR-related SEI messages to perform associated processing. Accordingly, even when multiple display rectangles exist, the processing unit of each rectangle can be clearly distinguished.

[0370] The above rectangular identifier information may include an identification number that can be used to identify DR-related SEI messages. For example, the value of the rectangular identifier information may be within the range of 0 to 16,383 (inclusive).

[0371] Rectangle identifier information may take various forms and may be represented by various names. For example, the rectangle identifier information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the rectangle identifier information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, the rectangle identifier information may include dr_rect_id[ i ], but is not limited thereto.

[0372] For example, if rectangle identifier information is represented as dr_rect_id[ i ], dr_rect_id[ i ] may include an identification number that can be used to identify the i-th display rectangle. The value of dr_rect_id[ i ] may be in the range of 0 to 16383 (inclusive).

[0373] Additionally, the DR-related SEI message may further include square purpose information corresponding to each of a plurality of display squares. The square purpose information is information intended to indicate what purpose each display square is used for, and the decoding device may interpret and process each display square with a different meaning based on the square purpose information. For example, the square purpose information corresponding to each of a plurality of display squares may be configured so that the DR-related SEI message indicates thumbnail information of the original picture or indicates spatial area information of the original picture.

[0374] The above rectangular purpose information may indicate the purpose of the display rectangle. For example, the value of the rectangular purpose information may be in the range of 0 to 1 (inclusive). Additionally, values ​​in the range of 2 to 255 (inclusive) for the value of the rectangular purpose information may be reserved for future use.

[0375] Rectangular purpose information may take various forms and may be represented by various names. For example, rectangular purpose information may be a syntax element or a syntax structure containing one or more syntax elements. For example, rectangular purpose information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, rectangular purpose information may include dr_rect_purpose[ i ], but is not limited thereto.

[0376] For example, if the rectangle purpose information is represented as dr_rect_purpose[ i ], dr_rect_purpose[ i ] can represent the purpose of the i-th display rectangle.

[0377] Specifically, when the value of the rectangle purpose information is 0, the display rectangle can be interpreted as indicating a thumbnail of the original video content. Accordingly, the decoding device can generate or obtain a thumbnail picture from the original picture based on rectangle-related information (e.g., position, size, etc.) corresponding to the display rectangle. On the other hand, when the value of the rectangle purpose information is 1, the display rectangle can be interpreted as indicating a spatial portion of the original video content, and the decoding device can identify the area indicated by the display rectangle as a specific spatial area of ​​the original picture or use it to construct the display picture.

[0378] Meanwhile, values ​​of the rectangular target information in the range of 2 to 255 (inclusive) are reserved for future use and may not be included in the bitstream conforming to the present disclosure. However, considering future specification extensibility or interoperability, a decoding device conforming to the present disclosure may be configured to allow any value of the rectangular target information within the range of 0 to 255 (inclusive) to appear in the syntax. For example, the decoding device may not stop bitstream processing even when a reserved value is received, and may omit processing corresponding to the target value or treat it as an undefined value.

[0379] Additionally, the definition method of the thumbnail can be specified on a per-display rectangle basis. For example, based on the fact that rectangle target information corresponding to each of a plurality of display rectangles indicates that each display rectangle indicates thumbnail information of the original picture, the DR-related SEI message may not include filling-related information for the display rectangles, such as information on the existence of a filling method. This may indicate that the information on the existence of a filling method is disabled, so additional syntax elements regarding the filling processing of the display rectangles may not be signaled. Accordingly, by excluding display correction information that is not essential for the definition and acquisition of the thumbnail picture, the amount of additional information included in the bitstream is reduced, and implementation complexity can be mitigated by preventing the decoding device from performing additional operations and parameter interpretation for filling processing.

[0380] Meanwhile, the DR-related SEI message may include rectangle-based thumbnail existence information to indicate a definition method for a thumbnail picture corresponding to each of a plurality of display rectangles. Based on the rectangle-based thumbnail existence information, the decoding device can determine whether the thumbnail picture is defined by the corresponding display rectangle or whether the entire picture to which the DR-related SEI message is applied is defined as the thumbnail picture. That is, the thumbnail picture may be determined by the rectangle area information included in the DR-related SEI message, or determined as the entire picture to which the DR-related SEI message is applied, based on the rectangle-based thumbnail existence information for each of the plurality of display rectangles.

[0381] For example, if the rectangle-based thumbnail existence information for each of the plurality of display rectangles indicates that the thumbnail picture is determined by the rectangle area information, the decoding device can derive variables for configuring the thumbnail picture (e.g., derived variables related to rectangle coordinates / size) based on the rectangle coordinate information and rectangle size information included in the DR-related SEI message. That is, if the rectangle-based thumbnail existence information indicates a rectangle area-based thumbnail definition, the DR-related SEI message may be configured to include parameters for obtaining the thumbnail picture from a specific spatial area of ​​the original picture.

[0382] Additionally, the rectangle-based thumbnail existence information may indicate whether syntax elements regarding the position and size of a display rectangle exist within a DR-related SEI message, and the decoding device may derive variables representing the position and size of the display rectangle (e.g., DrRectTopLeftX[ i ], DrRectTopLeftY[ i ], DrRectWidth[ i ], DrRectHeight[ i ]) from the syntax elements. Accordingly, the decoding device may determine whether rectangle area information is provided within a DR-related SEI message based on the rectangle-based thumbnail existence information.

[0383] For example, if rectangle-based thumbnail presence information is represented by dr_rect_info_present_flag[ i ] and dr_rect_info_present_flag[ i ] is equal to 1, the display rectangle area indicated by the variables DrRectWidth[ i ], DrRectHeight[ i ], DrRectTopLeftX[ i ], and DrRectTopLeftY[ i ] can be determined as a thumbnail image or thumbnail video. That is, the decoding device can obtain a thumbnail picture by extracting from the original picture an area defined by the top-left coordinates and size of the display rectangle. In this case, the DR-related SEI message may be configured to include rectangle coordinate information and rectangle size information to indicate the display rectangle area, and filling-related information may be optionally included as needed.

[0384] On the other hand, if dr_rect_info_present_flag[ i ] is equal to 0, the picture(s) to which the DR-related SEI message is applied can be determined as a thumbnail image or thumbnail video. That is, in this case, even if separate display rectangle area information is not provided, the decoding device can treat the entire picture to which the DR-related SEI message is applied as a thumbnail picture.

[0385] The information regarding the display rectangles described above can be obtained based on display rectangle cancellation information for each of the multiple display rectangles. That is, in the process of parsing a DR-related SEI message from a bitstream, the decoding device can identify display rectangle cancellation information corresponding to each display rectangle and determine whether to obtain information regarding the display rectangles and the scope of application based on the value of the display rectangle cancellation information.

[0386] Here, the display rectangle cancel information may indicate whether to apply persistence to the SEI message related to the DR. The display rectangle cancel information may take various forms and may be represented by various names. For example, the display rectangle cancel information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle cancel information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the display rectangle cancel information may include, but is not limited to, dr_rect_cancel_flag[ i ].

[0387] For example, if the display rectangle cancellation information is represented by dr_rect_cancel_flag[ i ], dr_rect_cancel_flag[ i ] can indicate whether the information regarding the i-th display rectangle persists, that is, whether the definition for that display rectangle is cancelled.

[0388] The decoding device can obtain a thumbnail picture from the original picture based on DR-related information included in a DR-related SEI message (S520). For example, the processor of the decoding device can obtain a thumbnail picture from the original picture based on DR-related information included in a DR-related SEI message obtained from a bitstream. Here, the DR-related information may include parameters necessary for obtaining the thumbnail picture, such as information on the number of display rectangles, coordinate and size information of the display rectangles, purpose information of the display rectangles, existence information of rectangle-based thumbnails, and / or persistence information.

[0389] According to one embodiment, the decoding device may determine whether a DR-related SEI message indicates a thumbnail of an original picture based on purpose information (e.g., dr_purpose or dr_rect_purpose[ i ]). If the purpose information is determined to indicate a thumbnail, the decoding device may determine whether the thumbnail picture is defined by a display rectangular area or by the entire picture to which the DR-related SEI message is applied by referring to rectangle-based thumbnail presence information (e.g., dr_rect_info_present_flag or dr_rect_info_present_flag[ i ]).

[0390] For example, if the existence information of a rectangle-based thumbnail indicates a definition of a thumbnail based on a rectangle area, the decoding device can extract a specific spatial area of ​​the original picture defined by the syntax elements related to rectangle coordinates and the syntax elements related to rectangle size included in the SEI message related to DR, and obtain it as a thumbnail picture. Additionally, if the existence information of a rectangle-based thumbnail indicates a definition of a thumbnail based on a rectangle area, the decoding device can derive variables (e.g., DrRectTopLeftX, DrRectTopLeftY, DrRectWidth, DrRectHeight) representing the top-left coordinates and size of the display rectangle based on the syntax elements related to rectangle coordinates and the syntax elements related to rectangle size included in the SEI message related to DR. The decoding device can extract a specific spatial area of ​​the original picture defined by the derived variables and obtain it as a thumbnail picture. At this time, if necessary, the decoding device may correct the outer edge or empty area of ​​the extraction area based on information related to the filling of the display rectangle, or perform scaling, extrapolation, blurring, etc. However, if the target information indicates a thumbnail, the filling-related information may be omitted, and the process of obtaining a thumbnail may be simplified accordingly.

[0391] On the other hand, if the existence information of a rectangle-based thumbnail indicates a thumbnail definition based on the entire picture, the decoding device can acquire the entire original picture(s) to which the DR-related SEI message applies as a thumbnail picture, even if separate display rectangle coordinates and / or size information is not provided. Accordingly, a thumbnail provision method, such as a full-frame-based thumbnail image or preview video, can be implemented with concise signaling.

[0392] Additionally, the decoding device can determine the scope of application of DR-related information included in a DR-related SEI message based on persistence information. For example, if the persistence information indicates that it is applied only to the current picture, the DR-related information is applied only to the current picture, and if the persistence information indicates that it is applied to subsequent pictures, the DR-related information is applied to subsequent pictures in the output order until a predefined termination condition is satisfied, thereby allowing the use of a rule to obtain the same thumbnail across multiple pictures.

[0393] Accordingly, the decoding device can efficiently obtain a thumbnail picture from the original picture based on DR-related information included in the DR-related SEI message, and by flexibly controlling the definition method of the thumbnail (e.g., definition based on a rectangular area or based on the entire picture, etc.) and the scope of application of the DR-related information (e.g., whether it is applied only to the current picture or to subsequent pictures until a predefined termination condition is satisfied, etc.), it can provide consistent thumbnails in various service and terminal environments.

[0394] In addition, the present invention integrates thumbnail extraction information, which was previously provided separately as a TEI-related SEI message, into a DR-related SEI message (message unit or display rectangle unit), thereby enabling a decoding device to consistently determine a thumbnail based on a rectangle area or the entire picture based on identifier information, purpose information, persistence information, cancellation information, and rectangle-based thumbnail existence information included in the DR-related SEI message. Furthermore, if necessary, the decoding device can derive parameters such as the coordinates and size of the rectangle to efficiently obtain a thumbnail picture from the original picture, and can alleviate implementation complexity by reducing repetitive parsing and unnecessary processing.

[0395] The names of syntax elements and semantics described in this specification are merely exemplary names for the convenience of explanation, and the scope of the rights of this disclosure is not limited by such names. That is, any configuration that performs the same or substantially corresponding functions as the syntax elements and semantics described in this specification may be included within the spirit and scope of this disclosure, even if its name, notation, or terminology differs.

[0396] FIG. 6 is a diagram illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0397] The encoding method (S600) may include the operations described below.

[0398] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0399] The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the decoding method according to one embodiment, and the previously described operations may be added.

[0400] The sequence of actions illustrated in the drawings regarding the actions to be described below is merely an example, and the actions to be described below may be performed in a different order as long as it does not contradict the causal relationship of the actions to be described.

[0401] The operations described below form a single embodiment integrally with the configurations and / or operations described above, unless they conflict with the configurations and / or operations described above, and do not form a separate embodiment distinct from the configurations and / or operations described above.

[0402] The encoding method (S600) can be executed by an encoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0403] The encoding device can generate SEI (Supplemental Enhancement Information) messages (S610).

[0404] SEI messages can convey specific types of information that assist in processes related to the decoding, display, or other purposes of image information. Here, SEI messages may not be necessary for the decoding process to determine the sample values ​​of the decoded picture.

[0405] For example, SEI messages may include SEI messages related to display rectangles.

[0406] A SEI message related to a display rectangle may indicate one or more display rectangles that can be used to form a display picture from a cropped decoding picture, or may include information regarding such rectangles.

[0407] The SEI message related to the display rectangle may have various names such as display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, DR SEI message, DR related message, DR related information, DR message, DR information, display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, display rectangle SEI message, display rectangle related message, display rectangle related information, display rectangle message, display rectangle information, etc., and the names are not limited.

[0408] SEI messages related to display rectangles can take various forms. For example, SEI messages related to DR can be syntax elements or syntax structures containing one or more syntax elements. Additionally, SEI messages related to display rectangles can be raw byte sequence payloads (RBSP) containing one or more syntax elements or syntax structures. For example, SEI messages related to display rectangles can be expressed as display_rectangles ( payloadSize ), but are not limited thereto.

[0409] The encoding device may generate a display-related SEI message containing information related to a display rectangle. For example, the encoding device may generate DR-related information regarding at least one display rectangle that can be used to construct a display picture from a cropped picture and include it in the DR-related SEI message.

[0410] A DR-related SEI message may include information on the number of display rectangles, information on the ratio of display rectangles, information on the fill of display rectangles, information on the size of display rectangles, information on the position of display rectangles, information on canceling display rectangles, information on the unit of display rectangles, information on the length of display rectangles, and / or information on updating display rectangles. Additionally, a DR-related SEI message may include at least one of identifier information identifying the DR-related SEI message, purpose information indicating the purpose of the DR-related SEI message, persistence information indicating the persistence of the DR-related SEI message, and rectangle-based thumbnail existence information indicating definition information of a thumbnail picture.

[0411] The above identifier information may include an identification number that can be used to identify DR-related SEI messages. For example, the value of the identifier information may be within the range of 0 to 16,383 (inclusive).

[0412] Identifier information may take various forms and may be represented by various names. For example, identifier information may be a syntax element or a syntax structure containing one or more syntax elements. For example, identifier information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, identifier information may include dr_id, but is not limited thereto.

[0413] The above purpose information may indicate the purpose of a DR-related SEI message. For example, the value of the purpose information may be in the range of 0 to 1 (inclusive). Additionally, values ​​for the purpose information in the range of 2 to 255 (inclusive) may be reserved for future use.

[0414] The above-mentioned purpose information may take various forms and may be expressed by various names. For example, the purpose information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the purpose information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the purpose information may include dr_purpose, but is not limited thereto.

[0415] According to one example, the purpose information may be configured so that the DR-related SEI message indicates thumbnail information of the original picture or indicates spatial area information of the original picture. For example, if the value of the purpose information is 0, the DR-related SEI message may be interpreted as indicating a thumbnail of the original video content. Accordingly, the decoding device may generate or obtain a thumbnail picture from the original picture based on rectangle-related information (e.g., location, size, etc.) included in the DR-related SEI message. On the other hand, if the value of the purpose information is 1, the DR-related SEI message may be interpreted as indicating a spatial portion of the original video content, and the decoding device may utilize the DR-related SEI message to identify a specific spatial area of ​​the original picture or to construct a display picture according to the purpose.

[0416] Meanwhile, values ​​of the target information in the range of 2 to 255 (inclusive) may be reserved for future use and may not be included in the bitstream conforming to the present disclosure. However, considering future specification extensibility or interoperability, a decoding device conforming to the present disclosure may allow the target information to have any value within the range of 0 to 255 (inclusive). For example, the decoding device may not stop processing the bitstream even when a reserved value is received, and may omit processing for the corresponding target value or treat it as undefined.

[0417] For example, based on the fact that a DR-related SEI message indicates thumbnail information of the original picture, the encoding device may generate purpose information indicating that the DR-related SEI message includes only information regarding a single display rectangle. Additionally, according to the purpose information, the encoding device may generate display rectangle count information such that the number of display rectangles included in the DR-related SEI message is 1. That is, when a DR-related SEI message indicates thumbnail information of the original picture, the encoding device may construct the DR-related SEI message based on a single display rectangle and signal position information and size information, etc., regarding the single display rectangle. Since the thumbnail picture can be defined as a single representative area or a single representative picture in this way, the encoding device can concisely encode the information necessary for thumbnail acquisition through a single display rectangle rather than describing multiple display rectangles.

[0418] Here, the display rectangle count information may represent the number of display rectangles described within the SEI message.

[0419] The display rectangle count information may take various forms and may be represented by various names. For example, the display rectangle count information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle count information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, the display rectangle count information may include dr_num_rect_minus1, but is not limited thereto.

[0420] For example, if the information regarding the number of display rectangles is expressed as dr_num_rect_minus1, the value obtained by adding 1 to dr_num_rect_minus1 may represent the number of display rectangles described within the SEI message. Thus, if the DR-related SEI message indicates thumbnail information of the original picture, dr_num_rect_minus1 may be set to 0.

[0421] Additionally, when a DR-related SEI message indicates thumbnail information of the original picture, the encoding device may configure the DR-related SEI message not to include information related to filling for the display rectangle, such as information on the existence of a filling method. For example, by disabling information on the existence of a filling method, the encoding device may generate a DR-related SEI message so that additional syntax elements regarding the filling of the display rectangle are not included. This reduces the amount of additional information included in the bitstream by excluding information for display correction that is not essential for defining and acquiring the thumbnail picture during the encoding process, and can alleviate the complexity of the decoder implementation by preventing the decoding device from performing additional operations and parameter interpretation for filling processing.

[0422] Fill-related information for a display rectangle may include information on the existence of a display rectangle fill method, information on the display rectangle fill method, information on the display rectangle fill color, information on the display rectangle fill depth, information on the display rectangle fill value, information on the display rectangle fill blur, and / or information on the width of the display rectangle fill content.

[0423] Information regarding the existence of a display rectangle fill method may indicate whether a DR-related SEI message includes information regarding a display rectangle fill method. For example, information regarding the existence of a display rectangle fill method with a value equal to 1 may indicate that a DR-related SEI message includes syntax elements for the rectangle fill method information. Additionally, information regarding the existence of a display rectangle fill method with a value equal to 0 may indicate that a DR-related SEI message does not include syntax elements for the display rectangle fill method information. However, this is not limited thereto, and what is indicated by information regarding the existence of a display rectangle fill method with a value equal to 1 may be interchangeable with what is indicated by information regarding the existence of a display rectangle fill method with a value equal to 0.

[0424] Information regarding the existence of a display rectangle fill method may take various forms and may be represented by various names. For example, information regarding the existence of a display rectangle fill method may be a syntax element or a syntax structure containing one or more syntax elements. For example, information regarding the existence of a display rectangle fill method may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, information regarding the existence of a display rectangle fill method may include, but is not limited to, dr_fill_method_present_flag[ i ], dr_fill_method_present_idc[ i ], prs_fill_method_present_flag[ i ], or prs_fill_method_present_idc[ i ].

[0425] For example, if the information on the existence of a display rectangle fill method is represented by dr_fill_method_present_flag[ i ], a dr_fill_method_present_flag[ i ] equal to 1 may indicate that the dr_fill_method_idc[ i ] syntax element exists. Also, a dr_fill_method_present_flag[ i ] equal to 0 may indicate that the dr_fill_method_idc[ i ] syntax element does not exist.

[0426] For example, if the information on the existence of a display rectangle fill method is represented by prs_fill_method_present_flag[ i ], prs_fill_method_present_flag[ i ] equal to 1 may indicate that the dr_fill_method_idc[ i ] syntax element exists. Also, prs_fill_method_present_flag[ i ] equal to 0 may indicate that the dr_fill_method_idc[ i ] syntax element does not exist.

[0427] The display rectangle filling method information may indicate a filling method applied to the display rectangle, if present. For example, during the process of the display rectangle forming the display picture, a blank area may occur due to a difference in size and / or shape between the output area and the cropped decoding picture area, and the display rectangle filling method information may include a method for how to fill the blank area. For example, the display rectangle filling method information may indicate a filling method applied to the rectangle as defined in Table 3 above.

[0428] The display rectangle fill method information may take various forms and may be represented by various names. For example, the display rectangle fill method information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle fill method information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the display rectangle fill method information may include dr_fill_method_idc[ i ], but is not limited thereto.

[0429] For example, if the display rectangle fill method information is represented by dr_fill_method_idc[ i ], and the value does not exist, the value of dr_fill_method_idc[ i ] can be inferred to be equal to 0. That is, if the decoding device does not obtain a separate fill method from the DR-related SEI message, the decoding device may apply a constant value-based fill method to the blank area by default. Additionally, dr_fill_method_idc[ i ] equal to 1 may represent a spatial extrapolation fill method, and dr_fill_method_idc[ i ] equal to 2 may represent a replication with scaling and blurring fill method.

[0430] Additionally, in a bitstream suitable for the present disclosure, the value of dr_fill_method_idc[ i ] may be a value within the range of 0 to 2 (inclusive). Meanwhile, among the values ​​of dr_fill_method_idc[ i ], 3 to 8 (inclusive) may be reserved values ​​for future use. In the present disclosure, it may be required that the value of dr_fill_method_idc[ i ] be within the range of 0 to 2, but the decoding device may allow the value of dr_fill_method_idc[ i ] to be obtained as a value within the range of 3 to 7 (inclusive), in which case the corresponding fill method may be treated as undefined. That is, the decoding device may determine a fill method for the i-th display rectangle according to the value of dr_fill_method_idc[ i ] and construct a display picture based on the determined fill method.

[0431] Display rectangle fill color information may indicate whether the fill value for the display rectangle is obtained based on multiple color components. For example, depending on the display rectangle fill color information, the sample value applied in the fill process for the display rectangle may be a single color component value or a value for multiple color components (e.g., three color components).

[0432] For example, display rectangle fill color information with a value equal to 1 may indicate that the dr_target_init_sample[ i ] syntax element is included for three color components of the rectangle within a DR-related SEI message. Additionally, display rectangle fill color information with a value equal to 0 may indicate that display rectangle fill value information for one color component is included within a DR-related SEI message. However, this is not limited to the above, and what is indicated by display rectangle fill color information with a value equal to 1 may be interchangeable with what is indicated by display rectangle fill color information with a value equal to 0.

[0433] Display rectangle fill color information may take various forms and may be represented by various names. For example, display rectangle fill color information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill color information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable length bit. For example, display rectangle fill color information may include dr_three_colour_comp_flag[ i ], but is not limited thereto.

[0434] For example, if the display rectangle fill color information is represented by dr_three_colour_comp_flag[ i ], dr_three_colour_comp_flag[ i ], which is equal to 1, may indicate that the dr_target_init_sample[ i ] syntax element will be signaled for each of the three color components of the i-th rectangle. Also, dr_three_colour_comp_flag, which is equal to 0, may indicate that the dr_fill_value[ i ] syntax element will be signaled for one color component.

[0435] Display rectangle fill depth information can represent the bit depth of the fill value for the display rectangle.

[0436] Display rectangle fill depth information may take various forms and may be represented by various names. For example, display rectangle fill depth information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill depth information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, display rectangle fill depth information may include dr_fill_bit_depth_minus8[ i ], but is not limited thereto.

[0437] For example, if the display rectangle fill depth information is represented as dr_fill_bit_depth_minus8[ i ], the value obtained by adding 8 to dr_fill_bit_depth_minus8[ i ] may represent the bit depth of the fill value for the i-th display rectangle. For example, dr_fill_bit_depth_minus8[ i ] may be in the range of 0 to 8 (inclusive), but this is merely illustrative and is not limited to values ​​within the above range.

[0438] The display rectangle fill value information can represent the fill value for the color component of the rectangle.

[0439] The display rectangle fill value information may take various forms and may be represented by various names. For example, the display rectangle fill value information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle fill value information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the display rectangle fill value information may include dr_fill_value[ i ][ j ], but is not limited thereto.

[0440] For example, if the display rectangle fill depth information is represented by dr_fill_value[ i ][ j ], dr_fill_value[ i ][ j ] may represent the fill value for the j-th color component of the i-th rectangle. For example, the length of the dr_fill_value[ i ][ j ] syntax element may be initBitDepth[ i ] bits. Here, the variable initBitDepth[ i ] may be set to equal to dr_fill_bit_depth_minus8[ i ] + 8 based on the display rectangle fill depth information. For example, if the display rectangle fill depth information does not exist, the value of dr_fill_value

[0000] is inferred to be equal to 0, and the values ​​of dr_fill_value

[0001] and dr_fill_value

[0002] may be inferred to be equal to 1 << (BitDepth - 1 ).

[0441] Display rectangle fill blur information can represent the sigma parameter of the Gaussian blur to be applied to the fill of the display rectangle.

[0442] Display rectangle fill blur information may take various forms and may be represented by various names. For example, display rectangle fill blur information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill blur information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, display rectangle fill blur information may include dr_fill_blur_sigma[ i ], but is not limited thereto.

[0443] For example, if the display rectangle fill blur information is represented as dr_fill_blur_sigma[ i ], dr_fill_blur_sigma[ i ] may represent the sigma parameter of the Gaussian blur to be applied to the fill of the i-th display rectangle.

[0444] Display rectangle fill content width information can indicate the width of the content area used to fill the display rectangle.

[0445] Display rectangle fill content width information may take various forms and may be represented by various names. For example, display rectangle fill content width information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle fill content width information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, display rectangle fill content width information may include dr_fill_content_width[ i ], but is not limited thereto.

[0446] For example, if the display rectangle fill content width information is represented as dr_fill_content_width[ i ], dr_fill_content_width[ i ] may represent the width of the content area used to fill the i-th display rectangle in units of DrUnitSizeInit. Here, if dr_pos_updates_flag described later is equal to 0, the variable DrUnitSizeInit may be set to be equal to DrUnitSize. If such information does not exist, the value of dr_fill_content_width[ i ] may be set to the value of the corresponding syntax element within the previous DR-related SEI message in the output order.

[0447] The display rectangle location information may include the display rectangle top-left x-coordinate information, the display rectangle top-left x-coordinate sign information, the display rectangle top-left y-coordinate information, the display rectangle top-left y-coordinate sign information, the display rectangle fill area top-left x-coordinate information, and / or the display rectangle fill area top-left y-coordinate information.

[0448] The display rectangle top-left x-coordinate information can represent the top-left x-coordinate of the rectangle.

[0449] The display rectangle top-left x-coordinate information may take various forms and be represented by various names. For example, the display rectangle top-left x-coordinate information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the display rectangle top-left x-coordinate information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the display rectangle top-left x-coordinate information may include dr_rect_top_left_x[ i ], but is not limited thereto.

[0450] For example, if the top-left x-coordinate information of a display rectangle is represented as dr_rect_top_left_x[ i ], dr_rect_top_left_x[ i ] can represent the top-left x-coordinate of the i-th rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0451] The y-coordinate information of the top-left corner of the display rectangle can represent the top-left x-coordinate of the rectangle.

[0452] The y-coordinate information of the top-left corner of the display rectangle may take various forms and be represented by various names. For example, the y-coordinate information of the top-left corner of the display rectangle may be a syntax element or a syntax structure containing one or more syntax elements. For example, the y-coordinate information of the top-left corner of the display rectangle may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the y-coordinate information of the top-left corner of the display rectangle may include dr_rect_top_left_y[ i ], but is not limited thereto.

[0453] For example, if the y-coordinate information of the top-left corner of a display rectangle is represented as dr_rect_top_left_y[ i ], dr_rect_top_left_y[ i ] can represent the y-coordinate of the top-left corner of the i-th rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0454] The sign information for the top-left x-coordinate of the display rectangle can indicate the sign to be applied to the top-left x-coordinate of the display rectangle.

[0455] The sign information for the top-left x-coordinate of a display rectangle can take various forms and be represented by various names. For example, the sign information for the top-left x-coordinate of a display rectangle may be a syntax element or a syntax structure containing one or more syntax elements. For example, the sign information for the top-left x-coordinate of a display rectangle may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an integer without a sign consisting of two or more bits or a variable-length bit. For example, the sign information for the top-left x-coordinate of a display rectangle may include dr_rect_top_left_x_sign[ i ], but is not limited thereto.

[0456] For example, if the sign information for the top-left x-coordinate of a display rectangle is represented as dr_rect_top_left_x_sign[ i ], dr_rect_top_left_x_sign[ i ] can represent the sign to be applied to the top-left x-coordinate of the i-th rectangle.

[0457] The sign information of the top-left y-coordinate of the display rectangle can indicate the sign to be applied to the top-left y-coordinate of the display rectangle.

[0458] The sign information for the top-left y-coordinate of the display rectangle may take various forms and be represented by various names. For example, the sign information for the top-left y-coordinate of the display rectangle may be a syntax element or a syntax structure containing one or more syntax elements. For example, the sign information for the top-left y-coordinate of the display rectangle may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an integer without a sign consisting of two or more bits or a variable-length bit. For example, the sign information for the top-left y-coordinate of the display rectangle may include dr_rect_top_left_y_sign[ i ], but is not limited thereto.

[0459] For example, if the sign information for the top-left y-coordinate of a display rectangle is represented as dr_rect_top_left_y_sign[ i ], dr_rect_top_left_y_sign[ i ] can represent the sign to be applied to the top-left y-coordinate of the i-th rectangle.

[0460] The top-left x-coordinate information of the display rectangle fill area can represent the top-left x-coordinate of the area used for filling the display rectangle.

[0461] The x-coordinate information of the top-left corner of the display rectangle fill area may take various forms and may be represented by various names. For example, the x-coordinate information of the top-left corner of the display rectangle fill area may be a syntax element or a syntax structure containing one or more syntax elements. For example, the x-coordinate information of the top-left corner of the display rectangle fill area may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the x-coordinate information of the top-left corner of the display rectangle fill area may include dr_fill_content_region_top_left_x[ i ], but is not limited thereto.

[0462] For example, if the top-left x-coordinate information of the display rectangle fill area is expressed as dr_fill_content_region_top_left_x[ i ], dr_fill_content_region_top_left_x[ i ] may represent the top-left x-coordinate of the area used for filling the i-th display rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0463] The top-left y-coordinate information of the display rectangle filling area can represent the top-left y-coordinate of the area used for filling the display rectangle.

[0464] The y-coordinate information of the top-left corner of the display rectangle fill area may take various forms and may be represented by various names. For example, the y-coordinate information of the top-left corner of the display rectangle fill area may be a syntax element or a syntax structure containing one or more syntax elements. For example, the y-coordinate information of the top-left corner of the display rectangle fill area may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the y-coordinate information of the top-left corner of the display rectangle fill area may include dr_fill_content_region_top_left_y[ i ], but is not limited thereto.

[0465] For example, if the top-left y-coordinate information of the display rectangle filling area is expressed as dr_fill_content_region_top_left_y[ i ], dr_fill_content_region_top_left_y[ i ] may represent the top-left y-coordinate of the area used for filling the i-th display rectangle in units of DrUnitSize. Here, the variable DrUnitSize can be derived as described above.

[0466] Meanwhile, the DR-related SEI message may include rectangle-based thumbnail existence information to indicate the definition method of the thumbnail picture. For example, an encoding device may include the rectangle-based thumbnail existence information in the DR-related SEI message to signal whether the thumbnail picture is defined by rectangle area information or by the entire picture to which the DR-related SEI message is applied.

[0467] Specifically, when a thumbnail picture is defined based on a rectangular area, the encoding device may include rectangular area information, such as rectangular coordinate information and rectangular size information, in a DR-related SEI message so that the thumbnail picture is obtained from a specific spatial area of ​​the original picture. On the other hand, when a thumbnail picture is defined based on the entire picture, the encoding device may omit signaling of separate rectangular area information and configure the entire picture to which the DR-related SEI message applies to be considered as the thumbnail picture. That is, rectangular-based thumbnail existence information may be generated based on the fact that the thumbnail picture is determined by the rectangular area information included in the DR-related SEI message, or is determined by the entire picture to which the DR-related SEI message applies. Accordingly, the encoding device may generate rectangular-based thumbnail existence information indicating that the thumbnail picture is defined by the rectangular area information included in the DR-related SEI message or is defined by the entire picture to which the DR-related SEI message applies, and include this in the DR-related SEI message.

[0468] For example, based on the fact that a thumbnail picture is determined by rectangular area information, the encoding device may generate rectangular coordinate information, rectangular size information, and / or rectangular fill information and include them in a DR-related SEI message. That is, when the thumbnail picture is determined by the rectangular area information, the encoding device may configure the DR-related SEI message to include parameters for obtaining the thumbnail picture from a specific spatial area of ​​the original picture.

[0469] Here, the rectangle-based thumbnail presence information can indicate whether syntax elements for settings related to the position of the display rectangle exist in the DR-related SEI message. That is, the decoding device can determine whether rectangle area information is included in the DR-related SEI message based on the rectangle-based thumbnail presence information. For example, rectangle-based thumbnail presence information with a value equal to 1 can indicate that syntax elements regarding display rectangle size information, display rectangle position information, display rectangle unit information, display rectangle length information, and / or display rectangle update information may be included in the DR-related SEI message. Additionally, rectangle-based thumbnail presence information with a value equal to 1 can indicate that syntax elements regarding display rectangle fill information may optionally be included in the DR-related SEI message. Additionally, square-based thumbnail existence information with a value equal to 0 may indicate that syntax elements regarding display square size information, display square position information, display square unit information, display square length information, display square update information and / or display square fill information are not included in the DR-related SEI message.

[0470] Rectangle-based thumbnail presence information may take various forms and may be represented by various names. For example, rectangle-based thumbnail presence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, rectangle-based thumbnail presence information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, rectangle-based thumbnail presence information may include dr_rect_info_present_flag or dr_rect_info_present_flag[ i ], but is not limited thereto.

[0471] Specifically, the above DR-related SEI message may include square coordinate information, for example, display square top-left x-coordinate information and display square top-left y-coordinate information indicating the coordinates of the top-left corner of the square, and display square top-left x-coordinate sign information and display square top-left y-coordinate sign information indicating the sign of each coordinate. Additionally, the above DR-related SEI message may include square size information, for example, display square width information and / or display square height information (or a height value derived according to aspect ratio information) indicating the width and height of the square. Accordingly, the encoding device can be configured so that the thumbnail picture is determined from the original picture as a square area defined by the coordinate information and size information.

[0472] For example, display rectangle size information may include display rectangle width information and / or display rectangle height information.

[0473] Display rectangle width information can be expressed in units of rectangle width. If such information is not present, the value of the display rectangle width information can be inferred to be the same as the value of the corresponding syntax element within the previous display rectangle-related SEI message in terms of output order.

[0474] Display rectangle width information may take various forms and may be represented by various names. For example, display rectangle width information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle width information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, display rectangle width information may include dr_rect_width[ i ], but is not limited thereto.

[0475] For example, if the display rectangle width information is represented as dr_rect_width[ i ], dr_rect_width[ i ] can represent the width of the i-th display rectangle in units.

[0476] The display rectangle height information can represent the height of the rectangle if such information exists. If such information does not exist, the value of the display rectangle height information can be inferred to be the same as the value of the corresponding syntax element within the previous display rectangle-related SEI message in terms of output order.

[0477] Display rectangle height information may take various forms and may be represented by various names. For example, display rectangle height information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle height information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, display rectangle height information may include dr_rect_height[ i ], but is not limited thereto.

[0478] For example, if the display rectangle height information is represented by dr_rect_height[ i ], and such information exists, dr_rect_height[ i ] can represent the height of the i-th rectangle in units of the variable DrUnitSizeInit. Here, the variable DrUnitSizeInit can be set to be equal to DrUnitSize, which is determined according to the above, when dr_pos_updates_flag described later is equal to 0. If such information does not exist and the display rectangle aspect ratio is equal to 0, the value of dr_rect_height[ i ] can be inferred to be equal to the value of the corresponding syntax element in the previous display rectangle-related SEI message in terms of output order.

[0479] Additionally, the display rectangle ratio information may represent the display aspect ratio of the display rectangle. For example, the display rectangle ratio information may be specified as one of a plurality of predefined aspect ratios, and the horizontal and vertical components of the aspect ratio corresponding to each display rectangle may be determined according to the value of the display rectangle ratio information.

[0480] Display aspect ratio information may take various forms and may be represented by various names. For example, display aspect ratio information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display aspect ratio information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or variable-length bits. For example, display aspect ratio information may include dr_display_aspect_ratio[ i ], but is not limited thereto.

[0481] For example, if the display rectangle ratio information is represented as dr_display_aspect_ratio[ i ], dr_display_aspect_ratio[ i ] may represent the display aspect ratio of the i-th display rectangle. Meanwhile, the value of dr_display_aspect_ratio[ i ] may be interpreted as specified in Table 2 above. For example, if the value of dr_display_aspect_ratio[ i ] is 1, the aspect ratio of the i-th display rectangle may be set to 16:9; if the value of dr_display_aspect_ratio[ i ] is 2, the aspect ratio may be set to 5:4; and if the value of dr_display_aspect_ratio[ i ] is 5, the aspect ratio may be set to 1:1. Additionally, if the value of dr_display_aspect_ratio[ i ] is 0, the aspect ratio may be interpreted as undefined. If the value of dr_display_aspect_ratio[ i ] is in the range of 10 to 15, such values ​​may be reserved values ​​that are not used in the bitstream of the present disclosure or are reserved for future expansion.

[0482] Display square unit information can indicate the granularity of the display square size and position.

[0483] Display square unit information may take various forms and may be represented by various names. For example, display square unit information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display square unit information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable length bit. For example, display square unit information may include dr_log2_unit_size, etc., but is not limited thereto.

[0484] Display rectangle length information can represent the lengths of syntax elements of display rectangle position information and / or display rectangle size information.

[0485] Display rectangle length information may take various forms and may be represented by various names. For example, display rectangle length information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle length information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable length bit, or an unsigned integer consisting of two or more bits or a variable length bit. For example, display rectangle length information may include dr_rect_size_len_minus1, but is not limited thereto.

[0486] For example, if the display rectangle length information is represented by dr_rect_size_len_minus1, the value obtained by adding 1 to dr_rect_size_len_minus1 can represent the lengths of the syntax elements dr_rect_top_left_x[ i ], dr_rect_top_left_y[ i ], dr_rect_width[ i ], dr_rect_height[ i ], dr_fill_top_left_x[ i ], dr_fill_top_left_y[ i ], dr_fill_width[ i ] and dr_fill_height[ i ].

[0487] Additionally, display rectangle update information may indicate whether the DR-related SEI message is a message that provides an explanation of the display rectangle information, or a message that includes an update to the existing display rectangle.

[0488] For example, display rectangle update information with a value equal to 1 may indicate that a DR-related SEI message provides a description of one or more display rectangles used to form a display picture from a cropped decoded picture. Additionally, display rectangle update information with a value equal to 0 may indicate that a DR-related SEI message includes updates for one or more display rectangles. However, this is not limited to these, and what is indicated by display rectangle update information with a value equal to 1 may be interchangeable with what is indicated by display rectangle update information with a value equal to 0.

[0489] Display rectangle update information may take various forms and may be represented by various names. For example, display rectangle update information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle update information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, display rectangle update information may include dr_pos_updates_flag or dr_pos_updates_idc, but is not limited thereto.

[0490] Furthermore, the above DR-related SEI message may include, as necessary, information on the existence of a display rectangle fill method, information on the display rectangle fill method, information on the color of the display rectangle fill, information on the depth of the display rectangle fill, information on the value of the display rectangle fill, information on the blur of the display rectangle fill, and / or information on the width of the content of the display rectangle fill. The above display rectangle fill information may be utilized, for example, in the process of expanding a rectangular area extracted from an original picture to match the display resolution (upscaling), or in the process of correcting the outer edges or empty areas of the extracted area.

[0491] Meanwhile, the DR-related SEI message may include persistence information, and said persistence information may indicate to which range of pictures the DR-related information signaled by the DR-related SEI message applies. That is, the encoding device may generate persistence information indicating whether to limit the scope of application of the DR-related SEI message to the current picture or to extend it to subsequent pictures in the output order.

[0492] The above persistence information may indicate whether the DR-related SEI message applies only to the current picture, or to the current picture and subsequent pictures in the output order based on satisfying a predefined termination condition. For example, persistence information with a value equal to 1 may indicate that the DR-related SEI message applies to the current picture and all pictures after the current layer in the output order. Additionally, persistence information with a value equal to 0 may indicate that the DR-related SEI message applies only to the current picture.

[0493] The above persistence information may take various forms and may be represented by various names. For example, the persistence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the persistence information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, the persistence information may include dr_persistence_flag, but is not limited thereto.

[0494] For example, if persistence information is represented by dr_persistence_flag, the encoding device may set dr_persistence_flag to 0 if the DR-related SEI message applies only to the current picture. In this case, the DR-related information included in the DR-related SEI message may be configured to apply only to the current picture. Accordingly, it is possible to provide a thumbnail definition or a display rectangle definition for a specific picture on a one-time basis.

[0495] Additionally, if the above DR-related SEI message is applied to the current picture and subsequent pictures of the current layer in the output order, the encoding device may set dr_persistence_flag to 1. In this case, the DR-related information may be configured to be continuously applied to subsequent pictures as well. At this time, the application may be maintained until at least one of a predefined termination condition is satisfied. For example, the termination condition may include at least one of the following: when a new encoded layered image sequence (CLVS) starts in the current layer, when the bitstream ends, or when a picture associated with the DR-related SEI message is output in the current layer, and said picture corresponds to a subsequent picture of the current picture in the output order.

[0496] Accordingly, the encoding device can reduce the repetitive signaling of the same DR-related information for multiple consecutive pictures. For example, the encoding device may generate persistence information indicating that specific DR-related information applies only to the current picture or continues to apply to subsequent pictures in the output sequence until a predefined termination condition is satisfied, and include this in the DR-related SEI message. Accordingly, the encoding device can reduce bitstream overhead by including the same or similar display rectangle definitions and / or thumbnail-related definitions in the DR-related SEI message only when necessary, instead of repeatedly transmitting them for every picture.

[0497] In addition, the information regarding the display rectangle described above can be generated based on the display rectangle cancellation information. That is, the encoding device can set the display rectangle cancellation information and, depending on the value of the cancellation information, selectively generate display rectangle-related information and include it in the DR-related SEI message.

[0498] Here, the display square cancellation information may indicate whether to apply persistence to DR-related SEI messages. For example, a display square cancellation information with a value equal to 1 may indicate that the persistence of any previous DR-related SEI messages is canceled. Additionally, a display square cancellation information with a value equal to 0 may indicate that display square information follows. However, this is not limited to this, and what a display square cancellation information with a value equal to 1 indicates may be different from what a display square cancellation information with a value equal to 0 indicates.

[0499] Display rectangle cancellation information may take various forms and may be represented by various names. For example, display rectangle cancellation information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle cancellation information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, display rectangle cancellation information may include, but is not limited to, dr_cancel_flag, dr_rect_cancel_flag[ i ], dr_cancel_idc, or dr_rect_cancel_idc[ i ].

[0500] According to one embodiment, a DR-related SEI message may further include rectangle information for each of a plurality of display rectangles to provide definitions of display rectangles that can be applied to one picture or a plurality of pictures. Here, since the rectangle information can be provided independently for each display rectangle, it is possible to describe a plurality of display rectangles simultaneously within a single DR-related SEI message while assigning different operations and meanings to each display rectangle.

[0501] Specifically, the rectangle information may include rectangle identifier information for distinguishing each display rectangle. The encoding device may generate rectangle identifier information so that each display rectangle can be identified within a DR-related SEI message, or generate rectangle identifier information so that the same display rectangle is indicated by the same identifier even between different DR-related SEI messages. Accordingly, even when multiple display rectangles exist, each display rectangle can be clearly distinguished and signaled, and the decoding device can perform association processing at the rectangle level.

[0502] The above rectangular identifier information may include an identification number that can be used to identify DR-related SEI messages. For example, the value of the rectangular identifier information may be within the range of 0 to 16,383 (inclusive).

[0503] Rectangle identifier information may take various forms and may be represented by various names. For example, the rectangle identifier information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the rectangle identifier information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, the rectangle identifier information may include dr_rect_id[ i ], but is not limited thereto.

[0504] For example, if rectangle identifier information is represented as dr_rect_id[ i ], dr_rect_id[ i ] may include an identification number that can be used to identify the i-th display rectangle. The value of dr_rect_id[ i ] may be in the range of 0 to 16383 (inclusive).

[0505] Additionally, the DR-related SEI message may further include square purpose information corresponding to each of a plurality of display squares. The square purpose information is information intended to indicate what purpose each display square is used for, and the encoding device may generate the square purpose information so that each display square can be interpreted and processed with different meanings. For example, the square purpose information corresponding to each of a plurality of display squares may be configured so that the DR-related SEI message indicates thumbnail information of the original picture and spatial area information of the original picture.

[0506] The above rectangular purpose information may indicate the purpose of the display rectangle. For example, the value of the rectangular purpose information may be in the range of 0 to 1 (inclusive). Additionally, values ​​in the range of 2 to 255 (inclusive) for the value of the rectangular purpose information may be reserved for future use.

[0507] Rectangular purpose information may take various forms and may be represented by various names. For example, rectangular purpose information may be a syntax element or a syntax structure containing one or more syntax elements. For example, rectangular purpose information may include a flag consisting of one bit or an unsigned integer consisting of two or more bits or a variable length bit. For example, rectangular purpose information may include dr_rect_purpose[ i ], but is not limited thereto.

[0508] For example, if the rectangle purpose information is represented as dr_rect_purpose[ i ], dr_rect_purpose[ i ] can represent the purpose of the i-th display rectangle.

[0509] Specifically, if the display rectangle indicates a thumbnail of the original video content, the encoding device may set the value of the rectangle target information corresponding to the display rectangle to 0. Accordingly, the encoding device may generate rectangle-related information (e.g., location information and size information, etc.) corresponding to the display rectangle and include this in a DR-related SEI message to signal that a thumbnail picture can be obtained from the original picture. On the other hand, if the display rectangle indicates a spatial portion of the original video content, the encoding device may set the value of the rectangle target information corresponding to the display rectangle to 1. Accordingly, the encoding device may include rectangle-related information (e.g., location / size information, etc.) indicating the area indicated by the display rectangle in a DR-related SEI message so that the decoding device can use it to identify a specific spatial area of ​​the original picture or to configure the display picture.

[0510] Meanwhile, since values ​​of the rectangular target information in the range of 2 to 255 (inclusive) are reserved for future use, an encoding device generating a bitstream suitable for the present disclosure may set the values ​​of the rectangular target information so as not to include the reserved values. For example, the encoding device may be configured to generate a bitstream suitable for the present disclosure by signaling the rectangular target information within the value range of the target information defined in the present disclosure.

[0511] In addition, the definition method of the thumbnail can be specified on a per-display rectangle basis. For example, if each of a plurality of display rectangles indicates thumbnail information of the original picture, the encoding device may generate rectangle target information corresponding to each display rectangle for each display rectangle and include it in a DR-related SEI message. Additionally, based on the rectangle target information, the encoding device may configure the DR-related SEI message so that it does not include filling-related information for each display rectangle, such as information on the existence of a filling method. That is, the encoding device may generate a DR-related SEI message by disabling information on the existence of a filling method so that syntax elements regarding the filling process of each display rectangle are not signaled.

[0512] Meanwhile, the DR-related SEI message may include rectangle-based thumbnail existence information to indicate the definition method of a thumbnail picture corresponding to each of a plurality of display rectangles. For example, the encoding device may generate the rectangle-based thumbnail existence information for each display rectangle and include it in the DR-related SEI message in order to signal whether, for each display rectangle, the thumbnail picture is defined by the display rectangle area or by the entire picture to which the DR-related SEI message is applied.

[0513] Specifically, when a thumbnail picture is defined based on a rectangular area, the encoding device may include rectangular area information (e.g., coordinate information and size information) for the display rectangle in a DR-related SEI message so that the thumbnail picture is obtained from a specific spatial area of ​​the original picture. On the other hand, when a thumbnail picture is defined based on the entire picture, the encoding device may omit signaling of separate rectangular area information and configure the DR-related SEI message so that the entire picture to which the DR-related SEI message applies is considered as a thumbnail picture.

[0514] For example, the encoding device can determine, for each of a plurality of display rectangles, whether the thumbnail picture corresponding to the corresponding display rectangle is determined by the rectangle area information included in the DR-related SEI message, or whether it is determined as the entire picture to which the DR-related SEI message is applied. Based on the result of this determination, the encoding device can generate rectangle-based thumbnail existence information corresponding to each display rectangle and include it in the DR-related SEI message. That is, the rectangle-based thumbnail existence information for each of the plurality of display rectangles can be generated based on whether the thumbnail picture is determined by the rectangle area information included in the DR-related SEI message, or whether it is determined as the entire picture to which the DR-related SEI message is applied.

[0515] Specifically, when a thumbnail picture is determined by rectangular area information for a specific display rectangle, the encoding device may generate rectangular-based thumbnail existence information corresponding to the display rectangle to indicate a rectangular area-based thumbnail definition. In this case, the encoding device may include rectangular area information, such as rectangular coordinate information and rectangular size information, in a DR-related SEI message so that the decoding device extracts a specific spatial area of ​​the original picture and obtains a thumbnail picture.

[0516] On the other hand, if the thumbnail picture for a specific display rectangle is determined to be the entire picture to which the DR-related SEI message applies, the encoding device may generate rectangle-based thumbnail existence information corresponding to that display rectangle to indicate a picture-wide based thumbnail definition. In this case, the encoding device may omit or minimize the process of signaling separate rectangle area information so that the decoding device considers the entire picture to which the DR-related SEI message applies as the thumbnail picture. In this way, by generating rectangle-based thumbnail existence information for each of a plurality of display rectangles, the encoding device can provide rectangle area-based thumbnails and picture-wide based thumbnails on a display rectangle basis.

[0517] Additionally, the rectangle-based thumbnail existence information may indicate whether syntax elements regarding the position and size of the display rectangle are included in the DR-related SEI message. If the thumbnail picture is configured to be defined by the rectangle area information, the encoding device may set the value of the rectangle-based thumbnail existence information so that syntax elements for indicating the position and size of the display rectangle (e.g., syntax elements related to the top-left coordinate and size) are included in the DR-related SEI message. Conversely, if the thumbnail picture is configured to be defined by the entire picture to which the DR-related SEI message applies, the encoding device may set the value of the rectangle-based thumbnail existence information so that the syntax elements are omitted.

[0518] Accordingly, the decoding device determines whether rectangular area information is provided within the DR-related SEI message based on the existence information of a rectangle-based thumbnail, and if rectangular area information is provided, it can derive variables representing the position and size of the display rectangle (e.g., DrRectTopLeftX[i], DrRectTopLeftY[i], DrRectWidth[i], DrRectHeight[i]) from the syntax elements.

[0519] For example, rectangle-based thumbnail presence information can be represented by dr_rect_info_present_flag[i], and the encoding device can set the value of dr_rect_info_present_flag[i] according to the definition method of the thumbnail picture. Specifically, if the display rectangle area indicated by the variables DrRectWidth[i], DrRectHeight[i], DrRectTopLeftX[i], and DrRectTopLeftY[i] is determined to be a thumbnail image or a thumbnail video, the encoding device can set the value of dr_rect_info_present_flag[i] to 1. That is, to specify the area where the thumbnail picture is to be obtained from the original picture, the encoding device can include rectangle coordinate information and rectangle size information corresponding to the top-left coordinates and size of the display rectangle in the DR-related SEI message. Additionally, if necessary, the encoding device can configure the DR-related SEI message so that filling-related information is optionally included in the DR-related SEI message.

[0520] On the other hand, if the picture(s) to which the DR-related SEI message is applied are determined to be a thumbnail image or a thumbnail video, the encoding device may set the value of dr_rect_info_present_flag[i] to 0. That is, in this case, the encoding device may not signal or omit separate display rectangle area information, thereby allowing the decoding device to treat the entire picture to which the DR-related SEI message is applied as a thumbnail picture.

[0521] The information regarding the display rectangles described above can be generated based on display rectangle cancellation information for each of the multiple display rectangles. That is, the encoding device can generate information about the display rectangles according to the display rectangle cancellation information corresponding to each display rectangle and include it in the DR-related SEI message.

[0522] Here, display rectangle cancellation information may indicate whether to apply persistence to SEI messages related to DR. Display rectangle cancellation information may take various forms and may be represented by various names. For example, display rectangle cancellation information may be a syntax element or a syntax structure containing one or more syntax elements. For example, display rectangle cancellation information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, display rectangle cancellation information may include, but is not limited to, dr_rect_cancel_flag[ i ]. For example, if display rectangle cancellation information is represented as dr_rect_cancel_flag[ i ]. dr_rect_cancel_flag[ i ]. may indicate whether the information regarding the i-th display rectangle is persisted, that is, whether the definition for the i-th display rectangle is cancelled.

[0523] The encoding device can encode video information (S630).

[0524] For example, the processor of the encoding device can encode video information including DR-related SEI messages.

[0525] An encoding device can encode an image or image information. Here, the image may include a still image or a video, and the image information may refer to pixel data of the image, object information, additional information (e.g., SEI messages, etc.), or a combination thereof.

[0526] An encoding device can generate a bitstream by performing an encoding process on an input image or image information. At this time, the encoding process may include one or more steps such as prediction, transformation, quantization, and entropy encoding, and the generated bitstream can be used as data for reconstructing the image or image information by a decoding device.

[0527] In addition, the present disclosure may be applied not only to general encoding structures for encoding images or image information, but also to encoding methods for DR-related SEI messages that include additional information related to display rectangles.

[0528] The encoding device may generate a DR-related SEI message based on information regarding one or more display rectangles that can be used to form a thumbnail picture from an original picture, and the generated DR-related SEI message may be included in a bitstream so that it can be utilized in a decoding or post-processing process. For example, the DR-related SEI message may include at least one of the following: number information of display rectangles, identifier information, persistence information, cancellation information, destination information, existence information of a rectangle-based area, size information for each of a plurality of display rectangles, filling information, and / or location information, and the decoding device may parse the information to determine the attributes of the display rectangles and use them to construct a thumbnail picture.

[0529] In addition, the present invention integrates thumbnail extraction information, which was previously provided separately as a TEI (Thumbnail Extraction Information) related SEI message, into a DR related SEI message (message unit or display rectangle unit) and signals it, thereby enabling an encoding device to generate and manage the thumbnail definition method, application scope, identification and usage per rectangle, and even the update or invalidation by a cancellation flag as a single consistent structure, thereby reducing redundant signals and reducing bitstream overhead. Accordingly, the provision of thumbnail information suitable for various service / terminal environments can be flexibly and efficiently encoded.

[0530] The names of syntax elements and semantics described in this specification are merely exemplary names for the convenience of explanation, and the scope of the rights of this disclosure is not limited by such names. That is, any configuration that performs the same or substantially corresponding functions as the syntax elements and semantics described in this specification may be included within the spirit and scope of this disclosure, even if its name, notation, or terminology differs.

[0531] Video information encoded according to the encoding method (S600) described above can be output in the form of a bitstream. In other words, the bitstream can be generated based on the video information encoded according to the encoding method (S600) described above.

[0532] A bitstream generated based on video information encoded according to the encoding method (S600) described above can be stored on a computer-readable storage medium.

[0533] A bitstream generated based on video information encoded according to the encoding method (S600) described above can be transmitted through a transmission unit and / or a transmission medium.

[0534] According to one embodiment, image information may include SEI messages. SEI messages may convey specific types of information that assist in processes related to the decoding, display, or other processing purposes of image information. Here, SEI messages may include DR-related SEI messages.

[0535] FIG. 7 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0536] As illustrated in FIG. 7, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0537] The above encoding server compresses content input from multimedia input devices, such as smartphones, cameras, and camcorders, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and camcorders, generate the bitstream directly, the encoding server may be omitted.

[0538] The bitstream may be generated by a video encoding method and / or encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0539] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0540] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0541] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0542] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0543] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0544] An embodiment according to the present disclosure can be used to encode / decode images.

Claims

In a video decoding method performed by a decoding device, A step of obtaining a DR (Display Rectangles) related SEI (Supplemental Enhancement Information) message from a bitstream; and A video decoding method comprising the step of obtaining a thumbnail picture from an original picture based on DR-related information included in the above DR-related SEI message. In paragraph 1, A video decoding method characterized in that the above DR-related SEI message includes at least one of identifier information identifying the above DR-related SEI message, purpose information indicating the purpose of the above DR-related SEI message, persistence information indicating the persistence of the above DR-related SEI message, and rectangle-based thumbnail existence information indicating definition information of the above thumbnail picture. In paragraph 2, A video decoding method characterized in that the above-mentioned purpose information is configured such that the above-mentioned DR-related SEI message indicates thumbnail information of the above-mentioned original picture or indicates spatial area information of the above-mentioned original picture. In paragraph 3, A video decoding method characterized by the fact that, based on the above-mentioned purpose information indicating that the above-mentioned DR-related SEI message indicates thumbnail information of the above-mentioned original picture, the above-mentioned DR-related SEI message includes one display rectangle and does not include information on the existence of a filling method for the above-mentioned display rectangle. In paragraph 2, An image decoding method characterized in that the thumbnail picture is determined by rectangular area information included in the DR-related SEI message based on the rectangular-based thumbnail existence information, or is determined as the entire picture to which the DR-related SEI message is applied. In paragraph 5, A video decoding method characterized in that, based on the fact that the above-mentioned rectangle-based thumbnail existence information indicates that the above-mentioned thumbnail picture is determined by the above-mentioned rectangle area information, the above-mentioned DR-related SEI message includes rectangle coordinate information, rectangle size information, and rectangle fill information. In paragraph 2, An image decoding method characterized by the above persistence information indicating whether the above DR-related SEI message is applied only to the current picture, or whether it is applied to the current picture and subsequent pictures in the output order based on satisfying a predefined termination condition. In paragraph 1, The above DR-related SEI message further includes rectangle information for each of the multiple display rectangles, and A video decoding method characterized in that the above-described rectangle information includes, for each of the plurality of display rectangles, at least one of rectangle identifier information identifying the display rectangle, rectangle purpose information indicating the purpose of the display rectangle, and rectangle-based thumbnail existence information indicating definition information of the thumbnail picture. In paragraph 8, A video decoding method characterized in that the rectangle target information corresponding to each of the plurality of display rectangles is configured such that the DR-related SEI message indicates thumbnail information of the original picture and spatial area information of the original picture. In Paragraph 9, A video decoding method characterized in that, based on the fact that the square target information corresponding to each of the plurality of display squares indicates that each display square indicates thumbnail information of the original picture, the DR-related SEI message does not include information on the existence of a fill method for the display squares. In Paragraph 10, An image decoding method characterized in that the thumbnail picture is determined by rectangular area information included in the DR-related SEI message based on the rectangular-based thumbnail existence information for each of the plurality of display rectangles, or is determined as the entire picture to which the DR-related SEI message is applied. In a video encoding method performed by an encoding device, A step of generating a DR-related SEI (Supplemental Enhancement Information) message containing DR (Display Rectangles) related information; and It includes the step of encoding video information including the above DR-related SEI message, and A video encoding method characterized in that the thumbnail picture is generated from the original picture based on the DR-related information. In Paragraph 12, A video encoding method characterized in that the above DR-related SEI message includes at least one of identifier information identifying the above DR-related SEI message, purpose information indicating the purpose of the above DR-related SEI message, persistence information indicating the persistence of the above DR-related SEI message, and rectangle-based thumbnail existence information indicating definition information of the above thumbnail picture. In Paragraph 13, A video encoding method characterized in that the above-mentioned purpose information is configured such that the above-mentioned DR-related SEI message indicates thumbnail information of the above-mentioned original picture or indicates spatial area information of the above-mentioned original picture. In Paragraph 13, A video encoding method characterized in that the above-mentioned rectangle-based thumbnail existence information is generated based on the thumbnail picture being determined by the rectangle area information included in the DR-related SEI message, or being determined as the entire picture to which the DR-related SEI message is applied. In Paragraph 12, The above DR-related SEI message further includes rectangle information for each of the multiple display rectangles, and A video encoding method characterized in that the above-described rectangle information includes, for each of the plurality of display rectangles, at least one of rectangle identifier information identifying the display rectangle, rectangle purpose information indicating the purpose of the display rectangle, and rectangle-based thumbnail existence information indicating definition information of the thumbnail picture. In Paragraph 16, A video encoding method characterized in that the square target information corresponding to each of the plurality of display squares is configured such that the DR-related SEI message indicates thumbnail information of the original picture and spatial area information of the original picture. In Paragraph 16, A video encoding method characterized in that the rectangle-based thumbnail existence information for each of the plurality of display rectangles is generated based on the thumbnail picture being determined by rectangle area information included in the DR-related SEI message, or being determined as the entire picture to which the DR-related SEI message is applied. A non-transitory computer-readable storage medium for storing a bitstream generated by the image encoding method described in paragraph 12. A method for transmitting data regarding an image, wherein the method comprises the step of acquiring image information, wherein the image information includes a DR-related SEI (Supplemental Enhancement Information) message containing DR (Display Rectangles) related information; and The method includes the step of transmitting the data including the above image information, A data transmission method characterized in that the thumbnail picture is generated from the original picture based on the DR-related information.

Images