Image decoding method, image encoding method, computer-readable storage medium, and image data transmission method

Abstract

An image decoding method performed by a decoding device according to the present disclosure may comprise the steps of: acquiring, from a bitstream, a thumbnail-related supplemental enhancement information (SEI) message associated with a picture; and acquiring thumbnail-related information from the thumbnail-related SEI message, wherein the thumbnail-related information includes at least one among segment output sequence information, display duration information, luma gain coefficient information, and blank picture information about the thumbnail picture.

Description

Video decoding method, video encoding method, computer-readable storage medium and method for transmitting data for video

[0001] The present disclosure relates to a method for decoding / encoding image information, a computer-readable storage medium for storing image information, and a method for transmitting image information.

[0002] Recently, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), has been increasing across various fields. As video data becomes higher in resolution and quality, the relative amount of information or bits transmitted increases compared to conventional video data. This increase in transmitted information or bits leads to higher transmission and storage costs.

[0003] Accordingly, high-efficiency video compression technology is required to effectively transmit, store, and play back high-resolution, high-quality video information.

[0004] The present disclosure aims to improve the reliability of a coding system including an encoding device and a decoding device.

[0005] The present disclosure aims to improve the coding efficiency of a coding system including an encoding device and a decoding device.

[0006] The present disclosure aims to improve the data transmission efficiency of a coding system including an encoding device and a decoding device.

[0007] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0008] According to one aspect of the present disclosure, a video decoding method comprises: a step of obtaining a thumbnail-related SEI (Supplemental Enhancement Information) message associated with a picture from a bitstream; and a step of obtaining thumbnail-related information from the thumbnail-related SEI message, wherein the thumbnail-related information may include at least one of segment output order information, display duration information, luminance gain factor information, or blank picture information of the thumbnail picture.

[0009] According to one aspect of the present disclosure, a decoding device for decoding image information may be characterized by obtaining a thumbnail-related SEI (Supplemental Enhancement Information) message associated with a picture from a bitstream, obtaining thumbnail-related information from the thumbnail-related SEI message, and the thumbnail-related information may include at least one of segment output order information, display duration information, luminance gain coefficient information, or blank picture information of the thumbnail picture.

[0010] In a method or device for decoding the above image information, the thumbnail-related SEI message may be characterized by including a TSI (Thumbnail Source Information) related SEI message containing source information of the thumbnail picture, and a TEI (Thumbnail Extraction Information) related SEI message containing extraction information of the thumbnail picture.

[0011] In a method or device for decoding the above image information, the output order of the segment to which the current picture belongs within the thumbnail picture is determined based on the segment output order information, and the segment output order information may be characterized by including output order existence information included in the TSI-related SEI message and output order information included in the TEI-related SEI message.

[0012] In a method or device for decoding the above image information, the picture to which the TEI-related SEI message containing the same output sequence information is applied may be characterized as being included in the same segment.

[0013] In a method or device for decoding the above image information, the output order information is not included in the TEI-related SEI message, and based on the fact that the TEI-related SEI message is the first TEI-related SEI message in the bitstream according to the output order, the value of the output order information can be inferred to be 0.

[0014] In a method or device for decoding the above image information, the output order information is not included in the TEI-related SEI message, and based on the fact that the TEI-related SEI message is not the first TEI-related SEI message in the bitstream according to the output order, the value of the output order information may be inferred as the value of the output order information of the immediately preceding TEI-related SEI message according to the output order plus 1.

[0015] In a method or apparatus for decoding the above image information, the display duration information may include at least one of duration existence information, time scale information, or duration unit information included in the TSI-related SEI message, and the time scale information and the duration unit information may be obtained based on the duration existence information.

[0016] In a method or device for decoding the above image information, the display duration of the current picture is determined based on the time scale information and the duration unit information, and the display duration of the current picture is determined based on dividing the duration unit information by the time scale information.

[0017] In a method or apparatus for decoding the above image information, the luminance gain coefficient information may include at least one of luminance gain coefficient existence information included in the TSI-related SEI message, luminance gain coefficient numerator information included in the TEI-related SEI message, or luminance gain coefficient denominator information, and the luminance gain coefficient numerator information and the luminance gain coefficient denominator information may be characterized as being obtained based on the luminance gain coefficient existence information.

[0018] In a method or apparatus for decoding the above image information, at least one of time scale information, duration unit information, blank picture duration existence information, or blank picture duration unit information may be obtained based on duration existence information included in the TSI-related SEI message.

[0019] In a method or device for decoding the above image information, the SEI message related to the TEI may further include blank picture existence information, blank picture count information, and blank picture sample information, wherein the blank picture count information indicates the number of blank pictures following the current picture in the output order, and the blank picture sample information indicates sample information regarding the color components of the blank picture following the current picture in the output order.

[0020] According to one aspect of the present disclosure, a video encoding method comprises: a step of generating a thumbnail-related SEI (Supplemental Enhancement Information) message including thumbnail-related information associated with a picture; and a step of encoding video information including the thumbnail-related SEI message, wherein the thumbnail-related information may include at least one of segment output order information, display duration information, luminance gain factor information, or blank picture information of the thumbnail picture.

[0021] According to one aspect of the present disclosure, an apparatus for encoding image information may be characterized by generating a thumbnail-related SEI (Supplemental Enhancement Information) message including thumbnail-related information associated with a picture, encoding image information including the thumbnail-related SEI message, and wherein the thumbnail-related information may include at least one of segment output order information of the thumbnail picture, display duration information, luminance gain coefficient information, or blank picture information.

[0022] In a method or device for encoding the above-mentioned image information, the thumbnail-related SEI message may be characterized by including a TSI (Thumbnail Source Information) related SEI message containing source information of the thumbnail picture, and a TEI (Thumbnail Extraction Information) related SEI message containing extraction information of the thumbnail picture.

[0023] In a method or device for encoding the above-mentioned image information, the segment output order information is determined based on the output order of the segment to which the current picture belongs within the thumbnail picture, and the segment output order information may be characterized by including output order existence information included in the TSI-related SEI message and output order information included in the TEI-related SEI message.

[0024] In a method or device for encoding the above-mentioned image information, the display duration information may include at least one of duration existence information, time scale information, or duration unit information included in the TSI-related SEI message, and the duration existence information may be generated based on the time scale information and the duration unit information.

[0025] In a method or apparatus for encoding the above image information, the luminance gain coefficient information may include at least one of luminance gain coefficient existence information included in the TSI-related SEI message, luminance gain coefficient numerator information included in the TEI-related SEI message, or luminance gain coefficient denominator information, and the luminance gain coefficient existence information may be characterized as being generated based on the luminance gain coefficient numerator information and the luminance gain coefficient denominator information.

[0026] In a method or apparatus for encoding the above image information, the TEI-related SEI message further includes blank picture existence information, blank picture duration existence information, blank picture duration unit information, blank picture count information, and blank picture sample information, and the duration existence information included in the TSI-related SEI message may be characterized as being generated based on at least one of time scale information, the duration unit information, the blank picture duration existence information, or the blank picture duration unit information.

[0027] According to one aspect of the present disclosure, a bitstream generated by a video encoding method is stored in a non-transitory computer-readable storage medium. The video encoding method comprises the steps of: generating a thumbnail-related SEI (Supplemental Enhancement Information) message including thumbnail-related information associated with a picture; and encoding video information including the thumbnail-related SEI message, wherein the thumbnail-related information may include at least one of segment output order information, display duration information, luminance gain factor information, or blank picture information of the thumbnail picture.

[0028] According to one aspect of the present disclosure, a method for transmitting data for an image comprises: a step of acquiring image information, wherein the image information includes a thumbnail-related SEI (Supplemental Enhancement Information) message including thumbnail-related information associated with a picture; and a step of transmitting the data including the image information, wherein the thumbnail-related information includes at least one of segment output order information of the thumbnail picture, display duration information, luminance gain coefficient information, or blank picture information.

[0029] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0030] According to the present disclosure, the reliability of a coding system including an encoding device and a decoding device can be improved.

[0031] According to the present disclosure, the coding efficiency of a coding system including an encoding device and a decoding device can be improved.

[0032] According to the present disclosure, the data transmission efficiency of a coding system including an encoding device and a decoding device can be improved.

[0033] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0034] FIG. 1 is a schematic diagram illustrating a video coding system to which an embodiment according to the present disclosure can be applied.

[0035] FIG. 2 is a schematic diagram showing an encoding device to which an embodiment according to the present disclosure can be applied.

[0036] FIG. 3 is a schematic diagram showing a decoding device to which an embodiment according to the present disclosure can be applied.

[0037] Figure 4 illustrates an exemplary hierarchical structure for a coded video / image.

[0038] FIG. 5 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0039] FIG. 6 is a diagram illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0040] FIG. 7 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0041] Hereinafter, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0042] In describing the embodiments of the present disclosure, detailed descriptions of known configurations or functions are omitted if it is determined that such descriptions could obscure the essence of the present disclosure. Additionally, parts of the drawings unrelated to the description of the present disclosure have been omitted, and similar parts are denoted by similar reference numerals.

[0043] In the present disclosure, when a component is described as being "connected," "combined," or "joined" with another component, this may include not only a direct connection but also an indirect connection in which another component exists in between. Furthermore, when a component is described as "comprising" or "having" another component, this means that, unless specifically stated otherwise, it does not exclude the other component but may include an additional component.

[0044] In the present disclosure, terms such as first, second, etc. are used solely for the purpose of distinguishing one component from another and do not limit the order or importance of the components unless specifically stated otherwise. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and likewise, a second component in one embodiment may be referred to as a first component in another embodiment.

[0045] In this disclosure, distinct components are intended to clearly describe their respective features and do not imply that the components are separate. That is, multiple components may be integrated to form a single hardware or software unit, or a single component may be distributed to form multiple hardware or software units. Accordingly, such integrated or distributed embodiments are included within the scope of this disclosure, unless otherwise noted.

[0046] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Furthermore, embodiments including additional components in addition to the components described in various embodiments are also included within the scope of the present disclosure.

[0047] The present disclosure relates to the encoding and decoding of images. For example, the methods and embodiments disclosed in this document may be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard) or next-generation video / image coding standards (e.g., H.267 or H.268).

[0048] The present disclosure presents various embodiments relating to video / image coding, and unless otherwise stated, said embodiments may be performed in combination with one another.

[0049] Unless newly defined in this disclosure, the terms used herein may have the ordinary meanings commonly used in the technical field to which this disclosure belongs.

[0050] In this disclosure, "video" may refer to a set of images over time. In this disclosure, "picture" generally refers to a unit representing a single image at a specific time, and a slice / tile is a unit that constitutes a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular area of ​​rows of CTUs within a tile in a picture. In this document, tile groups and slices may be used interchangeably. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.

[0051] In the present disclosure, "pixel" or "pel" may refer to the smallest unit constituting a picture (or image). Additionally, "sample" may be used as a term corresponding to pixel. A sample may generally represent a pixel or a pixel value, may represent only the pixel / pixel value of the luminance component, or may represent only the pixel / pixel value of the chroma component.

[0052] In this disclosure, "unit" may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to that area. A unit may include one luminance block and two chroma (e.g., cb, cr) blocks. Depending on the case, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include samples (or sample arrays) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0053] In the present disclosure, "current block" may mean one of "current coding block," "current coding unit," "block to be encoded," "block to be decoded," or "block to be processed." When prediction is performed, "current block" may mean "current prediction block" or "block to be predicted." When transformation (inverse transformation) / quantization (inverse quantization) is performed, "current block" may mean "current transformation block" or "block to be transformed." When filtering is performed, "current block" may mean "block to be filtered."

[0054] In the present disclosure, "current block" may mean a block comprising both a luminous component block and a chroma component block, or "luma block of the current block," unless explicitly stated as a chroma block. The luminous component block of the current block may be expressed by including an explicit description of a luminous component block, such as "luma block" or "current luminous block." Additionally, the chroma component block of the current block may be expressed by including an explicit description of a chroma component block, such as "chroma block" or "current chroma block."

[0055] In the present disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Additionally, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0056] In the present disclosure, "or" may be interpreted as "and / or". For example, "A or B" may mean 1) "A" only, 2) "B" only, or 3) "A and B". Alternatively, in the present disclosure, "or" may mean "additionally or alternatively".

[0057] FIG. 1 is a schematic diagram illustrating a video / image coding system to which an embodiment according to the present disclosure can be applied.

[0058] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image or data in the form of a file or streaming to the receiving device via a digital storage medium or a network.

[0059] The source device may include a video source, an encoding device, and a transmission unit. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0060] A video source may acquire video / images through processes such as video / image capture, synthesis, or generation. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and may generate video / images (electronically). For example, virtual video / images may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0061] The encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0062] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device in the form of a file or streaming via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0063] The decoding device can decode video / images by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operation of the encoding device.

[0064] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0065] FIG. 2 is a schematic diagram illustrating an encoding device to which an embodiment according to the present disclosure can be applied.

[0066] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor (221) and an intra-predictor (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit (250) may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to the embodiment. Additionally, the memory (270) may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0067] The image segmentation unit (210) can divide an input image (or picture, frame) input to an encoding device (200) into one or more processing units. For example, the processing unit may be called a coding unit (CU). A coding unit may be recursively divided into a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, a single coding unit may be divided into multiple coding units of a deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. For example, a quad-tree structure may be applied first, and a binary-tree structure and / or a ternary-tree structure may be applied later. Alternatively, a binary-tree structure may be applied first. A coding procedure according to the present disclosure may be performed based on the final coding unit that is no longer divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be used directly as the final coding unit, or, if necessary, the maximum coding unit may be recursively divided into lower-depth coding units so that a coding unit of the optimal size is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may each be divided or partitioned from the final coding unit.The above prediction unit may be a unit of sample prediction, and the above transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0068] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. A sample may be used to refer to a single picture (or image) as a term corresponding to a pixel or pel.

[0069] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, the unit that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231). The prediction unit (220) can perform a prediction for a block to be processed (hereinafter, current block) and generate a predicted block (predicted block) containing prediction samples for said current block. The prediction unit (220) can determine whether intra prediction is applied or inter prediction is applied in units of the current block or CU. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0070] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0071] The inter prediction unit (221) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different from each other. The temporal neighboring blocks may be referred to by names such as collocated reference block, collocated CU (colCU), etc. A reference picture containing the aforementioned temporal surrounding blocks may be called a collocated picture (colPic). For example, the inter prediction unit (221) may construct a list of motion information candidates based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of surrounding blocks as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0072] The prediction unit (220) may generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit (220) may apply intra prediction or inter prediction for the prediction of the current block, as well as apply intra prediction and inter prediction simultaneously. A prediction method that applies intra prediction and inter prediction simultaneously for the prediction of the current block may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit (220) may be based on an intra block copy (IBC) prediction mode or a palette mode for the prediction of the block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. Palette mode can be viewed as an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within a picture can be signaled based on information regarding palette tables and palette indices.

[0073] The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or to generate a residual signal. The subtraction unit (231) can generate a residual signal (residual signal, residual block, residual sample array) by subtracting the prediction signal (predicted block, prediction sample array) output from the prediction unit (220) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit (232).

[0074] The transformation unit (232) can generate transform coefficients by applying a transformation technique to a residual signal. For example, the transformation technique may include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transformation obtained based on a prediction signal generated using all previously reconstructed pixels. The transformation process may be applied to a block of pixels of the same size in a square, or to a block of variable size that is not square.

[0075] The quantization unit (233) can quantize the transformation coefficients and transmit them to the entropy encoding unit (240). The entropy encoding unit (240) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (233) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients.

[0076] The entropy encoding unit (240) can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit (190) may encode information required for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The signaling information, transmitted information, and / or syntax elements mentioned in the present disclosure may be included in the video / image information. The video / image information may be encoded through the encoding procedure described above and included in the bitstream.

[0077] The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmitting a signal output from the entropy encoding unit (240) and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the encoding device (200), or the transmission unit may be provided as a component of the entropy encoding unit (240).

[0078] The quantized transformation coefficients output from the quantization unit (233) can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) can be restored by applying inverse quantization and inverse transformation to the quantized transformation coefficients through the inverse quantization unit (234) and the inverse transformation unit (235).

[0079] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0080] The adder (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The adder (250) may be called a reconstructed unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0081] The filtering unit (260) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (260) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (270), specifically in the DPB of memory (170). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240), as described below in the description of each filtering method. The information regarding filtering can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0082] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter-prediction unit (221). Through this, the encoding device (200) can avoid prediction mismatches between the encoding device (200) and the decoding device when inter-prediction is applied, and can also improve encoding efficiency.

[0083] The DPB in memory (270) can store a modified restored picture to be used as a reference picture in the inter prediction unit (221). Memory (270) can store motion information of blocks from which motion information is derived (or encoded) in the current picture and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter prediction unit (221) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory (270) can store restoration samples of restored blocks in the current picture and transmit them to the intra prediction unit (222).

[0084] FIG. 3 is a schematic diagram illustrating a decoding device to which an embodiment according to the present disclosure can be applied.

[0085] As illustrated in FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor (332) and an intra-predictor (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The aforementioned entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) may be configured by a single hardware component (e.g., a decoder chipset or a processor) according to an embodiment. Additionally, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0086] When a bitstream containing video / image information is input, the decoding device (300) can restore the image by performing a process corresponding to the process performed by the encoding device (200) of FIG. 2. For example, the decoding device (300) can perform decoding using a processing unit applied in the encoding device (200). Thus, the processing unit for decoding may be, for example, a coding unit. The coding unit may be a coding tree unit, or a maximum coding unit may be obtained by dividing it according to a quad tree structure, a binary tree structure, and / or a binary tree structure. And, the restored image signal decoded and output through the decoding device (300) can be played back through a playback device (not shown).

[0087] The decoding device (300) can receive a signal output from the encoding device (200) of FIG. 2 in the form of a bitstream. The received signal can be decoded through an entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The decoding device (300) can decode the picture based on the information regarding the parameter sets and / or the general constraint information. The signaling / received information and / or syntax elements described below can be obtained from the bitstream by decoding through the decoding procedure. For example, the entropy decoding unit (310) can decode information within the bitstream based on coding methods such as exponential coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transformation coefficients regarding residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information on the syntax element to be decoded and decoding information of surrounding and decoding target blocks or information on symbols / bins decoded in the previous step, predict the probability of occurrence of the bin according to the determined context model, and perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin after determining the context model. Among the information decoded in the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (330), and residual values ​​for which entropy decoding was performed in the entropy decoding unit (310), i.e., quantized transformation coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive residual signals (residual blocks, residual samples, residual sample array). Additionally, among the information decoded in the entropy decoding unit (310), information regarding filtering can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), inverse transform unit (322), adder (340), filtering unit (350), memory (360), inter prediction unit (332), and intra prediction unit (331).

[0088] In the inverse quantization unit (321), the quantized transformation coefficients can be inversely quantized to output transformation coefficients. The inverse quantization unit (321) can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device (200). The inverse quantization unit (321) can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain transformation coefficients.

[0089] In the inverse conversion unit (322), the conversion coefficients can be inversely converted to obtain a residual signal (residual block, residual sample array).

[0090] The prediction unit (330) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and palette index can be included in the above video / image information and signaled.

[0091] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The description of the intra prediction unit (222) may be applied equally to the intra prediction unit (331). The referenced samples may be located in the neighborhood of the current block or located away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit (331) may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0092] The inter prediction unit (332) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-prediction can be performed based on various prediction modes (techniques), and information regarding the prediction may include information indicating the mode (technique) of inter-prediction for the current block.

[0093] The adder (340) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (330) (including the inter prediction unit (332) and / or intra prediction unit (331)). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block. The description of the adder (250) can be applied equally to the adder (340). The adder (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0094] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0095] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (360), specifically in the DPB of memory (360). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0096] The (modified) restored picture stored in the DPB of the memory (360) can be used as a reference picture in the inter-prediction unit (332). The memory (360) can store motion information of blocks from which motion information within the current picture has been derived (or decoded) and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (332) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (360) can store restoration samples of blocks restored within the current picture and transmit them to the intra-prediction unit (331).

[0097] In this specification, the embodiments described in the filtering unit (260), inter prediction unit (221), and intra prediction unit (222) of the encoding device (200) may be applied to the filtering unit (350), inter prediction unit (332), and intra prediction unit (331) of the decoding device (300) in the same or corresponding manner.

[0098] Figure 4 illustrates an exemplary hierarchical structure for a coded video / image.

[0099] Referring to Figure 4, the coded image is divided into a VCL (video coding layer) that handles the decoding processing of the image and the image itself, a subsystem that transmits and stores the encoded information, and a NAL (network abstraction layer) that exists between the VCL and the subsystem and is responsible for network adaptation functions.

[0100] In VCL, VCL data containing compressed image data (slice data) can be generated, or parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages that are additionally required in the decoding process of the image can be generated.

[0101] In NAL, a NAL unit can be created by adding header information (NAL unit header) to the Raw Byte Sequence Payload (RBSP) generated in VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated in VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the NAL unit.

[0102] As shown in FIG. 4, NAL units can be classified into VCL NAL units and Non-VCL NAL units depending on the RBSP generated in VCL. A VCL NAL unit may refer to a NAL unit containing information about an image (slice data), and a Non-VCL NAL unit may refer to a NAL unit containing information necessary to decode an image (parameter set or SEI message).

[0103] The aforementioned VCL NAL unit and Non-VCL NAL unit can be transmitted over a network by attaching header information according to the data specifications of the underlying system. For example, the NAL unit can be transformed into a data format of a specified specification, such as H.266 / VVC file format, RTP (Real-time Transport Protocol), TS (Transport Stream), etc., and transmitted over various networks.

[0104] As described above, the NAL unit type can be determined according to the RBSP data structure included in the NAL unit, and information about this NAL unit type can be stored in the NAL unit header and signaled.

[0105] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether they contain information about the image (slice data). VCL NAL unit types can be classified according to the properties and types of the picture included in the VCL NAL unit, while Non-VCL NAL unit types can be classified according to the types of parameter sets.

[0106] The following is an example of a NAL unit type specified according to the type of parameter set included in the Non-VCL NAL unit type.

[0107] - APS (Adaptation Parameter Set) NAL unit: Type for the NAL unit containing the APS

[0108] - DPS (Decoding Parameter Set) NAL unit: Type for the NAL unit containing the DPS

[0109] - VPS (Video Parameter Set) NAL unit: Type for the NAL unit containing the VPS

[0110] - SPS (Sequence Parameter Set) NAL unit: Type for the NAL unit containing the SPS

[0111] - PPS(Picture Parameter Set) NAL unit: Type for the NAL unit containing the PPS

[0112] The above-described NAL unit types have syntax information for the NAL unit type, and said syntax information can be stored in the NAL unit header and signaled. For example, said syntax information may be nal_unit_type, and NAL unit types may be specified by the nal_unit_type value.

[0113] The slice header (slice header syntax) may include information / parameters that can be commonly applied to the slice. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be commonly applied to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that can be commonly applied to one or more sequences. The VPS (VPS syntax) may include information / parameters that can be commonly applied to multiple layers. The DPS (DPS syntax) may include information / parameters that can be commonly applied to the entire video. The DPS may include information / parameters related to the concatenation of the CVS (coded video sequence). In this document, the term High level syntax (HLS) may include at least one of the above APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, and slice header syntax.

[0114] In the present disclosure, image / video information encoded from an encoding device to a decoding device and signaled in the form of a bitstream includes not only information related to partitioning within a picture, intra / inter prediction information, residual information, in-loop filtering information, etc., but may also include information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, and / or information included in the VPS.

[0115] In the present disclosure, the following descriptors may define the parsing process for each syntax element.

[0116] - ae(v) is a context-adaptive arithmetic entropy-coded syntax element.

[0117] - b(8) is a byte with an arbitrary bit string pattern of length 8 bits. The parsing process of this descriptor is defined by the return value of the function read_bits(8).

[0118] - f(n) is an n-bit fixed-pattern bit string where the left bits are written first. The parsing process of this descriptor is defined by the return value of the function read_bits(n).

[0119] - i(n) is a signed integer using n bits, where n is denoted as "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process of this descriptor is defined by interpreting the return value of the function read_bits(n) into a two's complement integer representation where the most significant bit is written first.

[0120] - se(v) is a 0th-order Exp-Golomb encoded signed integer syntax element, written starting from the left bit. The parsing process of this descriptor is defined with order k set to 0.

[0121] - st(v) is a null-terminated string encoded in UCS (Universal Coded Character Set) transmission format 8 (UTF-8) characters as defined in ISO / IEC 10646. The parsing process is defined as follows.

[0122] st(v) starts at a byte-aligned position within the bitstream and reads and returns a sequence of consecutive bytes from the bitstream, starting from the current position and excluding the next byte-aligned byte with a value of 0x00. The bitstream pointer then advances by (stringLength + 1) * 8 bit positions, where stringLength is equal to the number of returned bytes.

[0123] For reference, the st(v) syntax descriptor is used in this disclosure only when the current position in the bitstream is a byte-aligned position.

[0124] - tu(v) is a truncated unary encoding scheme that uses up to the number of bits defined by the maximum value (maxVal), and maxVal is defined in the semantics of the corresponding syntax element.

[0125] - u(n) is an unsigned integer using n bits. If n is marked as "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process of this descriptor is defined by interpreting the return value of the function read_bits(n) as a binary representation of an unsigned integer where the most significant bit is written first.

[0126] - ue(v) is a 0th-order Exp-Golomb encoded unsigned integer syntax element, written starting from the left bit. The parsing process of this descriptor is defined with order k set to 0.

[0127] The SEI message related to the present disclosure will be described below.

[0128] Table 1 shows an example of a thumbnail source information (TSI) SEI message syntax according to one embodiment.

[0129] [Table 1]

[0130]

[0131] An example of the semantics of a thumbnail source information (TSI) SEI message according to one embodiment is described.

[0132] Thumbnail Source Information (TSI) SEI messages provide information about the source of a thumbnail image or video for associated video content.

[0133] If a TSI SEI message exists in any AU of a CVS, a TSI SEI message must exist in the first AU of that CVS. All TSI SEI messages within a CVS must have the same content.

[0134] tsi_thumbnail_id contains an identification number that can be used to uniquely identify a thumbnail image or video within a bitstream.

[0135] tsi_source_type indicates the source of the thumbnail image or video as specified in Table 2.

[0136] [Table 2]

[0137]

[0138] If tsi_layer_id exists, it specifies the layer identifier of the layer from which the thumbnail image or video is obtained.

[0139] If present, tsi_uri identifies the source of a thumbnail image or video and includes a URI having the syntax and semantics specified by IETF Internet Standard 66.

[0140] A tsi_region_info_present_flag equal to 1 indicates that the tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y syntax elements exist in the thumbnail extraction information SEI message associated with the current TSI SEI message. A tsi_region_info_present_flag equal to 0 indicates that the tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y syntax elements do not exist in the thumbnail extraction information SEI message associated with the current TSI SEI message.

[0141] Table 3 below shows an example of a thumbnail extraction information (TEI) SEI message syntax according to one embodiment.

[0142] [Table 3]

[0143]

[0144] An example of the semantics of a thumbnail extraction information (TEI) message according to one embodiment is described.

[0145] A Thumbnail Extraction Information (TEI) SEI message indicates that one or more pictures to which the SEI message applies, or a specific area of ​​the picture(s), is a thumbnail image or video for the associated video content.

[0146] If there is a TSI SEI message in CLVS where tsi_source_type is equal to 0 and tsi_thumbnail_id is equal to curThumbId, then there must be at least one TEI SEI message in CLVS where tei_thumbnail_id is equal to curThumbId. If there is a TEI SEI message in CLVS where tei_thumbnail_id is equal to curThumbId, then there must be a TSI SEI message in CVS where tsi_source_type is less than 2 and tsi_thumbnail_id is equal to curThumbId.

[0147] tei_thumbnail_id includes an identification number that can be used to uniquely identify a thumbnail image or video within a bitstream.

[0148] A tei_cancel_flag equal to 1 indicates that the TEI SEI message cancels the persistence of the previous TEI SEI message in the output order applied to the current layer. A tei_cancel_flag equal to 0 indicates that thumbnail extraction information follows.

[0149] tei_persistence_flag specifies the persistence of thumbnail extraction information provided by this SEI message. tei_persistence_flag equal to 0 specifies that thumbnail extraction information is applied only to the current picture. tei_persistence_flag equal to 1 specifies that thumbnail extraction information is applied to the current picture and all subsequent pictures of the current layer in the output order, provided that it is applied until one or more of the following conditions are true.

[0150] - New CLVS of the current layer have started.

[0151] - Bitstream terminated.

[0152] - The picture of the current layer associated with the TEI SEI message is output, and that picture is positioned after the current picture in the output order.

[0153] If tei_width_minus1 and tei_height_minus1 exist, tei_width_minus1 + 1 and tei_height_minus1 + 1 specify the width and height of the corresponding area in units of luma samples, respectively.

[0154] As a requirement for bitstream conformity, (tei_width_minus1 + 1) % SubWidthC must be equal to 0, and (tei_height_minus1 + 1) % SubHeightC must be equal to 0.

[0155] If tei_top_left_x and tei_top_left_y exist, tei_top_left_x and tei_top_left_y specify the horizontal and vertical positions of the top-left sample of the corresponding area, respectively.

[0156] As a requirement for bitstream conformity, tei_top_left_x % SubWidthC must be equal to 0, and tei_top_left_y % SubHeightC must be equal to 0.

[0157] The area(s) indicated by tei_width_minus1, tei_height_minus1, tei_top_left_x and tei_top_left_y are thumbnail images or videos, if the corresponding syntax element exists. If tei_width_minus1, tei_height_minus1, tei_top_left_x and tei_top_left_y do not exist, the picture(s) to which the TEI SEI message applies are thumbnail images or videos.

[0158] Thumbnails serve as a means to facilitate previewing and navigation of video content. They are typically created by extracting and editing specific portions of the original video, or provided as separately encoded video content. Depending on the implementation, they may be presented as full-screen preview videos or as small icons to support efficient content browsing.

[0159] Since thumbnails can be generated by extracting and editing specific parts of the original video content, it is useful to include an SEI message indicating the source of the thumbnail and how the original video content was extracted and edited to obtain the thumbnail. This approach provides common signaling across different systems and devices while ensuring a clear association between the original content and the corresponding thumbnail. Additionally, integrating thumbnail signaling within the video stream can improve storage efficiency and facilitate synchronization between the thumbnail and video playback.

[0160] Accordingly, the present invention proposes SEI messages for indicating the source and extraction information of a thumbnail video of an image.

[0161] Thumbnails are representative images or videos that provide a quick preview of content, helping users identify and navigate videos more easily. They can be determined by creators or content providers to reflect their intentions, and nowadays, AI is also increasingly being used to automatically generate thumbnails based on content analysis.

[0162] Thumbnails can be generated by extracting and editing temporal and / or spatial portions of the original video content. They may also be provided as separately encoded video content. The encoded video content of the thumbnail may be a specific layer of the current bitstream, or it may be a bitstream separate from the bitstream of the original video content.

[0163] In Content Provider services, thumbnails are frequently used as full-screen preview videos to enhance the browsing experience. These previews allow users to quickly grasp the essence of the content before playback, making it easier to choose what to watch.

[0164] On user devices, thumbnails are typically generated by capturing a portion of a video and displaying it as an icon. These icons help users efficiently navigate videos they have saved or recently played, and provide a convenient way to browse their content library.

[0165] Each item according to one embodiment may be applied individually or in combination.

[0166] 1. If the thumbnail contains two or more segments and the segments contain one or more pictures, add a syntax element that indicates the output order of the segments.

[0167] 2. Add syntax elements that indicate the display duration of the pictures within the thumbnail.

[0168] 3. Add syntax elements that indicate luminance gain for brightness expression of pictures.

[0169] 4. Add syntax elements that indicate blank pictures with a constant sample value following the current picture.

[0170] One embodiment relates to Item 1 described above. One embodiment is based on the VSEI and VVC specifications. The thumbnail source information (TSI) SEI message syntax elements according to one embodiment are as shown in Table 4 below.

[0171] [Table 4]

[0172]

[0173] An example of semantics of a Thumbnail Source Information (TSI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Source Information SEI message described above are described, and descriptions of semantics identical to the semantics of the Thumbnail Source Information SEI message described above are replaced with descriptions of the semantics of the Thumbnail Source Information SEI message described above.

[0174] A tsi_segment_output_order_present_flag equal to 1 may indicate that the syntax element tei_segment_output_order exists in the TEI SEI message associated with the current TSI SEI message. A tsi_segment_output_order_present_flag equal to 0 may indicate that the syntax element tei_segment_output_order does not exist in the TEI SEI message associated with the current TSI SEI message.

[0175] The thumbnail extraction information (TEI) SEI message syntax elements according to one embodiment are as shown in Table 5 below.

[0176] [Table 5]

[0177]

[0178] An example of the semantics of a Thumbnail Extraction Information (TEI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Extraction Information SEI message described above are described, and any description of semantics identical to the semantics of the Thumbnail Extraction Information SEI message described above is replaced by the description of the semantics of the Thumbnail Extraction Information SEI message described above.

[0179] tei_segment_output_order can specify the output order of the current segment to which the current picture belongs, where a segment may contain one or more consecutive pictures in the output order, and a thumbnail may contain two or more segments. Pictures to which TEI SEI messages with the same tei_segment_output_order value are applied may belong to the same segment. A segment with a tei_segment_output_order value equal to segOutOrderA may be designated as segA, and a segment with a tei_segment_output_order value equal to segOutOrderB may be designated as segB. If segOutOrderB is greater than segOutOrderA, segB may follow segA in the output order within the thumbnail. The output order of pictures within the same segment may be the same as the output order within the current bitstream. The value of tei_segment_output_order must be within the range of 0 to 255 (inclusive).

[0180] Alternatively, tei_segment_output_order may specify the output order of the current segment to which the current picture belongs, where a segment may contain one or more consecutive pictures in the output order, and a thumbnail may contain two or more segments. Pictures to which TEI SEI messages with the same tei_segment_output_order value are applied may belong to the same segment. A segment with a tei_segment_output_order value equal to segOutOrderA may be designated as segA, and a segment with a tei_segment_output_order value equal to segOutOrderB may be designated as segB. If segOutOrderB is greater than segOutOrderA, segB may follow segA in the output order within the thumbnail. The output order of pictures within the same segment may be the same as the output order within the current bitstream. The value of tei_segment_output_order must be within the range of 0 to 255 (inclusive).

[0181] If the corresponding syntax does not exist, the value of tei_segment_output_order can be inferred as follows.

[0182] - If SEI is the first TEI SEI message in the bitstream based on output order, the value of tei_segment_output_order can be inferred to be equal to 0.

[0183] - In other cases, the value of tei_segment_output_order can be inferred to be the same as the value of the tei_segment_output_order of the previous TEI SEI message plus 1, based on the output order.

[0184] One embodiment relates to Item 2 described above. One embodiment is based on the VSEI and VVC specifications. The syntax elements of the Thumbnail Source Information (TSI) SEI message according to one embodiment are as shown in Table 6 below.

[0185] [Table 6]

[0186]

[0187] An example of semantics of a Thumbnail Source Information (TSI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Source Information SEI message described above are described, and descriptions of semantics identical to the semantics of the Thumbnail Source Information SEI message described above are replaced with descriptions of the semantics of the Thumbnail Source Information SEI message described above.

[0188] A tsi_timing_info_present_flag (or tsi_timing_info_present) equal to 1 may indicate that the syntax element tsi_time_scale exists in the SEI message and that the syntax element tei_num_units_in_dur exists in the TEI SEI message associated with the current TSI SEI message. A tsi_timing_info_present_flag (or tsi_timing_info_present) equal to 0 may indicate that the syntax element tsi_time_scale does not exist in the SEI message and that the syntax element tei_num_units_in_dur does not exist in the TEI SEI message associated with the current TSI SEI message.

[0189] tsi_time_scale can be the number of time units that elapse in one second. For example, a time coordinate system that measures time using a 27 MHz clock has a tsi_time_scale of 27,000,000. The value of tsi_time_scale must be greater than 0.

[0190] The thumbnail extraction information (TEI) SEI message syntax elements according to one embodiment are as shown in Table 7 below.

[0191] [Table 7]

[0192]

[0193] An example of the semantics of a Thumbnail Extraction Information (TEI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Extraction Information SEI message described above are described, and any description of semantics identical to the semantics of the Thumbnail Extraction Information SEI message described above is replaced by the description of the semantics of the Thumbnail Extraction Information SEI message described above.

[0194] tei_num_units_in_dur may be the number of time units corresponding to the display duration of the current picture among the time units of a clock operating at a frequency of tsi_time_scale Hz. tei_num_units_in_dur must be greater than 0. If the display duration of the current picture is expressed in seconds, it may be equal to the quotient obtained by dividing tei_num_units_in_dur by tsi_time_scale. For example, if the display duration of the current picture is 0.04 seconds, tsi_time_scale and tei_num_units_in_dur may be 27,000,000 and 1,080,000, respectively.

[0195] One embodiment relates to Item 3 described above. One embodiment is based on the VSEI and VVC specifications. The syntax elements of the Thumbnail Source Information (TSI) SEI message according to one embodiment are as shown in Table 8 below.

[0196] [Table 8]

[0197]

[0198] An example of semantics of a Thumbnail Source Information (TSI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Source Information SEI message described above are described, and descriptions of semantics identical to the semantics of the Thumbnail Source Information SEI message described above are replaced with descriptions of the semantics of the Thumbnail Source Information SEI message described above.

[0199] A tsi_luminance_gain_factor_present_flag equal to 1 may indicate that syntax elements tsi_luminance_gain_factor_num_minus1 and tsi_luminance_gain_factor_denom_minus1 exist in the TEI SEI message associated with the current TSI SEI message. A tsi_luminance_gain_factor_present_flag equal to 0 may indicate that syntax elements tei_luminance_gain_factor_num_minus1 and tei_luminance_gain_factor_denom_minus1 do not exist in the TEI SEI message associated with the current TSI SEI message.

[0200] The syntax elements of the Thumbnail Extraction Information (TEI) SEI message according to one embodiment are as shown in Table 9 below.

[0201] [Table 9]

[0202]

[0203] An example of the semantics of a Thumbnail Extraction Information (TEI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Extraction Information SEI message described above are described, and any description of semantics identical to the semantics of the Thumbnail Extraction Information SEI message described above is replaced by the description of the semantics of the Thumbnail Extraction Information SEI message described above.

[0204] The use of this SEI message may require the definition of the following variables.

[0205] - Bit depth for the sample of the luma component, which may be denoted as BitDepthY below.

[0206] The value obtained by adding 1 to tei_luminance_gain_factor_num_minus1 and the value obtained by adding 1 to tei_luminance_gain_factor_denom_minus1 may specify the numerator and denominator, respectively, of the luminance gain factor for the thumbnail picture. Both tei_luminance_gain_factor_num_minus1 and tei_luminance_gain_factor_denom_minus1 must be within the range of 0 to 65,535 (inclusive).

[0207] The luma sample value of any luma sample in the current picture can be denoted as curPicYVal, and the corresponding luma sample value in the thumbnail picture can be denoted as thumbPicYVal.

[0208] thumbPicYVal can be derived as follows.

[0209] thumbYVal = Max( ( 1 << BitDepth Y ) - 1,

[0210] Round( curPicYVal * ( tei_luminance_gain_factor_num_minus1 + 1 ) χ ( tei_luminance_gain_factor_denom_minus1 + 1 ) ) )

[0211] One embodiment relates to Item 4 described above. One embodiment is based on the VSEI and VVC specifications. The syntax elements of the Thumbnail Source Information (TSI) SEI message according to one embodiment are as shown in Table 10 below.

[0212] [Table 10]

[0213]

[0214] An example of semantics of a Thumbnail Source Information (TSI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Source Information SEI message described above are described, and descriptions of semantics identical to the semantics of the Thumbnail Source Information SEI message described above are replaced with descriptions of the semantics of the Thumbnail Source Information SEI message described above.

[0215] A tsi_timing_info_present_flag (or tsi_timing_info_present) identical to 1 may indicate that the syntax element tsi_time_scale exists in the SEI message, and that the syntax element tei_num_units_in_dur exists in the TEI SEI message associated with the current TSI SEI message, and that the syntax elements tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] may exist in the TEI SEI message associated with the current TSI SEI message. A tsi_timing_info_present_flag (or tsi_timing_info_present) equal to 0 may indicate that the syntax element tsi_time_scale does not exist in the SEI message, and that the syntax elements tei_num_units_in_dur, tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] do not exist in the TEI SEI message associated with the current TSI SEI message.

[0216] tsi_time_scale can be the number of time units that elapse in one second. For example, a time coordinate system that measures time using a 27 MHz clock has a tsi_time_scale of 27,000,000. The value of tsi_time_scale must be greater than 0.

[0217] The syntax elements of the Thumbnail Extraction Information (TEI) SEI message according to one embodiment are as shown in Table 11 below.

[0218] [Table 11]

[0219]

[0220] An example of the semantics of a Thumbnail Extraction Information (TEI) SEI message according to one embodiment is described. Semantics different from the semantics of the Thumbnail Extraction Information SEI message described above are described, and any description of semantics identical to the semantics of the Thumbnail Extraction Information SEI message described above is replaced by the description of the semantics of the Thumbnail Extraction Information SEI message described above.

[0221] The use of this SEI message may require the definition of the following variables.

[0222] - As a chroma format indicator of the input picture, ChromaFormatIdc.

[0223] - Bit depth for the sample of the luma component, which may be denoted as BitDepthY below.

[0224] A tei_blank_pic_present_flag identical to 1 may indicate that syntax elements tei_num_blank_pics_minus1 and tei_blank_pic_sample[ i ][ c ] exist in the SEI message, and that syntax elements tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] exist in the SEI message. A tei_blank_pic_present_flag equal to 0 may indicate that the syntax elements tei_num_blank_pics_minus1, tei_blank_pic_sample[ i ][ c ], tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] do not exist in the SEI message.

[0225] The value obtained by adding 1 to tei_num_blank_pics_minus1 can represent the number of blank pictures following the current picture in the output order.

[0226] tei_blank_pic_sample[ c ] may represent the sample value for the c-th color component of the i-th blank picture following the current picture in the output order. The length of the syntax element must be BitDepthY bits.

[0227] tei_num_units_in_blank_pic_dur_present_flag[ i ] equal to 1 may indicate that the syntax element tei_num_units_in_blank_pic_dur[ i ] exists. tei_num_units_in_blank_pic_dur_present_flag[ i ] equal to 0 may indicate that the syntax element tei_num_units_in_blank_pic_dur[ i ] does not exist.

[0228] tei_num_units_in_blank_pic_dur[ i ] may be the number of time units corresponding to the display duration of the i-th blank picture among the time units of a clock operating at a frequency of tsi_time_scale Hz. tei_num_units_in_blank_pic_dur[ i ] must be greater than 0. If the display duration of the i-th blank picture is expressed in seconds, it is equal to the quotient obtained by dividing tei_num_units_in_blank_pic_dur[ i ] by tsi_time_scale. For example, if the display duration of the i-th blank picture is 0.04 seconds, tsi_time_scale and tei_num_units_in_dur may be 27,000,000 and 1,080,000, respectively.

[0229] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms, etc. described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0230] The operations described below do not constitute an essential component of one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of one embodiment, and previously described operations may be added. Moreover, unless they contradict previously described operations, the operations described below form one embodiment integrally with previously described operations and do not form a separate embodiment distinct from previously described operations.

[0231] FIG. 5 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0232] The decoding method (S500) may include the operations described below.

[0233] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0234] The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the decoding method according to one embodiment, and the previously described operations may be added.

[0235] The operations described below form a single embodiment integrally with the configurations and / or operations described above, unless they conflict with the configurations and / or operations described above, and do not form a separate embodiment distinct from the configurations and / or operations described above.

[0236] The decoding method (S500) can be executed by a decoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0237] A decoding device can acquire image information. For example, a processor of a decoding device can acquire image information. The image information may include a picture to be decoded.

[0238] For example, the video information may include thumbnail-related information that can be used to generate a thumbnail picture, as additional information related to at least one picture. The thumbnail-related information may include thumbnail source information and thumbnail extraction information, as information to support video preview and navigation. Additionally, to provide a thumbnail picture for the associated video content, the encoding device may generate a thumbnail-related message and provide it to a decoding device via a bitstream.

[0239] The decoding device can acquire a SEI (supplemental enhancement information) message (S510).

[0240] SEI messages can convey specific types of information that assist in processes related to the decoding, display, or other purposes of image information. Here, SEI messages may not be necessary for the decoding process to determine the sample values ​​of the decoded picture.

[0241] For example, SEI messages may include thumbnail-related SEI messages.

[0242] A thumbnail-related SEI message may include at least one of a TSI (Thumbnail Source Information) related SEI message associated with the source of the thumbnail picture and / or a TEI (Thumbnail Extraction Information) related SEI message associated with the extraction of the thumbnail picture. For example, a TSI-related SEI message may include source information of the thumbnail picture, and a TEI-related SEI message may include extraction information of the thumbnail picture.

[0243] TSI-related SEI messages may have various names such as thumbnail source information SEI message, thumbnail source information related message, thumbnail source information related information, thumbnail source message, thumbnail source information, thumbnail source information SEI message, thumbnail source information related message, thumbnail source information related information, thumbnail source message, thumbnail source information, TSI SEI message, TSI-related message, TSI-related information, TSI message, TSI information, etc., and such names are not limited.

[0244] TSI-related SEI messages can take various forms. For example, a TSI-related SEI message may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, a TSI-related SEI message may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. For example, a TSI-related SEI message may be represented as thumbnail_source_info( payloadSize ), but is not limited thereto.

[0245] TEI-related SEI messages may have various names such as thumbnail extraction information SEI message, thumbnail extraction information related message, thumbnail extraction information related information, thumbnail extraction message, thumbnail extraction information, TEI SEI message, TEI-related message, TEI-related information, TEI message, TEI information, etc., and the names are not limited.

[0246] TEI-related SEI messages can take various forms. For example, a TEI-related SEI message may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, a TEI-related SEI message may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. For example, a TEI-related SEI message may be represented as thumbnail_extraction_info( payloadSize ), but is not limited thereto.

[0247] For example, a decoding device can obtain a thumbnail-related SEI message associated with a picture from a bitstream. For example, a processor of the decoding device can parse the bitstream to obtain at least one SEI message applied to the picture, and among them, obtain a thumbnail-related SEI message for conveying information related to the thumbnail.

[0248] The decoding device can obtain thumbnail-related information from a thumbnail-related SEI message. For example, the thumbnail-related information may include at least one of thumbnail identifier information, thumbnail source information, thumbnail extraction area information, thumbnail size information, thumbnail location information, thumbnail cancellation information, thumbnail persistence information, segment output order information of the thumbnail picture, display duration information, luminance gain factor information, and / or blank picture information.

[0249] The decoding device can obtain thumbnail-related information based on the SEI message (S520).

[0250] The decoding device can obtain thumbnail-related information for a decoded picture based on at least one thumbnail-related SEI message. For example, the decoding device can obtain at least one of thumbnail identifier information, thumbnail source information, thumbnail extraction information, thumbnail size information, thumbnail location information, thumbnail cancellation information, thumbnail persistence information, segment output order information of the thumbnail picture, display duration information, luminance gain factor information, and / or blank picture information from at least one thumbnail-related SEI message.

[0251] A thumbnail-related SEI message may include at least one of a TSI (Thumbnail Source Information) related SEI message associated with the source of a thumbnail picture and / or a TEI (Thumbnail Extraction Information) related SEI message associated with the extraction of a thumbnail picture. For example, a TSI-related SEI message may include source information of the thumbnail picture, and a TEI-related SEI message may include extraction information of the thumbnail picture. Accordingly, the thumbnail-related information may include source information and extraction information of the thumbnail picture.

[0252] Additionally, the decoding device can identify a thumbnail picture corresponding to a decoded picture (or a picture to be decoded) based on a thumbnail-related SEI message. For example, if the thumbnail-related SEI message includes a TSI-related SEI message, the decoding device can obtain at least one of source type information, layer identifier information, or URI information indicating the source of the thumbnail picture from the TSI-related SEI message, and based thereon, determine whether the thumbnail picture is derived from a picture of the current layer, from another layer within the current bitstream, or from another bitstream.

[0253] Additionally, if a thumbnail-related SEI message includes a TEI-related SEI message, the decoding device can obtain persistence information and cancellation information indicating the scope of application of thumbnail extraction information, as well as thumbnail size information and thumbnail location information, from the TEI-related SEI message. At this time, the decoding device can identify the area within the picture indicated by the thumbnail size information and thumbnail location information as a thumbnail, and if the information is not present, the entire picture to which the TEI-related SEI message applies can be identified as a thumbnail.

[0254] A TSI-related SEI message may include at least one of TSI identifier information for identifying a thumbnail picture within a bitstream, thumbnail source information indicating the source of the thumbnail picture, thumbnail layer information, thumbnail URI information, and thumbnail extraction area existence information.

[0255] TSI identifier information for representing a thumbnail picture within a TSI-related SEI message may include an identification number that can be used to uniquely identify a thumbnail image or video within a bitstream.

[0256] TSI identifier information for representing a thumbnail picture within a TSI-related SEI message may take various forms and be expressed by various names. For example, TSI identifier information for representing a thumbnail picture may be a syntax element or a syntax structure containing one or more syntax elements. For example, TSI identifier information for representing a thumbnail picture may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, TSI identifier information for representing a thumbnail picture may include tsi_thumbnail_id, but is not limited thereto.

[0257] Thumbnail source information may indicate the source of a thumbnail image or video. For example, thumbnail source information may indicate whether the thumbnail picture was derived from the current layer within the bitstream, another layer different from the current layer, or another bitstream different from the bitstream.

[0258] Thumbnail source information may take various forms and may be represented by various names. For example, thumbnail source information may be a syntax element or a syntax structure containing one or more syntax elements. For example, thumbnail source information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, thumbnail source information may include tsi_source_type, etc., but is not limited thereto.

[0259] For example, when thumbnail source information is expressed as tsi_source_type, tsi_source_type may indicate the source of the thumbnail image or video as specified in Table 2 above.

[0260] Thumbnail source information may be information indicating from which source a thumbnail image or thumbnail video was obtained or sourced. For example, if the value of the thumbnail source information is 0, it may indicate that the thumbnail image or video is sourced from one or more pictures within the current layer. Additionally, if the value of the thumbnail source information is 1, it may indicate that the thumbnail image or video is sourced from a different layer within the current bitstream that is different from the current layer. Additionally, if the value of the thumbnail source information is 2, it may indicate that the thumbnail image or video is sourced from a different bitstream that is different from the current bitstream. Meanwhile, if the value of the thumbnail source information is 3, the value may be a reserved value and may be a value predefined for future extension, and the value used in the embodiments of the present disclosure is not limited thereto. Accordingly, the decoding device can determine the reference target of the thumbnail based on the thumbnail source information, identify the thumbnail using the picture of the current layer, or provide a video preview by accessing and linking thumbnail content existing in another layer or another bitstream.

[0261] Thumbnail layer information may represent the layer identifier of the layer from which the thumbnail image or video is obtained.

[0262] Thumbnail layer information may take various forms and may be represented by various names. For example, thumbnail layer information may be a syntax element or a syntax structure containing one or more syntax elements. For example, thumbnail layer information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, thumbnail layer information may include tsi_layer_id, etc., but is not limited thereto.

[0263] For example, when thumbnail layer information is represented as tsi_layer_id, tsi_layer_id may represent the layer identifier of the layer from which the thumbnail image or video is obtained, if present.

[0264] Thumbnail URI information identifies the source of a thumbnail image or video and may include a URI having syntax and semantics.

[0265] Thumbnail URI information may take various forms and may be represented by various names. For example, thumbnail layer information may be a syntax element or a syntax structure containing one or more syntax elements. For example, thumbnail URI information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, thumbnail URI information may include tsi_uri, but is not limited thereto.

[0266] For example, when thumbnail URI information is represented as tsi_uri, tsi_uri, if present, may include a URI having the syntax and semantics specified by IETF Internet Standard 66 as identifying the source of the thumbnail image or video.

[0267] Thumbnail extraction area existence information may indicate whether the thumbnail extraction information (TEI) associated with the current TSI-related SEI message contains syntax elements representing the thumbnail extraction (or capture) target area.

[0268] For example, thumbnail extraction area existence information with a value equal to 1 may indicate that the syntax elements tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y exist in the thumbnail extraction information SEI message associated with the current TSI SEI message. Additionally, thumbnail extraction area existence information with a value equal to 0 may indicate that the syntax elements tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y do not exist in the thumbnail extraction information SEI message associated with the current TSI SEI message. However, this is not limited thereto, and what thumbnail extraction area existence information with a value equal to 1 indicates may be interchangeable with what thumbnail extraction area existence information with a value equal to 0 indicates.

[0269] Information regarding the existence of a thumbnail extraction region may take various forms and may be expressed by various names. For example, information regarding the existence of a thumbnail extraction region may be a syntax element or a syntax structure containing one or more syntax elements. For example, information regarding the existence of a thumbnail extraction region may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, information regarding the existence of a thumbnail extraction region may include tsi_region_info_present_flag or tsi_region_info_present_idc, but is not limited thereto.

[0270] For example, if the thumbnail extraction area existence information is represented by tsi_region_info_present_flag, tsi_region_info_present_flag equal to 1 may indicate that the tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y syntax elements exist in the thumbnail extraction information SEI message associated with the current TSI SEI message. tsi_region_info_present_flag equal to 0 may indicate that the tei_width_minus1, tei_height_minus1, tei_top_left_x, and tei_top_left_y syntax elements do not exist in the thumbnail extraction information SEI message associated with the current TSI SEI message.

[0271] Meanwhile, thumbnail-related information may include at least one of segment output order information of the thumbnail picture, display duration information, luminance gain factor information, or blank picture information.

[0272] For example, based on the segment output order information above, the output order of the segment to which the current picture belongs within the thumbnail picture can be determined. Here, the segment output order information of the thumbnail picture may include output order existence information included in the TSI-related SEI message and output order information included in the EI-related SEI message.

[0273] Output order existence information may indicate whether the TEI-related SEI message associated with the current TSI-related SEI message contains syntax elements of output order information that indicate the segment output order. Additionally, output order existence information with a value equal to 1 may indicate that the TEI-related SEI message associated with the current TSI-related SEI message contains syntax elements of output order information. Additionally, output order existence information with a value equal to 0 may indicate that the TEI-related SEI message associated with the current TSI-related SEI message does not contain syntax elements of output order information. However, this is not limited thereto, and what output order existence information with a value equal to 1 indicates may be interchangeable with what output order existence information with a value equal to 0 indicates.

[0274] Output order present information may take various forms and may be represented by various names. For example, output order present information may be a syntax element or a syntax structure containing one or more syntax elements. For example, output order present information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, output order present information may include tsi_segment_output_order_present_flag or tsi_segment_output_order_present_idc, but is not limited thereto.

[0275] For example, if output order existence information is represented by tsi_segment_output_order_present_flag, tsi_segment_output_order_present_flag equal to 1 may indicate that the syntax element tei_segment_output_order exists in the TEI-related SEI message associated with the current TSI SEI message. tsi_segment_output_order_present_flag equal to 0 may indicate that the syntax element tei_segment_output_order does not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0276] For example, output order existence information can be obtained based on thumbnail source information. That is, the decoding device can obtain output order existence information based on thumbnail source information. For example, if the thumbnail source information indicates that the thumbnail image or video is sourced from one or more pictures within the current layer, or indicates that the thumbnail image or video is sourced from a different layer within the current bitstream that is different from the current layer, the decoding device can obtain output order existence information.

[0277] Output order information included in the TEI-related SEI message can be obtained based on the output order existence information included in the TSI-related SEI message. That is, if the output order existence information indicates that output order information is included within the TEI-related SEI message, the decoding device can obtain output order information from the TEI-related SEI message associated with the current TSI-related SEI message. On the other hand, if the output order existence information indicates that output order information is not included within the TEI-related SEI message, the decoding device may not obtain output order information from the TEI-related SEI message.

[0278] Output order information may indicate the output order of the current segment to which the current picture belongs. Here, a segment may include one or more consecutive pictures in the output order, and a thumbnail may include two or more segments.

[0279] Output order information may take various forms and may be represented by various names. For example, output order information may be a syntax element or a syntax structure containing one or more syntax elements. For example, output order information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, output order information may include tei_segment_output_order, but is not limited thereto.

[0280] For example, when output order information is represented as tei_segment_output_order, tei_segment_output_order may indicate the output order of the current segment to which the current picture belongs. Here, a segment may include one or more consecutive pictures in the output order, and a thumbnail may include two or more segments. Additionally, the value of tei_segment_output_order must be within the range of 0 to 255 (inclusive).

[0281] Additionally, pictures to which TEI-related SEI messages containing the same output order information are applied may be included in the same segment. That is, the decoding device can group multiple pictures to which TEI-related SEI messages having the same output order information value into a single segment by using the value of the output order information included in or inferred from the TEI-related SEI message.

[0282] For example, if output order information is represented as tei_segment_output_order, a segment with a tei_segment_output_order value equal to segOutOrderA can be called segA, and a segment with a tei_segment_output_order value equal to segOutOrderB can be called segB. In this case, if segOutOrderB is greater than segOutOrderA, segB may follow segA in the output order within the thumbnail. Additionally, the output order of pictures included within the same segment may be maintained as the same as the output order within the current bitstream. Accordingly, the decoding device identifies segments based on the value of the output order information and determines the output order between segments, thereby enabling the configuration or provision of thumbnails on a segment basis even when the thumbnail includes multiple segments.

[0283] Additionally, if the output order information is not included in the TEI-related SEI message, the decoding device may infer and use the value of the output order information. For example, the output order information may be inferred to be 0 based on the fact that the TEI-related SEI message is the first TEI-related SEI message in the bitstream according to the output order.

[0284] Meanwhile, based on the fact that the TEI-related SEI message does not contain output order information and that the TEI-related SEI message is not the first TEI-related SEI message in the bitstream based on the output order, the value of the output order information can be inferred as the value of the output order information of the previous TEI-related SEI message plus 1 based on the output order.

[0285] That is, if the syntax element of the output order information is not included in the above TEI-related SEI message, the value of the output order information can be inferred as follows. If the SEI is the first TEI-related SEI message in the bitstream based on the output order, the value of the output order information can be inferred to be equal to 0. Otherwise, the value of the output order information can be inferred to be equal to the value of the output order information of the immediately preceding TEI-related SEI message plus 1 based on the output order. Accordingly, the decoding device can determine the output order of the segment to which the current picture belongs within the thumbnail by obtaining the output order information from the TEI-related SEI message based on the output order existence information, or by inferring the value of the output order information according to the inference rule as above.

[0286] However, the inference method and application conditions of the output order information described above may be merely exemplary explanations to aid understanding, and the scope of the present invention is not limited thereto. For example, the inference value when the output order information is not included in the TEI-related SEI message, the criteria used for inference (e.g., the "first" judgment method of the output order criteria), and the value added to the value of the preceding TEI-related SEI message (e.g., 1) may vary depending on the implementation or design, and various modifications may be possible within the scope of achieving the same technical concept and effect.

[0287] Meanwhile, the display duration information may include at least one of the duration existence information, time scale information, or duration unit information included in the TEI-related SEI message.

[0288] Duration existence information may indicate whether a syntax element of time scale information exists in the SEI message and whether a syntax element of duration unit information exists in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, duration existence information with a value equal to 1 may indicate that a syntax element of time scale information exists in the SEI message and that a syntax element of duration unit information exists in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, duration existence information with a value equal to 0 may indicate that a syntax element of time scale information does not exist in the SEI message and that a syntax element of duration unit information does not exist in the TEI-related SEI message associated with the current TSI-related SEI message. However, this is not limited thereto, and what duration existence information with a value equal to 1 indicates may be interchangeable with what duration existence information with a value equal to 0 indicates.

[0289] Timing information may take various forms and may be represented by various names. For example, the timing information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the timing information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the timing information may include tsi_timing_info_present, tsi_timing_info_present_flag, or tsi_timing_info_present_idc, but is not limited thereto.

[0290] For example, if duration existence information is represented by tsi_timing_info_present_flag, tsi_timing_info_present_flag equal to 1 may indicate that the syntax element tsi_time_scale exists in the SEI message and that the syntax element tei_num_units_in_dur exists in the TEI-related SEI message associated with the current TSI-related SEI message. tsi_timing_info_present_flag equal to 0 may indicate that the syntax element tsi_time_scale does not exist in the SEI message and that the syntax element tei_num_units_in_dur does not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0291] For example, time scale information included in a TSI-related SEI message and duration unit information included in a TEI-related SEI message can be obtained based on the duration existence information. That is, the decoding device can obtain at least one of the time scale information or the duration unit information based on the duration existence information. For example, if the duration existence information indicates that duration-related information is included within a thumbnail-related SEI message, the decoding device can obtain time scale information from the current TSI-related SEI message and obtain duration unit information from a TEI-related SEI message associated with the current TSI-related SEI message. On the other hand, if the duration existence information indicates that duration-related information is not included within a thumbnail-related SEI message, the decoding device may not obtain the time scale information and / or the duration unit information. Accordingly, by obtaining at least one of the time scale information or the duration unit information based on the duration existence information, the decoding device can obtain information for determining the display duration of the current picture within the thumbnail.

[0292] In addition, time scale information can indicate the number of time units that elapse in one second. For example, a time coordinate system that measures time using a 27 MHz clock has a time scale information value of 27,000,000. The value of the time scale information must be greater than 0.

[0293] Time scale information may take various forms and may be represented by various names. For example, time scale information may be a syntax element or a syntax structure containing one or more syntax elements. For example, time scale information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, time scale information may include tsi_time_scale, etc., but is not limited thereto.

[0294] The duration unit information may represent the number of time units corresponding to the display duration of the current picture among the time units of a clock operating with the above frequency time scale information (Hz). The value of the duration unit information must be greater than 0.

[0295] Duration unit information may take various forms and may be represented by various names. For example, duration unit information may be a syntax element or a syntax structure containing one or more syntax elements. For example, duration unit information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, duration unit information may include tei_num_units_in_dur, but is not limited thereto.

[0296] For example, if the duration unit information is expressed as tei_num_units_in_dur, tei_num_units_in_dur may be the number of time units corresponding to the display duration of the current picture among the time units of a clock operating at frequency tsi_time_scale (Hz).

[0297] Duration existence information, time scale information, and / or duration unit information can be obtained based on thumbnail source information. That is, the decoding device can obtain time scale information and / or duration unit information based on thumbnail source information. For example, if the thumbnail source information indicates that the thumbnail image or video is sourced from one or more pictures within the current layer, or indicates that the thumbnail image or video is sourced from a different layer within the current bitstream that is different from the current layer, the decoding device can obtain duration existence information, time scale information, and / or duration unit information.

[0298] Meanwhile, the display duration of the current picture can be determined based on time scale information and duration unit information. That is, the decoding device can determine the display duration of the current picture based on time scale information and duration unit information. For example, the display duration of the current picture can be determined based on dividing the duration unit information by the time scale information. That is, if the display duration of the current picture is expressed in seconds, it may be the same as the quotient obtained by dividing the duration unit information by the time scale information. For example, if the display duration of the current picture is 0.04 seconds, the time scale information and the duration unit information may be 27,000,000 and 1,080,000, respectively. Accordingly, the decoding device can calculate the display duration of the current picture based on the time scale information and duration unit information, and control the display time of the picture within the thumbnail according to the calculated display duration. However, the above figures and calculation methods are exemplary and are not limited thereto, and the settings of time scale information and duration unit information may vary depending on the implementation.

[0299] Meanwhile, the luminance gain factor information may include at least one of the luminance gain factor existence information included in the TSI-related SEI message, the luminance gain factor numerator information included in the TEI-related SEI message, or the luminance gain factor denominator information. For example, the luminance gain factor numerator information and the luminance gain factor denominator information may be obtained based on the luminance gain factor existence information. That is, the decoding device may obtain the luminance gain factor numerator information and the luminance gain factor denominator information based on the luminance gain factor existence information.

[0300] Here, the luminance gain coefficient existence information included in the TSI-related SEI message may indicate whether the syntactic elements of the luminance gain coefficient numerator information and the luminance gain coefficient denominator information exist in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, the luminance gain coefficient existence information with a value equal to 1 may indicate that the syntactic elements of the luminance gain coefficient numerator information and the luminance gain coefficient denominator information exist in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, the luminance gain coefficient existence information with a value equal to 0 may indicate that the syntactic elements of the luminance gain coefficient numerator information and the luminance gain coefficient denominator information do not exist in the TEI-related SEI message associated with the current TSI-related SEI message. However, this is not limited thereto, and what the luminance gain coefficient existence information with a value equal to 1 indicates may be interchangeable with what the luminance gain coefficient existence information with a value equal to 0 indicates.

[0301] The luminance gain factor present information may take various forms and may be represented by various names. For example, the luminance gain factor present information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the luminance gain factor present information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the luminance gain factor present information may include tsi_luminance_gain_factor_present_flag or tsi_luminance_gain_factor_present_idc, but is not limited thereto.

[0302] For example, if the luminance gain factor existence information is represented by tsi_luminance_gain_factor_present_flag, tsi_luminance_gain_factor_present_flag equal to 1 may indicate that the syntax elements tsi_luminance_gain_factor_num_minus1 and tsi_luminance_gain_factor_denom_minus1 described below exist in the TEI-related SEI message associated with the current TSI-related SEI message. tsi_luminance_gain_factor_present_flag equal to 0 may indicate that the syntax elements tei_luminance_gain_factor_num_minus1 and tei_luminance_gain_factor_denom_minus1 described below do not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0303] In addition, the luminance gain factor numerator information included in the SEI message related to TEI may represent the numerator of the luminance gain factor for the thumbnail picture. For example, the value of the luminance gain factor numerator information must be within the range of 0 to 65,535 (inclusive).

[0304] Luminance gain factor molecular information may take various forms and may be represented by various names. For example, luminance gain factor molecular information may be a syntax element or a syntax structure containing one or more syntax elements. For example, luminance gain factor molecular information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, luminance gain factor molecular information may include tei_luminance_gain_factor_num_minus1, etc., but is not limited thereto.

[0305] For example, if the luminance gain factor numerator information is expressed as tei_luminance_gain_factor_num_minus1, the value obtained by adding 1 to tei_luminance_gain_factor_num_minus1 may represent the numerator of the luminance gain factor for the thumbnail picture. tei_luminance_gain_factor_num_minus1 must be within the range of 0 to 65,535 (inclusive).

[0306] The luminance gain factor denominator information included in the SEI message related to TEI may represent the denominator of the luminance gain factor for the thumbnail picture. For example, the value of the luminance gain factor denominator information must be within the range of 0 to 65,535 (inclusive).

[0307] The luminance gain factor denominator information may take various forms and be expressed by various names. For example, the luminance gain factor denominator information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the luminance gain factor denominator information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the luminance gain factor denominator information may include tei_luminance_gain_factor_denom_minus1, but is not limited thereto.

[0308] For example, if the luminance gain factor denominator information is expressed as tei_luminance_gain_factor_denom_minus1, the value obtained by adding 1 to tei_luminance_gain_factor_denom_minus1 may represent the denominator of the luminance gain factor for the thumbnail picture. tei_luminance_gain_factor_denom_minus1 must be within the range of 0 to 65,535 (inclusive).

[0309] For example, the lumina sample value of any lumina sample in the current picture can be denoted as curPicYVal, and the corresponding lumina sample value in the thumbnail picture can be denoted as thumbPicYVal. In this case, the lumina sample value of the thumbnail picture can be derived by correcting it from the lumina sample value of the current picture based on lumina gain factor information (e.g., lumina gain factor numerator information (tei_luminance_gain_factor_num_minus1) and lumina gain factor denominator information (tei_luminance_gain_factor_denom_minus1)).

[0310] For example, the luma sample value thumbPicYVal can be derived as follows.

[0311] thumbYVal = Max( ( 1 << BitDepth Y ) - 1,

[0312] Round( curPicYVal * ( tei_luminance_gain_factor_num_minus1 + 1 ) χ ( tei_luminance_gain_factor_denom_minus1 + 1 ) ) )

[0313] Here, BitDepthY may represent the bit depth for a sample of the luminance component, and Round() may represent an operation of rounding the result of the operation according to a predetermined rounding rule. Additionally, Max() may represent an operation of selecting the larger of two values. Accordingly, the decoding device may apply a luminance gain to express the brightness of the thumbnail picture by deriving the luminance sample value of the thumbnail picture based on the above formula. However, the above formula and operation rule are exemplary, and various modifications may be possible within the scope of achieving the same technical concept and effect.

[0314] Additionally, based on duration existence information included in the TSI-related SEI message, at least one of time scale information, duration unit information, blank picture duration existence information, or blank picture duration unit information may be obtained. That is, the decoding device may obtain at least one of time scale information, duration unit information, blank picture duration existence information, or blank picture duration unit information based on the duration existence information.

[0315] For example, if the duration existence information indicates that duration-related information is included within a thumbnail-related SEI message, the decoding device may obtain time scale information from a TSI-related SEI message, or obtain at least one of duration unit information, blank picture duration existence information, or blank picture duration unit information from a TEI-related SEI message associated with a TSI-related SEI message. On the other hand, if the duration existence information indicates that duration-related information is not included within a thumbnail-related SEI message, the decoding device may not obtain time scale information, duration unit information, blank picture duration existence information, and / or blank picture duration unit information.

[0316] Duration existence information may indicate whether a syntax element of time scale information exists in the SEI message, whether a syntax element of duration unit information exists in the TEI-related SEI message associated with the current TSI-related SEI message, and whether blank picture duration existence information and the syntax element of the blank picture duration unit information exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0317] For example, duration existence information with a value equal to 1 may indicate that a syntax element of time scale information exists in the SEI message, a syntax element of duration unit information exists in the TEI-related SEI message associated with the current TSI-related SEI message, and that blank picture duration existence information and a syntax element of blank picture duration unit information exist in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, duration existence information with a value equal to 0 may indicate that a syntax element of time scale information does not exist in the SEI message, a syntax element of duration unit information does not exist in the TEI-related SEI message associated with the current TSI-related SEI message, and that a syntax element of blank picture duration existence information and a syntax element of blank picture duration unit information does not exist in the TEI-related SEI message associated with the current TSI-related SEI message. However, this is not limited thereto, and what duration existence information with a value equal to 1 represents may be interchangeable with what duration existence information with a value equal to 0 represents.

[0318] Timing information may take various forms and may be represented by various names. For example, the timing information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the timing information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the timing information may include tsi_timing_info_present, tsi_timing_info_present_flag, or tsi_timing_info_present_idc, but is not limited thereto.

[0319] For example, if duration existence information is represented by tsi_timing_info_present_flag, tsi_timing_info_present_flag identical to 1 may indicate that the syntax element tsi_time_scale exists in the SEI message, and may indicate that the syntax element tei_num_units_in_dur exists in the TEI-related SEI message associated with the current TSI-related SEI message, and may also indicate that the syntax elements tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] described below exist in the TEI-related SEI message associated with the current TSI-related SEI message. tsi_timing_info_present_flag equal to 0 may indicate that the syntax element tsi_time_scale does not exist in the SEI message, and that the syntax elements tei_num_units_in_dur, tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] described below do not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0320] For a detailed explanation of the above time scale information and the above duration unit information, refer to the foregoing.

[0321] Blank picture duration existence information may indicate whether a syntax element of blank picture duration unit information exists. For example, blank picture duration existence information with a value equal to 1 may indicate that a syntax element of blank picture duration unit information exists within a thumbnail-related SEI message. Additionally, blank picture duration existence information with a value equal to 0 may indicate that a syntax element of blank picture duration unit information does not exist within a thumbnail-related SEI message. However, this is not limited thereto, and what blank picture duration existence information with a value equal to 1 indicates may be interchangeable with what blank picture duration existence information with a value equal to 0 indicates.

[0322] The blank picture duration present information may take various forms and may be represented by various names. For example, the blank picture duration present information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the blank picture duration present information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the blank picture duration present information may include tei_num_units_in_blank_pic_dur_present_flag[ i ] or tei_num_units_in_blank_pic_dur_present_idc[ i ], but is not limited thereto.

[0323] For example, if the blank picture duration information is represented by tei_num_units_in_blank_pic_dur_present_flag[ i ], tei_num_units_in_blank_pic_dur_present_flag[ i ], which is equal to 1, may indicate that the syntax element tei_num_units_in_blank_pic_dur[ i ] described below exists. tei_num_units_in_blank_pic_dur_present_flag[ i ], which is equal to 0, may indicate that the syntax element tei_num_units_in_blank_pic_dur[ i ] described below does not exist.

[0324] Blank picture duration unit information can be obtained based on the blank picture duration existence information. That is, the decoding device can obtain blank picture duration unit information based on the blank picture duration existence information. For example, if the blank picture duration existence information included in the TEI-related SEI message indicates that the blank picture duration unit information is included in the TEI-related SEI message, the decoding device can obtain the blank picture duration unit information from the TEI-related SEI message. On the other hand, if the blank picture duration existence information indicates that the blank picture duration unit information is not included in the TEI-related SEI message, the decoding device may not obtain the blank picture duration unit information. Accordingly, by selectively obtaining the blank picture duration unit information based on the blank picture duration existence information, the decoding device can obtain information for determining the display duration of a specific blank picture.

[0325] Here, the blank picture duration unit information may represent the number of time units corresponding to the display duration of the blank picture among the time units of a clock operating with frequency time scale information (Hz). For example, the blank picture duration unit information may represent the number of time units corresponding to the display duration of the blank picture among the time units based on the above time scale information.

[0326] Blank picture duration unit information may take various forms and may be represented by various names. For example, blank picture duration unit information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture duration unit information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, blank picture duration unit information may include tei_num_units_in_blank_pic_dur[ i ], but is not limited thereto.

[0327] For example, if the blank picture duration unit information is represented as tei_num_units_in_blank_pic_dur[ i ], tei_num_units_in_blank_pic_dur[ i ] may represent the number of time units corresponding to the display duration of the i-th blank picture among the time units of a clock operating at frequency tsi_time_scale (Hz). tei_num_units_in_blank_pic_dur[ i ] must be greater than 0. If the display duration of the i-th blank picture is expressed in seconds, it may be equal to the quotient obtained by dividing tei_num_units_in_blank_pic_dur[ i ] by tsi_time_scale. For example, if the display duration of the i-th blank picture is 0.04 seconds, tsi_time_scale and tei_num_units_in_dur can be 27,000,000 and 1,080,000, respectively.

[0328] Meanwhile, SEI messages related to TEI may further include blank picture existence information, blank picture count information, and blank picture sample information.

[0329] Blank picture existence information may indicate whether syntax elements of blank picture count information and blank picture sample information exist in the SEI message. For example, blank picture existence information with a value equal to 1 may indicate that syntax elements of blank picture count information and blank picture sample information exist in the thumbnail-related SEI message. Additionally, blank picture existence information with a value equal to 0 may indicate that syntax elements of blank picture count information and blank picture sample information do not exist in the thumbnail-related SEI message. However, this is not limited thereto, and what blank picture existence information with a value equal to 1 indicates may be interchangeable with what blank picture existence information with a value equal to 0 indicates.

[0330] Blank picture presence information may take various forms and may be expressed by various names. For example, blank picture presence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture presence information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, blank picture presence information may include tei_blank_pic_present_flag or tei_blank_pic_present_idc, but is not limited thereto.

[0331] For example, if blank picture presence information is represented by tei_blank_pic_present_flag, tei_blank_pic_present_flag identical to 1 may indicate that syntax elements tei_num_blank_pics_minus1 and tei_blank_pic_sample[ i ][ c ] exist in the SEI message, and also that syntax elements tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] exist in the SEI message. A tei_blank_pic_present_flag equal to 0 may indicate that the syntax elements tei_num_blank_pics_minus1, tei_blank_pic_sample[ i ][ c ], tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] do not exist in the SEI message.

[0332] Information on the number of blank pictures and information on blank picture samples can be obtained based on information on the existence of blank pictures. That is, the decoding device can obtain information on the number of blank pictures and information on blank pictures based on information on the existence of blank pictures included in a thumbnail-related SEI message (e.g., a TEI-related SEI message). For example, if the information on the existence of blank pictures indicates that a syntax element related to blank pictures exists within the thumbnail-related SEI message, the decoding device can obtain information on the number of blank pictures and information on blank pictures from the TEI-related SEI message. On the other hand, if the information on the existence of blank pictures indicates that a syntax element related to blank pictures does not exist within the nail-related SEI message, the decoding device may not obtain information on the number of blank pictures and / or information on blank pictures. Accordingly, the decoding device can identify the number of blank pictures following the current picture in the output sequence and the sample values ​​for the color components of each blank picture by selectively acquiring blank picture count information and blank picture sample information based on blank picture existence information.

[0333] The blank picture count information can indicate the number of blank pictures following the current picture in the output order.

[0334] Blank picture count information may take various forms and may be represented by various names. For example, blank picture count information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture count information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, blank picture count information may include tei_num_blank_pics_minus1, but is not limited thereto.

[0335] For example, if the number of blank pictures is represented as tei_num_blank_pics_minus1, the value obtained by adding 1 to tei_num_blank_pics_minus1 can represent the number of blank pictures following the current picture in the output order.

[0336] Blank picture sample information can represent sample information regarding the color components of the blank picture following the current picture in the output sequence.

[0337] Blank picture sample information may take various forms and may be represented by various names. For example, blank picture sample information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture sample information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, blank picture sample information may include tei_blank_pic_sample[ c ], but is not limited thereto.

[0338] For example, if blank picture sample information is represented as tei_blank_pic_sample[ c ], tei_blank_pic_sample[ c ] may represent the sample value for the c-th color component of the i-th blank picture following the current picture in the output order. The length of the above syntax element must be BitDepthY bits.

[0339] Meanwhile, the SEI message related to TEI may include at least one of TEI identifier information for identifying a thumbnail picture within a bitstream, thumbnail cancellation information, thumbnail persistence information, thumbnail size information, and thumbnail position information.

[0340] TEI identifier information for representing a thumbnail picture within a TEI-related SEI message may include an identification number that can be used to uniquely identify a thumbnail image or video within a bitstream.

[0341] TEI identifier information for representing a thumbnail picture within a TEI-related SEI message may take various forms and be expressed by various names. For example, TEI identifier information for representing a thumbnail picture may be a syntax element or a syntax structure containing one or more syntax elements. For example, TEI identifier information for representing a thumbnail picture may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, TEI identifier information for representing a thumbnail picture may include tei_thumbnail_id, but is not limited thereto.

[0342] Thumbnail cancellation information may indicate whether to cancel the persistence of a previous thumbnail-related SEI message in the output order applied to the current layer. For example, thumbnail cancellation information may indicate whether to cancel the persistence of a previous TEI-related SEI message in the output order applied to the current layer.

[0343] For example, thumbnail cancellation information with a value equal to 1 may indicate that the TEI-related SEI message cancels the persistence of a previous TEI SEI message in the output order applied to the current layer. Additionally, thumbnail cancellation information with a value equal to 0 may indicate that thumbnail extraction information follows. However, this is not limited to the above, and what thumbnail cancellation information with a value equal to 1 indicates may be interchangeable with what thumbnail cancellation information with a value equal to 0 indicates.

[0344] Thumbnail cancellation information may take various forms and may be represented by various names. For example, thumbnail cancellation information may be a syntax element or a syntax structure containing one or more syntax elements. For example, thumbnail cancellation information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, thumbnail cancellation information may include tei_cancel_flag or tei_cancel_idc, but is not limited thereto.

[0345] Thumbnail persistence information can indicate the extent to which thumbnail extraction information provided by the TEI-related SEI message is validly applied, that is, the persistence of the thumbnail extraction information. For example, thumbnail persistence information can indicate whether the thumbnail extraction information according to the TEI-related SEI message is applied only to the current picture, or whether it is applied to the current picture and at least one subsequent picture in the output order.

[0346] For example, thumbnail persistence information with a value equal to 1 may indicate that thumbnail extraction information is applied to the current picture and all subsequent pictures of the current layer in the output order, provided that this application continues until at least one of the following conditions is satisfied. For example, the continued application of the thumbnail extraction information may be terminated if at least one of the following occurs: when a new CLVS of the current layer starts, when a bitstream ends, or when a picture of the current layer associated with a TEI-related SEI message located after the current picture in the output order is output. Additionally, thumbnail persistence information with a value equal to 0 may indicate that the thumbnail extraction information is applied only to the current picture. However, this is not limited thereto, and what is indicated by thumbnail persistence information with a value equal to 1 may be interchangeable with what is indicated by thumbnail persistence information with a value equal to 0.

[0347] Thumbnail persistence information may take various forms and may be represented by various names. For example, thumbnail persistence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, thumbnail persistence information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, thumbnail persistence information may include tei_persistence_flag or tei_persistence_idc, but is not limited thereto.

[0348] Thumbnail size information may include width information and height information of the thumbnail picture.

[0349] Width information of a thumbnail picture can represent the width of the area in units of luma samples if the corresponding syntax element exists.

[0350] Width information of a thumbnail picture may take various forms and may be represented by various names. For example, width information of a thumbnail picture may be a syntax element or a syntax structure containing one or more syntax elements. For example, width information of a thumbnail picture may include a flag consisting of one bit, an indicator consisting of two or more bits or variable-length bits, or an unsigned integer consisting of variable-length bits. For example, width information of a thumbnail picture may include tei_width_minus1, etc., but is not limited thereto.

[0351] For example, if the width information of a thumbnail picture is represented as tei_width_minus1, and tei_width_minus1 exists, the value obtained by adding 1 to tei_width_minus1 can represent the width of the area in units of luma samples.

[0352] Height information of a thumbnail picture can represent the height of the area in units of luma samples, if the corresponding syntax element exists.

[0353] Height information of a thumbnail picture may take various forms and may be represented by various names. For example, height information of a thumbnail picture may be a syntax element or a syntax structure containing one or more syntax elements. For example, height information of a thumbnail picture may include a flag consisting of one bit, an indicator consisting of two or more bits or variable-length bits, or an unsigned integer consisting of variable-length bits. For example, height information of a thumbnail picture may include tei_height_minus1, etc., but is not limited thereto.

[0354] For example, if the height information of a thumbnail picture is expressed as tei_height_minus1, and tei_height_minus1 exists, the value obtained by adding 1 to tei_height_minus1 can represent the height of the area in units of luma samples.

[0355] The thumbnail position information may include top-left horizontal position information and top-left vertical position information of the thumbnail picture.

[0356] The top-left horizontal position information of the thumbnail picture can represent the horizontal position of the top-left sample of the thumbnail picture area, that is, the x-coordinate of the thumbnail picture area.

[0357] The top-left horizontal position information of a thumbnail picture may take various forms and may be represented by various names. For example, the top-left horizontal position information of a thumbnail picture may be a syntax element or a syntax structure containing one or more syntax elements. For example, the top-left horizontal position information of a thumbnail picture may include a flag consisting of one bit, an indicator consisting of two or more bits or variable-length bits, or an unsigned integer consisting of variable-length bits. For example, the top-left horizontal position information of a thumbnail picture may include tei_top_left_x, etc., but is not limited thereto.

[0358] For example, if the top-left horizontal position information of a thumbnail picture is represented as tei_top_left_x, and tei_top_left_x exists, tei_top_left_x may represent the horizontal position of the top-left sample of the area.

[0359] The top-left vertical position information of the thumbnail picture can represent the vertical position of the top-left sample of the thumbnail picture area, that is, the y-coordinate of the thumbnail picture area.

[0360] The top-left vertical position information of a thumbnail picture may take various forms and be expressed by various names. For example, the top-left vertical position information of a thumbnail picture may be a syntax element or a syntax structure containing one or more syntax elements. For example, the top-left vertical position information of a thumbnail picture may include a flag consisting of one bit, an indicator consisting of two or more bits or variable-length bits, or an unsigned integer consisting of variable-length bits. For example, the top-left vertical position information of a thumbnail picture may include tei_top_left_y, etc., but is not limited thereto.

[0361] For example, if the top-left vertical position information of a thumbnail picture is represented as tei_top_left_y, and tei_top_left_y exists, tei_top_left_y can represent the vertical position of the top-left sample of the area.

[0362] A SEI message related to TEI may include information for signaling which picture(s) correspond to a thumbnail image or video for video preview, or which area within said picture(s) is used as a thumbnail.

[0363] For example, based on the fact that a TEI-related SEI message includes thumbnail size information and thumbnail position information, the thumbnail picture can be identified as an area within the entire picture that is set based on the thumbnail size information and thumbnail position information. That is, when a TEI-related SEI message includes thumbnail size information and thumbnail position information, the decoding device can set a specific area within the entire picture based on the thumbnail size information (e.g., width information and height information of the thumbnail picture) and the thumbnail position information (e.g., top-left horizontal position information and top-left vertical position information of the thumbnail picture), and can identify the set area as a thumbnail image or video. At this time, the width information and height information can specify the size of the area to be used as a thumbnail within the entire picture, and the top-left horizontal position information and top-left vertical position information can specify the position of the top-left sample of the area. Accordingly, the area(s) indicated by the width information, height information, top-left horizontal position information and top-left vertical position information can be used as a thumbnail image or video for the associated video content if the corresponding syntax element exists.

[0364] Additionally, based on the fact that the TEI-related SEI message does not include thumbnail size information and thumbnail position information, the thumbnail picture can be identified as the entire picture. That is, if the TEI-related SEI message does not include thumbnail size information and thumbnail position information, the decoding device determines that a specific area to be used as a thumbnail is not separately signaled, and can identify the entire picture(s) to which the TEI-related SEI message applies as a thumbnail image or video. For example, if width information, height information, top-left horizontal position information, and top-left vertical position information of the thumbnail picture do not exist, the picture(s) to which the TEI-related SEI message applies can constitute a thumbnail image or video as itself.

[0365] According to the present invention, a decoding device can identify and provide a thumbnail picture without separate thumbnail generation or complex extraction and / or editing operations by using information signaled by thumbnail-related SEI messages (e.g., TSI-related SEI messages and TEI-related SEI messages) within a bitstream. In particular, the decoding device can configure the thumbnail on a segment basis and determine the output order based on segment output order information, even when the thumbnail is composed of multiple segments, and can control the display time of the picture within the thumbnail based on display duration information. Additionally, the decoding device can correct and display the brightness of the thumbnail picture based on luminance gain coefficient information, and can control the playback or display operation of the thumbnail picture by applying a blank picture (and its display duration) following the current picture using blank picture information and blank picture duration information. Accordingly, the decoding device can reduce the amount of computation and power consumption during the decoding process, facilitate synchronization between thumbnails and original content, and improve the compatibility of thumbnail processing between different devices and systems.

[0366] The names of syntax elements and semantics described in this specification are merely exemplary names for the convenience of explanation, and the scope of the rights of this disclosure is not limited by such names. That is, any configuration that performs the same or substantially corresponding functions as the syntax elements and semantics described in this specification may be included within the spirit and scope of this disclosure, even if its name, notation, or terminology differs.

[0367] FIG. 6 is a diagram illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0368] The encoding method (S600) may include the operations described below.

[0369] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms or names described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0370] The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the decoding method according to one embodiment, and the previously described operations may be added.

[0371] The sequence of actions illustrated in the drawings regarding the actions to be described below is merely an example, and the actions to be described below may be performed in a different order as long as it does not contradict the causal relationship of the actions to be described.

[0372] The operations described below form a single embodiment integrally with the configurations and / or operations described above, unless they conflict with the configurations and / or operations described above, and do not form a separate embodiment distinct from the configurations and / or operations described above.

[0373] The encoding method (S600) can be executed by an encoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0374] The encoding device can generate a SEI (supplemental enhancement information) message (S610).

[0375] SEI messages may convey specific types of information that assist in processes related to the decoding, display, or other purposes of image information. Here, SEI messages may not be necessary for the decoding process to determine the sample values ​​of the decoded picture.

[0376] For example, SEI messages may include thumbnail-related SEI messages.

[0377] A thumbnail-related SEI message may include at least one of a TSI (Thumbnail Source Information) related SEI message associated with the source of the thumbnail picture and / or a TEI (Thumbnail Extraction Information) related SEI message associated with the extraction of the thumbnail picture. For example, a TSI-related SEI message may include source information of the thumbnail picture, and a TEI-related SEI message may include extraction information of the thumbnail picture.

[0378] TSI-related SEI messages may have various names such as thumbnail source information SEI message, thumbnail source information related message, thumbnail source information related information, thumbnail source message, thumbnail source information, thumbnail source information SEI message, thumbnail source information related message, thumbnail source information related information, thumbnail source message, thumbnail source information, TSI SEI message, TSI-related message, TSI-related information, TSI message, TSI information, etc., and such names are not limited.

[0379] TSI-related SEI messages can take various forms. For example, a TSI-related SEI message may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, a TSI-related SEI message may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. For example, a TSI-related SEI message may be represented as thumbnail_source_info( payloadSize ), but is not limited thereto.

[0380] TEI-related SEI messages may have various names such as thumbnail extraction information SEI message, thumbnail extraction information related message, thumbnail extraction information related information, thumbnail extraction message, thumbnail extraction information, TEI SEI message, TEI-related message, TEI-related information, TEI message, TEI information, etc., and the names are not limited.

[0381] TEI-related SEI messages can take various forms. For example, a TEI-related SEI message may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, a TEI-related SEI message may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. For example, a TEI-related SEI message may be represented as thumbnail_extraction_info( payloadSize ), but is not limited thereto.

[0382] For example, the encoding device may generate a thumbnail-related SEI (Supplemental Enhancement Information) message containing thumbnail-related information associated with a picture. For example, the processor of the encoding device may generate a thumbnail-related SEI message associated with a picture and include it in a bitstream. The processor of the encoding device may generate thumbnail-related information to provide a thumbnail for at least one picture of the original video, and may generate a thumbnail-related SEI message to transmit said thumbnail-related information.

[0383] The encoding device may include at least one of thumbnail identifier information, thumbnail source information, thumbnail extraction area information, thumbnail size information, thumbnail position information, thumbnail cancellation information, thumbnail persistence information, segment output order information of the thumbnail picture, display duration information, luminance gain factor information and / or blank picture information in the thumbnail-related SEI message, and include this in the bitstream.

[0384] The TSI-related SEI message and / or the TEI-related SEI message may be identical to the TSI-related SEI message and / or the TEI-related SEI message described in FIG. 5, respectively. The description of the TSI-related SEI message and / or the TEI-related SEI message may be replaced with the description of the TSI-related SEI message and / or the TEI-related SEI message described in FIG. 5.

[0385] Meanwhile, thumbnail-related information may include at least one of segment output order information of the thumbnail picture, display duration information, luminance gain factor information, or blank picture information.

[0386] For example, segment output order information may be determined based on the output order of the segment to which the current picture belongs within the thumbnail picture. Here, the segment output order information of the thumbnail picture may include output order existence information included in the TSI-related SEI message and output order information included in the TEI-related SEI message.

[0387] Output order existence information may indicate whether the TEI-related SEI message associated with the current TSI-related SEI message contains syntax elements of output order information that indicate the segment output order. Additionally, output order existence information with a value equal to 1 may indicate that the TEI-related SEI message associated with the current TSI-related SEI message contains syntax elements of output order information. Additionally, output order existence information with a value equal to 0 may indicate that the TEI-related SEI message associated with the current TSI-related SEI message does not contain syntax elements of output order information. However, this is not limited thereto, and what output order existence information with a value equal to 1 indicates may be interchangeable with what output order existence information with a value equal to 0 indicates.

[0388] Output order present information may take various forms and may be represented by various names. For example, output order present information may be a syntax element or a syntax structure containing one or more syntax elements. For example, output order present information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, output order present information may include tsi_segment_output_order_present_flag or tsi_segment_output_order_present_idc, but is not limited thereto.

[0389] For example, if output order existence information is represented by tsi_segment_output_order_present_flag, tsi_segment_output_order_present_flag equal to 1 may indicate that the syntax element tei_segment_output_order described below exists in the TEI-related SEI message associated with the current TSI-related SEI message. tsi_segment_output_order_present_flag equal to 0 may indicate that the syntax element tei_segment_output_order described below does not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0390] The encoding device may generate output order existence information based on whether output order information is included in the TEI-related SEI message and include it in the TSI-related SEI message. That is, if the TEI-related SEI message associated with the current TSI-related SEI message includes output order information, the encoding device may set the output order existence information to indicate that output order information is included within the TEI-related SEI message and include it in the TSI-related SEI message. Conversely, if the TEI-related SEI message does not include output order information, the encoding device may set the output order existence information to indicate that output order information is not included within the TEI-related SEI message and include it in the TSI-related SEI message.

[0391] Output order information may indicate the output order of the current segment to which the current picture belongs. Here, a segment may include one or more consecutive pictures in the output order, and a thumbnail may include two or more segments.

[0392] Output order information may take various forms and may be represented by various names. For example, output order information may be a syntax element or a syntax structure containing one or more syntax elements. For example, output order information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, output order information may include tei_segment_output_order, but is not limited thereto.

[0393] For example, when output order information is represented as tei_segment_output_order, tei_segment_output_order may indicate the output order of the current segment to which the current picture belongs. Here, a segment may include one or more consecutive pictures in the output order, and a thumbnail may include two or more segments. Additionally, the value of tei_segment_output_order must be within the range of 0 to 255 (inclusive).

[0394] Pictures to which a TEI-related SEI message containing the same output order information is applied may be included in the same segment. That is, when a thumbnail contains multiple segments, the encoding device can group multiple pictures that are set to have the same output order information value into one segment by setting the value of the output order information included in the TEI-related SEI message for each picture.

[0395] For example, when output order information is represented as tei_segment_output_order, a segment consisting of pictures with a tei_segment_output_order value set to be equal to segOutOrderA can be called segA, and a segment consisting of pictures with a tei_segment_output_order value set to be equal to segOutOrderB can be called segB. In this case, if segOutOrderB is set to be greater than segOutOrderA, an output order between segments can be assigned such that segB follows segA in the output order within the thumbnail. Additionally, the output order of pictures set to be included within the same segment can be configured to be the same as the output order within the current bitstream. Accordingly, by defining segments based on the values ​​of output order information and assigning an output order between segments, the encoding device can construct and signal the thumbnail on a segment basis even when the thumbnail contains multiple segments.

[0396] Additionally, if the output order information is not included in the TEI-related SEI message, the encoding device may be configured so that the decoding device can infer and use the value of the output order information. For example, if the output order information is not included in the TEI-related SEI message, the value of the output order information may be inferred as 0 for the first TEI-related SEI message in the bitstream based on the output order.

[0397] Meanwhile, if output order information is not included in the TEI-related SEI message, the decoding device may infer and use the value of the output order information, and the encoding device may configure the bitstream so that the decoding device can perform such inference. For example, based on the fact that the TEI-related SEI message is not included in the output order information and that the TEI-related SEI message is not the first TEI-related SEI message in the bitstream based on the output order, the value of the output order information may be inferred as the value of the output order information corresponding to the previous TEI-related SEI message based on the output order plus 1.

[0398] However, the inference method and application conditions of the output order information described above may be merely exemplary explanations to aid understanding, and the scope of the present invention is not limited thereto. For example, the inference value when the output order information is not included in the TEI-related SEI message, the criteria used for inference (e.g., the "first" judgment method of the output order criteria), and the value added to the value of the preceding TEI-related SEI message (e.g., 1) may vary depending on the implementation or design, and various modifications may be possible within the scope of achieving the same technical concept and effect.

[0399] Meanwhile, the display duration information may include at least one of the duration existence information, time scale information, or duration unit information included in the TEI-related SEI message.

[0400] Duration existence information included in a TSI-related SEI message may indicate whether a syntax element of time scale information exists in the SEI message and whether a syntax element of duration unit information exists in a TEI-related SEI message associated with the current TSI-related SEI message. Additionally, duration existence information with a value equal to 1 may indicate that a syntax element of time scale information exists in the SEI message and that a syntax element of duration unit information exists in a TEI-related SEI message associated with the current TSI-related SEI message. Additionally, duration existence information with a value equal to 0 may indicate that a syntax element of time scale information does not exist in the SEI message and that a syntax element of duration unit information does not exist in a TEI-related SEI message associated with the current TSI-related SEI message. However, this is not limited thereto, and what duration existence information with a value equal to 1 indicates may be interchangeable with what duration existence information with a value equal to 0 indicates.

[0401] Timing information may take various forms and may be represented by various names. For example, the timing information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the timing information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the timing information may include tsi_timing_info_present, tsi_timing_info_present_flag, or tsi_timing_info_present_idc, but is not limited thereto.

[0402] For example, if duration existence information is represented by tsi_timing_info_present_flag, tsi_timing_info_present_flag equal to 1 may indicate that the syntax element tsi_time_scale exists in the SEI message and that the syntax element tei_num_units_in_dur exists in the TEI-related SEI message associated with the current TSI-related SEI message. tsi_timing_info_present_flag equal to 0 may indicate that the syntax element tsi_time_scale does not exist in the SEI message and that the syntax element tei_num_units_in_dur does not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0403] For example, duration existence information may be generated based on time scale information and duration unit information. That is, if time scale information is included in a TSI-related SEI message and duration unit information is included in a TEI-related SEI message associated with the current TSI-related SEI message, the encoding device may generate duration existence information to indicate that duration-related information is included in the thumbnail-related SEI message and include it in the TSI-related SEI message. On the other hand, if at least one of the time scale information and / or the duration unit information is not included in the thumbnail-related SEI message, the encoding device may generate duration existence information to indicate that duration-related information is not included in the thumbnail-related SEI message and include it in the TSI-related SEI message.

[0404] Accordingly, the encoding device signals whether time scale information and duration unit information are included through the duration existence information, thereby enabling the decoding device to selectively acquire time scale information and / or duration unit information based on the duration existence information and determine the display duration of the current picture in the thumbnail.

[0405] In addition, time scale information can indicate the number of time units that elapse in one second. For example, a time coordinate system that measures time using a 27 MHz clock has a time scale information value of 27,000,000. The value of the time scale information must be greater than 0.

[0406] Time scale information may take various forms and may be represented by various names. For example, time scale information may be a syntax element or a syntax structure containing one or more syntax elements. For example, time scale information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, time scale information may include tsi_time_scale, etc., but is not limited thereto.

[0407] The duration unit information may represent the number of time units corresponding to the display duration of the current picture among the time units of a clock operating with frequency time scale information (Hz). The value of the duration unit information must be greater than 0.

[0408] Duration unit information may take various forms and may be represented by various names. For example, duration unit information may be a syntax element or a syntax structure containing one or more syntax elements. For example, duration unit information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, duration unit information may include tei_num_units_in_dur, but is not limited thereto.

[0409] For example, if the duration unit information is expressed as tei_num_units_in_dur, tei_num_units_in_dur may be the number of time units corresponding to the display duration of the current picture among the time units of a clock operating at frequency tsi_time_scale (Hz).

[0410] Additionally, the display duration of the current picture may be determined based on time scale information and duration unit information. For example, the display duration of the current picture may be determined based on dividing the duration unit information by the time scale information. That is, if the display duration of the current picture is expressed in seconds, it may be equal to the quotient obtained by dividing the duration unit information by the scale information. For example, if the display duration of the current picture is 0.04 seconds, the values ​​of the time scale information and the duration unit information may be 27,000,000 and 1,080,000, respectively. However, the above numerical values ​​and calculation methods are exemplary and are not limited thereto, and the settings of the time scale information and duration unit information may vary depending on the implementation.

[0411] Meanwhile, the luminance gain factor information may include at least one of the luminance gain factor existence information included in the TSI-related SEI message, the luminance gain factor numerator information included in the TEI-related SEI message, or the luminance gain factor denominator information. Here, the luminance gain factor existence information included in the TSI-related SEI message may indicate whether the syntax elements of the luminance gain factor numerator information and the luminance gain factor denominator information exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0412] The above information regarding the existence of the luminance gain coefficient can be generated based on the numerator information and denominator information of the luminance gain coefficient. That is, the encoding device can generate information regarding the existence of the luminance gain coefficient based on the numerator information and denominator information of the luminance gain coefficient. For example, if the TEI-related SEI message contains syntax elements of the numerator information and denominator information of the luminance gain coefficient, the encoding device can set the information regarding the existence of the luminance gain coefficient to indicate that the syntax elements exist within the TEI-related SEI message and include it in the TSI-related SEI message. Conversely, if the TEI-related SEI message does not contain the syntax elements, the encoding device can set the information regarding the existence of the luminance gain coefficient to indicate that the syntax elements do not exist within the TEI-related SEI message and include it in the TSI-related SEI message.

[0413] Here, the luminance gain coefficient existence information included in the TSI-related SEI message may indicate whether the syntactic elements of the luminance gain coefficient numerator information and the luminance gain coefficient denominator information exist in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, the luminance gain coefficient existence information with a value equal to 1 may indicate that the syntactic elements of the luminance gain coefficient numerator information and the luminance gain coefficient denominator information exist in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, the luminance gain coefficient existence information with a value equal to 0 may indicate that the syntactic elements of the luminance gain coefficient numerator information and the luminance gain coefficient denominator information do not exist in the TEI-related SEI message associated with the current TSI-related SEI message. However, this is not limited thereto, and what the luminance gain coefficient existence information with a value equal to 1 indicates may be interchangeable with what the luminance gain coefficient existence information with a value equal to 0 indicates.

[0414] The luminance gain factor present information may take various forms and may be represented by various names. For example, the luminance gain factor present information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the luminance gain factor present information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the luminance gain factor present information may include tsi_luminance_gain_factor_present_flag or tsi_luminance_gain_factor_present_idc, but is not limited thereto.

[0415] For example, if the luminance gain factor existence information is represented by tsi_luminance_gain_factor_present_flag, tsi_luminance_gain_factor_present_flag equal to 1 may indicate that the syntax elements tsi_luminance_gain_factor_num_minus1 and tsi_luminance_gain_factor_denom_minus1 described below exist in the TEI-related SEI message associated with the current TSI-related SEI message. tsi_luminance_gain_factor_present_flag equal to 0 may indicate that the syntax elements tei_luminance_gain_factor_num_minus1 and tei_luminance_gain_factor_denom_minus1 described below do not exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0416] For example, information on the existence of a luminance gain factor can be obtained based on thumbnail source information. That is, the decoding device can obtain information on the existence of a luminance gain factor based on thumbnail source information. For example, if the thumbnail source information indicates that the thumbnail image or video is sourced from one or more pictures within the current layer, or indicates that the thumbnail image or video is sourced from a different layer within the current bitstream that is different from the current layer, the decoding device can obtain information on the existence of a luminance gain factor.

[0417] In addition, the luminance gain factor numerator information included in the SEI message related to TEI may represent the numerator of the luminance gain factor for the thumbnail picture. For example, the value of the luminance gain factor numerator information must be within the range of 0 to 65,535 (inclusive).

[0418] Luminance gain factor molecular information may take various forms and may be represented by various names. For example, luminance gain factor molecular information may be a syntax element or a syntax structure containing one or more syntax elements. For example, luminance gain factor molecular information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, luminance gain factor molecular information may include tei_luminance_gain_factor_num_minus1, etc., but is not limited thereto.

[0419] For example, if the luminance gain factor numerator information is expressed as tei_luminance_gain_factor_num_minus1, the value obtained by adding 1 to tei_luminance_gain_factor_num_minus1 may represent the numerator of the luminance gain factor for the thumbnail picture. tei_luminance_gain_factor_num_minus1 must be within the range of 0 to 65,535 (inclusive).

[0420] The luminance gain factor denominator information included in the SEI message related to TEI may represent the denominator of the luminance gain factor for the thumbnail picture. For example, the value of the luminance gain factor denominator information must be within the range of 0 to 65,535 (inclusive).

[0421] The luminance gain factor denominator information may take various forms and be expressed by various names. For example, the luminance gain factor denominator information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the luminance gain factor denominator information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, the luminance gain factor denominator information may include tei_luminance_gain_factor_denom_minus1, but is not limited thereto.

[0422] For example, if the luminance gain factor denominator information is expressed as tei_luminance_gain_factor_denom_minus1, the value obtained by adding 1 to tei_luminance_gain_factor_denom_minus1 may represent the denominator of the luminance gain factor for the thumbnail picture. tei_luminance_gain_factor_denom_minus1 must be within the range of 0 to 65,535 (inclusive).

[0423] Meanwhile, the SEI message related to TEI may further include blank picture existence information, blank picture duration existence information, blank picture duration unit information, blank picture count information, and blank picture sample information.

[0424] Here, blank picture existence information may indicate whether syntax elements of blank picture count information and blank picture sample information exist in the SEI message. For example, blank picture existence information with a value equal to 1 may indicate that syntax elements of blank picture count information and blank picture sample information exist in the thumbnail-related SEI message. Additionally, blank picture existence information with a value equal to 0 may indicate that syntax elements of blank picture count information and blank picture sample information do not exist in the thumbnail-related SEI message. However, this is not limited thereto, and what blank picture existence information with a value equal to 1 indicates may be interchangeable with what blank picture existence information with a value equal to 0 indicates.

[0425] Blank picture presence information may take various forms and may be expressed by various names. For example, blank picture presence information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture presence information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, blank picture presence information may include tei_blank_pic_present_flag or tei_blank_pic_present_idc, but is not limited thereto.

[0426] For example, if blank picture presence information is represented by tei_blank_pic_present_flag, tei_blank_pic_present_flag identical to 1 may indicate that syntax elements tei_num_blank_pics_minus1 and tei_blank_pic_sample[ i ][ c ] exist in the SEI message, and also that syntax elements tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] exist in the SEI message. A tei_blank_pic_present_flag equal to 0 may indicate that the syntax elements tei_num_blank_pics_minus1, tei_blank_pic_sample[ i ][ c ], tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] do not exist in the SEI message.

[0427] Additionally, blank picture duration existence information may indicate whether a syntax element of blank picture duration unit information exists. For example, blank picture duration existence information with a value equal to 1 may indicate that a syntax element of blank picture duration unit information exists within a thumbnail-related SEI message. Additionally, blank picture duration existence information with a value equal to 0 may indicate that a syntax element of blank picture duration unit information does not exist within a thumbnail-related SEI message. However, this is not limited thereto, and what blank picture duration existence information with a value equal to 1 indicates may be interchangeable with what blank picture duration existence information with a value equal to 0 indicates.

[0428] The blank picture duration present information may take various forms and may be represented by various names. For example, the blank picture duration present information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the blank picture duration present information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the blank picture duration present information may include tei_num_units_in_blank_pic_dur_present_flag[ i ] or tei_num_units_in_blank_pic_dur_present_idc[ i ], but is not limited thereto.

[0429] For example, if the blank picture duration information is represented by tei_num_units_in_blank_pic_dur_present_flag[ i ], tei_num_units_in_blank_pic_dur_present_flag[ i ], which is equal to 1, may indicate that the syntax element tei_num_units_in_blank_pic_dur[ i ] described below exists. tei_num_units_in_blank_pic_dur_present_flag[ i ], which is equal to 0, may indicate that the syntax element tei_num_units_in_blank_pic_dur[ i ] described below does not exist.

[0430] Information regarding the existence of a blank picture duration can be generated based on information regarding the unit duration of a blank picture duration. That is, the encoding device can set information regarding the existence of a blank picture duration based on whether the unit duration information regarding a blank picture duration is included in the SEI message related to the TEI, and include this information in the SEI message related to the TEI.

[0431] For example, if a syntax element of blank picture duration unit information is included in the TEI-related SEI message, the encoding device may include the blank picture duration existence information in the TEI-related SEI message by setting the blank picture duration existence information to indicate that the blank picture duration unit information exists within the TEI-related SEI message. On the other hand, if the blank picture duration unit information is not included in the TEI-related SEI message, the encoding device may include the blank picture duration existence information in the TEI-related SEI message by setting the blank picture duration existence information to indicate that the blank picture duration unit information does not exist within the TEI-related SEI message.

[0432] Here, the blank picture duration unit information may represent the number of time units corresponding to the display duration of the blank picture among the time units of a clock operating with frequency time scale information (Hz). For example, the blank picture duration unit information may represent the number of time units corresponding to the display duration of the blank picture among the time units based on the above time scale information.

[0433] Blank picture duration unit information may take various forms and may be represented by various names. For example, blank picture duration unit information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture duration unit information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, blank picture duration unit information may include tei_num_units_in_blank_pic_dur[ i ], but is not limited thereto.

[0434] For example, if the blank picture duration unit information is represented as tei_num_units_in_blank_pic_dur[ i ], tei_num_units_in_blank_pic_dur[ i ] may represent the number of time units corresponding to the display duration of the i-th blank picture among the time units of a clock operating at frequency tsi_time_scale (Hz). tei_num_units_in_blank_pic_dur[ i ] must be greater than 0. If the display duration of the i-th blank picture is expressed in seconds, it may be equal to the quotient obtained by dividing tei_num_units_in_blank_pic_dur[ i ] by tsi_time_scale. For example, if the display duration of the i-th blank picture is 0.04 seconds, tsi_time_scale and tei_num_units_in_dur can be 27,000,000 and 1,080,000, respectively.

[0435] The blank picture count information can indicate the number of blank pictures following the current picture in the output order.

[0436] Blank picture count information may take various forms and may be represented by various names. For example, blank picture count information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture count information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, blank picture count information may include tei_num_blank_pics_minus1, but is not limited thereto.

[0437] For example, if the number of blank pictures is represented as tei_num_blank_pics_minus1, the value obtained by adding 1 to tei_num_blank_pics_minus1 can represent the number of blank pictures following the current picture in the output order.

[0438] Blank picture sample information can represent sample information regarding the color components of the blank picture following the current picture in the output sequence.

[0439] Blank picture sample information may take various forms and may be represented by various names. For example, blank picture sample information may be a syntax element or a syntax structure containing one or more syntax elements. For example, blank picture sample information may include a flag consisting of one bit, an indicator consisting of two or more bits or a variable-length bit, or an unsigned integer consisting of two or more bits or a variable-length bit. For example, blank picture sample information may include tei_blank_pic_sample[ c ], but is not limited thereto.

[0440] For example, if blank picture sample information is represented as tei_blank_pic_sample[ c ], tei_blank_pic_sample[ c ] may represent the sample value for the c-th color component of the i-th blank picture following the current picture in the output order. The length of the above syntax element must be BitDepthY bits.

[0441] Additionally, duration existence information included in the TSI-related SEI message may be generated based on at least one of time scale information, duration unit information, blank picture duration existence information, or blank picture duration unit information. That is, the encoding device may generate the duration existence information based on at least one of time scale information, duration unit information, blank picture duration existence information, or blank picture duration unit information.

[0442] Duration existence information can indicate whether a syntax element of time scale information exists in the SEI message, whether a syntax element of duration unit information exists in the TEI-related SEI message associated with the current TSI-related SEI message, and whether blank picture duration existence information and a syntax element of blank picture duration unit information exist in the TEI-related SEI message associated with the current TSI-related SEI message.

[0443] For example, duration existence information with a value equal to 1 may indicate that a syntax element of time scale information exists in the SEI message, a syntax element of duration unit information exists in the TEI-related SEI message associated with the current TSI-related SEI message, and that blank picture duration existence information and a syntax element of blank picture duration unit information exist in the TEI-related SEI message associated with the current TSI-related SEI message. Additionally, duration existence information with a value equal to 0 may indicate that a syntax element of time scale information does not exist in the SEI message, a syntax element of duration unit information does not exist in the TEI-related SEI message associated with the current TSI-related SEI message, and that a syntax element of blank picture duration existence information and a syntax element of blank picture duration unit information does not exist in the TEI-related SEI message associated with the current TSI-related SEI message. However, this is not limited thereto, and what duration existence information with a value equal to 1 represents may be interchangeable with what duration existence information with a value equal to 0 represents.

[0444] Timing information may take various forms and may be represented by various names. For example, the timing information may be a syntax element or a syntax structure containing one or more syntax elements. For example, the timing information may include a flag consisting of one bit or an indicator consisting of two or more bits or a variable-length bit. For example, the timing information may include tsi_timing_info_present, tsi_timing_info_present_flag, or tsi_timing_info_present_idc, but is not limited thereto.

[0445] For example, if duration existence information is represented by tsi_timing_info_present_flag, tsi_timing_info_present_flag identical to 1 may indicate that the syntax element tsi_time_scale exists in the SEI message, that the syntax element tei_num_units_in_dur exists in the TEI-related SEI message associated with the current TSI-related SEI message, and that the syntax elements tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] exist in the TEI-related SEI message associated with the current TSI-related SEI message. A tsi_timing_info_present_flag equal to 0 may indicate that the syntax element tsi_time_scale does not exist in the SEI message, and that the syntax elements tei_num_units_in_dur, tei_num_units_in_blank_pic_dur_present_flag[ i ] and tei_num_units_in_blank_pic_dur[ i ] do not exist in the TEI-related SEI message associated with the current TSI SEI message.

[0446] For example, the encoding device may set duration existence information and include it in the TSI-related SEI message based on whether it includes syntax elements of time scale information in the SEI message, and whether it includes syntax elements of duration unit information and blank picture duration existence information and / or syntax elements of blank picture duration unit information in the TEI-related SEI message associated with the current TSI-related SEI message. For example, if syntax elements of time scale information are included in the TSI-related SEI message, or if duration unit information and / or blank picture duration related syntax elements are included in the TEI-related SEI message, the encoding device may set duration existence information to indicate that the said duration related syntax elements exist within the thumbnail-related SEI message, and include the duration existence information in the TSI-related SEI message. On the other hand, if the TSI-related SEI message does not include syntax elements of time scale information, or the TEI-related SEI message does not include syntax elements of duration unit information and / or blank picture duration, the encoding device may include the duration existence information in the TSI-related SEI message by setting the duration existence information to indicate that the said duration-related syntax elements do not exist.

[0447] For a detailed explanation of the above time scale information and the above duration unit information, refer to the foregoing.

[0448] The encoding device can encode video information (S620).

[0449] For example, the processor of an encoding device can encode video information including thumbnail-related SEI messages.

[0450] An encoding device can encode an image or image information. Here, the image may include a still image or a video, and the image information may refer to pixel data of the image, object information, additional information (e.g., SEI messages, etc.), or a combination thereof.

[0451] An encoding device can generate a bitstream by performing an encoding process on an input image or image information. At this time, the encoding process may include one or more steps such as prediction, transformation, quantization, and entropy encoding, and the generated bitstream can be used as data for reconstructing the image or image information by a decoding device.

[0452] In addition, the present disclosure may be applied not only to general encoding structures for encoding images or image information, but also to encoding methods for thumbnail-related SEI messages (e.g., TSI-related SEI messages and / or TEI-related SEI messages) that include source information and extraction information of thumbnail pictures.

[0453] To provide video preview and search functions, the encoding device may generate one or more thumbnail-related SEI messages associated with a specific picture based on information required for configuring thumbnails for associated video content, and the generated thumbnail-related SEI messages may be included in a bitstream so that they can be utilized in the decoding or post-processing process.

[0454] For example, a TSI-related SEI message may include information indicating the source of a thumbnail image or thumbnail video, and said source information may indicate, for example, whether the thumbnail is sourced from a specific layer of the current bitstream, sourced from another layer, or provided from an external resource (e.g., a URI, etc.) or a separate bitstream. Additionally, the TSI-related SEI message may further include thumbnail identifier information for identifying the thumbnail. Furthermore, a TEI-related SEI message may include extraction information for specifying a picture or an area within a picture to be used as a thumbnail, and said extraction information may include, for example, syntax elements regarding size information and location information of the extraction area.

[0455] Additionally, according to the present invention, an encoding device may generate a thumbnail-related SEI message to signal the segment order of the thumbnail, the picture display time, brightness correction, and the insertion of a blank picture as syntax within a bitstream. Specifically, when the thumbnail consists of a plurality of segments, the encoding device may include syntax elements indicating the output order of each segment and syntax elements indicating the display duration of each picture within the thumbnail in the thumbnail-related SEI message. Additionally, the encoding device may include syntax elements indicating a luminance (luminance) gain for expressing the brightness of the picture and syntax elements indicating a blank picture of a constant sample value following the current picture in the thumbnail-related SEI message. Accordingly, the encoding device can induce the implementation of the same thumbnail representation and operation in various service and device environments, and the decoding device can generate or construct a thumbnail using the information signaled by the syntax elements.

[0456] Video information encoded according to the encoding method (S600) described above can be output in the form of a bitstream. In other words, the bitstream can be generated based on the video information encoded according to the encoding method (S600) described above.

[0457] A bitstream generated based on video information encoded according to the encoding method (S600) described above can be stored on a computer-readable storage medium.

[0458] A bitstream generated based on video information encoded according to the encoding method (S600) described above can be transmitted through a transmission unit and / or a transmission medium.

[0459] According to one embodiment, image information may include SEI messages. SEI messages may convey specific types of information that assist in processes related to the decoding, display, or other processing purposes of image information. Here, the SEI messages may include at least one of TSI-related SEI messages and / or TEI-related SEI messages.

[0460] The names of syntax elements and semantics described in this specification are merely exemplary names for the convenience of explanation, and the scope of the rights of this disclosure is not limited by such names. That is, any configuration that performs the same or substantially corresponding functions as the syntax elements and semantics described in this specification may be included within the spirit and scope of this disclosure, even if its name, notation, or terminology differs.

[0461] FIG. 7 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0462] As illustrated in FIG. 7, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0463] The above encoding server compresses content input from multimedia input devices, such as smartphones, cameras, and camcorders, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and camcorders, generate the bitstream directly, the encoding server may be omitted.

[0464] The bitstream may be generated by a video encoding method and / or encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0465] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0466] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0467] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0468] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0469] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0470] An embodiment according to the present disclosure can be used to encode / decode images.

Claims

In a video decoding method performed by a decoding device, A step of obtaining a Supplemental Enhancement Information (SEI) message related to a thumbnail associated with a picture from a bitstream; and It includes the step of obtaining thumbnail-related information from the above thumbnail-related SEI message, and A video decoding method characterized in that the above thumbnail-related information includes at least one of segment output order information, display duration information, luminance gain factor information, or blank picture information of the above thumbnail picture. In paragraph 1, A video decoding method characterized in that the above thumbnail-related SEI message includes a TSI (Thumbnail Source Information) related SEI message containing source information of the thumbnail picture, and a TEI (Thumbnail Extraction Information) related SEI message containing extraction information of the thumbnail picture. In paragraph 2, Based on the above segment output order information, the output order of the segment to which the current picture belongs within the thumbnail picture is determined, and An image decoding method characterized by the above segment output order information including output order existence information included in the TSI-related SEI message and output order information included in the TEI-related SEI message. In paragraph 3, A picture decoding method characterized in that the picture to which the TEI-related SEI message containing the same output sequence information is applied is included in the same segment. In paragraph 3, An image decoding method characterized by the fact that the output order information is not included in the above TEI-related SEI message, and the value of the output order information is inferred to be 0 based on the fact that the above TEI-related SEI message is the first TEI-related SEI message in the bitstream based on the output order. In paragraph 3, A video decoding method characterized by the fact that the output order information is not included in the above TEI-related SEI message, and based on the fact that the above TEI-related SEI message is not the first TEI-related SEI message in the bitstream according to the output order, the value of the output order information is inferred to be the value of the output order information of the immediately preceding TEI-related SEI message according to the output order plus 1. In paragraph 2, The above display duration information includes at least one of duration existence information, time scale information, or duration unit information included in the TSI-related SEI message, and An image decoding method characterized by the fact that the above time scale information and the above duration unit information are obtained based on the above duration existence information. In Paragraph 7, Based on the above time scale information and the above duration unit information, the display duration of the current picture is determined, and A video decoding method characterized in that the display duration of the current picture is determined based on dividing the duration unit information by the time scale information. In paragraph 2, The above-mentioned luminous gain coefficient information includes at least one of the luminous gain coefficient existence information included in the TSI-related SEI message, the luminous gain coefficient numerator information included in the TEI-related SEI message, or the luminous gain coefficient denominator information. An image decoding method characterized in that the numerator information of the above-mentioned luminance gain coefficient and the denominator information of the above-mentioned luminance gain coefficient are obtained based on the existence information of the above-mentioned luminance gain coefficient. In paragraph 2, A video decoding method characterized by obtaining at least one of time scale information, duration unit information, blank picture duration existence information, or blank picture duration unit information based on duration existence information included in the above TSI-related SEI message. In Paragraph 10, The above TEI-related SEI message further includes blank picture existence information, blank picture count information, and blank picture sample information, and The above blank picture count information indicates the number of blank pictures following the current picture in the output order, and An image decoding method characterized by the blank picture sample information representing sample information for the color components of a blank picture following the current picture in the output sequence. In Paragraph 10, The above blank picture duration unit information represents the number of time units corresponding to the display duration of the blank picture among the time units based on the above time scale information, and An image decoding method characterized by the fact that the blank picture duration unit information is obtained based on the blank picture duration existence information. In a video encoding method performed by an encoding device, A step of generating a thumbnail-related SEI (Supplemental Enhancement Information) message containing thumbnail-related information associated with a picture; and The method includes the step of encoding video information containing the above thumbnail-related SEI message, and A video encoding method characterized in that the above thumbnail-related information includes at least one of segment output order information, display duration information, luminance gain factor information, or blank picture information of the above thumbnail picture. In Paragraph 13, A video encoding method characterized in that the above thumbnail-related SEI message includes a TSI (Thumbnail Source Information) related SEI message containing source information of the thumbnail picture, and a TEI (Thumbnail Extraction Information) related SEI message containing extraction information of the thumbnail picture. In Paragraph 14, The segment output order information is determined based on the output order of the segment to which the current picture belongs within the thumbnail picture, and A video encoding method characterized by the fact that the segment output order information includes output order existence information included in the TSI-related SEI message and output order information included in the TEI-related SEI message. In Paragraph 14, The above display duration information includes at least one of duration existence information, time scale information, or duration unit information included in the TSI-related SEI message, and An image encoding method characterized in that the above duration existence information is generated based on the above time scale information and the above duration unit information. In Paragraph 14, The above-mentioned luminous gain coefficient information includes at least one of the luminous gain coefficient existence information included in the TSI-related SEI message, the luminous gain coefficient numerator information included in the TEI-related SEI message, or the luminous gain coefficient denominator information. An image encoding method characterized in that the above-mentioned luminance gain coefficient existence information is generated based on the above-mentioned luminance gain coefficient numerator information and the above-mentioned luminance gain coefficient denominator information. In Paragraph 14, The above TEI-related SEI message further includes blank picture existence information, blank picture duration existence information, blank picture duration unit information, blank picture count information, and blank picture sample information, and A video encoding method characterized in that the duration existence information included in the above TSI-related SEI message is generated based on at least one of the time scale information, the duration unit information, the blank picture duration existence information, or the blank picture duration unit information. A non-transitory computer-readable storage medium for storing a bitstream generated by the image encoding method described in paragraph 13. A method for transmitting data regarding an image, wherein the method comprises the step of acquiring image information, wherein the image information includes a thumbnail-related SEI (Supplemental Enhancement Information) message containing thumbnail-related information associated with a picture; and The method includes the step of transmitting the data including the above image information, A data transmission method characterized in that the above thumbnail-related information includes at least one of segment output order information, display duration information, luminance gain coefficient information, or blank picture information of the above thumbnail picture.

Images