Image decoding method, image encoding method, computer-readable storage medium, and data transmission method for image

Abstract

A method by which a decoding device decodes an image, according to the present disclosure, comprises the steps of: acquiring, from a bitstream, at least one image format metadata (IFM) supplemental enhancement information (SEI) message associated with the current picture; and acquiring the IFM on the basis of the at least one IFM SEI message, wherein the IFM includes information about whether a URI for the IFM is present, payload information about the IFM, and a zero bit for byte alignment in the IFM SEI message.

Description

Video decoding method, video encoding method, computer-readable storage medium and method for transmitting data for video

[0001] The present disclosure relates to a method for decoding / encoding image information, a computer-readable storage medium for storing image information, and a method for transmitting image information.

[0002] Recently, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), has been increasing across various fields. As video data becomes higher in resolution and quality, the relative amount of information or bits transmitted increases compared to conventional video data. This increase in transmitted information or bits leads to higher transmission and storage costs.

[0003] Accordingly, high-efficiency video compression technology is required to effectively transmit, store, and play back high-resolution, high-quality video information.

[0004] The present disclosure aims to ensure parsing stability and syntactic consistency of a bitstream by ensuring that image format metadata is parsed consistently in byte units through a byte alignment process.

[0005] The present disclosure enables consistency in bit parsing structures in both cases where the payload information of image format metadata is directly included within the bitstream and where it is referenced via an external URI.

[0006] The present disclosure prevents data parsing errors caused by bit boundary mismatches and enables stable parsing of metadata by performing common byte alignment using zero bits within supplemental enhancement information (SEI) messages, taking into account differences between data formats.

[0007] The present disclosure aims to improve bitstream conformance and interoperability by enabling the application of the same parsing structure in encoders and decoders regardless of the form in which image format metadata exists.

[0008] The technical problems to be solved in this disclosure are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0009] A method for decoding image information according to one aspect of the present disclosure comprises the steps of obtaining at least one Image Format Metadata (IFM) supplemental enhancement information (SEI) message associated with a current picture from a bitstream, and obtaining image format metadata based on the at least one IFM SEI message. The image format metadata is characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

[0010] According to one aspect of the present disclosure, an apparatus for decoding image information obtains at least one Image Format Metadata (IFM) supplemental enhancement information (SEI) message associated with a current picture from a bitstream, and obtains image format metadata based on the IFM SEI message. The image format metadata is characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

[0011] In a method or device for decoding the above image information, image format metadata can be obtained by obtaining information on whether a URI for the image format metadata exists from the IFM SEI message, obtaining a zero bit for byte alignment within the IFM SEI message based on the information on whether a URI for the image format metadata exists, and obtaining payload information for the image format metadata.

[0012] In a method or device for decoding the above image information, if the information on whether a URI for the image format metadata exists indicates that a URI for the image format metadata does not exist, the payload information for the image format metadata may be obtained after obtaining the zero bit for byte alignment within the IFM SEI message.

[0013] In a method or device for decoding the above image information, if the information regarding the existence of a URI for the image format metadata indicates that the URI for the image format metadata does not exist, and the current bit position within the IFM SEI message is not aligned to a byte unit position, the byte alignment may be performed based on the zero bit.

[0014] In a method or device for decoding the above image information, when information regarding the existence of a URI for the image format metadata indicates that a URI for the image format metadata exists, the URI information for the image format metadata may be characterized by being obtained after obtaining the zero bit for byte alignment within the IFM SEI message.

[0015] In a method or device for decoding the above image information, the payload information for the image format metadata and the URI information for the image format metadata may be characterized by having different data formats.

[0016] In a method or device for decoding the above image information, the zero bit may be characterized as being a single bit with a value set to 0.

[0017] A method for encoding image information according to one aspect of the present disclosure comprises the steps of: generating at least one Image Format Metadata (IFM) supplemental enhancement information (SEI) message associated with a current picture based on image format metadata; and encoding image information including the at least one IFM SEI message. The image format metadata is characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

[0018] According to one aspect of the present disclosure, an apparatus for encoding image information generates at least one Image Format Metadata (IFM) supplemental enhancement information (SEI) message associated with a current picture based on image format metadata, and encodes image information including said at least one IFM SEI message. The image format metadata is characterized by including information on whether a URI exists for said image format metadata, payload information for said image format metadata, and a zero bit for byte alignment within said IFM SEI message.

[0019] In a method or device for encoding the above image information, the IFM SEI message can be generated by encoding payload information for the image format metadata, generating and encoding information regarding the existence of a URI for the image format metadata, and inserting a zero bit for byte alignment within the IFM SEI message based on the image format metadata.

[0020] In a method or device for encoding the above image information, if a URI for the image format metadata does not exist and the current bit position within the IFM SEI message is not aligned to a byte unit position, the payload information for the image format metadata may be characterized by being encoded after inserting the zero bit for byte alignment within the IFM SEI message.

[0021] In a method or device for encoding the above image information, if a URI for the image format metadata exists, the URI information for the image format metadata may be characterized by being encoded after inserting the zero bit for byte alignment within the IFM SEI message.

[0022] In a method or device for encoding the above-mentioned image information, the payload information for the image format metadata and the URI information for the image format metadata may be characterized by having different data formats.

[0023] In a method or device for encoding the above-mentioned image information, the zero bit may be characterized as being a single bit with a value set to 0.

[0024] According to one aspect of the present disclosure, a bitstream generated by a video encoding method is stored in a computer-readable storage medium. The bitstream is generated based on image information including at least one Image Format Metadata (IFM) supplemental enhancement information (SEI) message associated with a current picture, based on image format metadata. The image format metadata is characterized by including information on whether a URI for the image format metadata exists, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

[0025] According to one aspect of the present disclosure, a method for transmitting data for an image comprises the steps of acquiring image information, wherein the image information includes at least one Image Format Metadata (IFM) supplemental enhancement information (SEI) message associated with a current picture, and transmitting the data including the image information. The image format metadata is characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

[0026] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0027] According to the present disclosure, by ensuring that image format metadata is parsed consistently in byte units through a byte alignment process, parsing stability and syntactic consistency of the bitstream can be secured.

[0028] According to the present disclosure, consistency of the bit parsing structure can be maintained in both cases where the payload information of image format metadata is directly included in the bitstream and where it is referenced via an external URI.

[0029] According to the present disclosure, by performing a common byte alignment using zero bits within a supplemental enhancement information (SEI) message while considering differences between data formats, data parsing errors caused by bit boundary mismatches are prevented, and stable parsing of metadata is enabled.

[0030] According to the present disclosure, the same parsing structure can be applied in the encoder and decoder regardless of the form in which image format metadata exists, thereby improving the conformance and interoperability of the bitstream.

[0031] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure belongs from the description below.

[0032] FIG. 1 is a schematic diagram illustrating a video coding system to which an embodiment according to the present disclosure can be applied.

[0033] FIG. 2 is a schematic diagram showing an encoding device to which an embodiment according to the present disclosure can be applied.

[0034] FIG. 3 is a schematic diagram showing a decoding device to which an embodiment according to the present disclosure can be applied.

[0035] Figure 4 illustrates an exemplary hierarchical structure for a coded video / image.

[0036] FIG. 5 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0037] FIG. 6 is a diagram illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0038] FIG. 7 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0039] Hereinafter, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0040] In describing the embodiments of the present disclosure, detailed descriptions of known configurations or functions are omitted if it is determined that such descriptions could obscure the essence of the present disclosure. Additionally, parts of the drawings unrelated to the description of the present disclosure have been omitted, and similar parts are denoted by similar reference numerals.

[0041] In the present disclosure, when a component is described as being "connected," "combined," or "joined" with another component, this may include not only a direct connection but also an indirect connection in which another component exists in between. Furthermore, when a component is described as "comprising" or "having" another component, this means that, unless specifically stated otherwise, it does not exclude the other component but may include an additional component.

[0042] In the present disclosure, terms such as first, second, etc. are used solely for the purpose of distinguishing one component from another and do not limit the order or importance of the components unless specifically stated otherwise. Accordingly, within the scope of the present disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and likewise, a second component in one embodiment may be referred to as a first component in another embodiment.

[0043] In this disclosure, distinct components are intended to clearly describe their respective features and do not imply that the components are separate. That is, multiple components may be integrated to form a single hardware or software unit, or a single component may be distributed to form multiple hardware or software units. Accordingly, such integrated or distributed embodiments are included within the scope of this disclosure, unless otherwise noted.

[0044] In the present disclosure, the components described in various embodiments do not necessarily mean essential components, and some may be optional components. Accordingly, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Furthermore, embodiments including additional components in addition to the components described in various embodiments are also included within the scope of the present disclosure.

[0045] The present disclosure relates to the encoding and decoding of images. For example, the methods and embodiments disclosed in this document may be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard) or next-generation video / image coding standards (e.g., H.267 or H.268).

[0046] The present disclosure presents various embodiments relating to video / image coding, and unless otherwise stated, said embodiments may be performed in combination with one another.

[0047] Unless newly defined in this disclosure, the terms used herein may have the ordinary meanings commonly used in the technical field to which this disclosure belongs.

[0048] In this disclosure, "video" may refer to a set of images over time. In this disclosure, "picture" generally refers to a unit representing a single image at a specific time, and a slice / tile is a unit that constitutes a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular area of ​​rows of CTUs within a tile in a picture. In this document, tile groups and slices may be used interchangeably. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.

[0049] In the present disclosure, "pixel" or "pel" may refer to the smallest unit constituting a picture (or image). Additionally, "sample" may be used as a term corresponding to pixel. A sample may generally represent a pixel or a pixel value, may represent only the pixel / pixel value of the luminance component, or may represent only the pixel / pixel value of the chroma component.

[0050] In this disclosure, "unit" may represent a basic unit of image processing. A unit may include at least one of a specific area of ​​a picture and information related to that area. A unit may include one luminance block and two chroma (e.g., cb, cr) blocks. Depending on the case, the term "unit" may be used interchangeably with terms such as "block" or "area." In general, an MxN block may include samples (or sample arrays) or a set (or array) of transform coefficients consisting of M columns and N rows.

[0051] In the present disclosure, "current block" may mean one of "current coding block," "current coding unit," "block to be encoded," "block to be decoded," or "block to be processed." When prediction is performed, "current block" may mean "current prediction block" or "block to be predicted." When transformation (inverse transformation) / quantization (inverse quantization) is performed, "current block" may mean "current transformation block" or "block to be transformed." When filtering is performed, "current block" may mean "block to be filtered."

[0052] In the present disclosure, "current block" may mean a block comprising both a luminous component block and a chroma component block, or "luma block of the current block," unless explicitly stated as a chroma block. The luminous component block of the current block may be expressed by including an explicit description of a luminous component block, such as "luma block" or "current luminous block." Additionally, the chroma component block of the current block may be expressed by including an explicit description of a chroma component block, such as "chroma block" or "current chroma block."

[0053] In the present disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Additionally, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0054] In the present disclosure, "or" may be interpreted as "and / or". For example, "A or B" may mean 1) "A" only, 2) "B" only, or 3) "A and B". Alternatively, in the present disclosure, "or" may mean "additionally or alternatively".

[0055] FIG. 1 is a schematic diagram illustrating a video / image coding system to which an embodiment according to the present disclosure can be applied.

[0056] Referring to FIG. 1, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device may transmit encoded video / image or data in the form of a file or streaming to the receiving device via a digital storage medium or a network.

[0057] The source device may include a video source, an encoding device, and a transmission unit. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0058] A video source may acquire video / images through processes such as video / image capture, synthesis, or generation. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and may generate video / images (electronically). For example, virtual video / images may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process in which related data is generated.

[0059] The encoding device can encode input video / images. The encoding device can perform a series of procedures, such as prediction, transformation, and quantization, for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0060] The transmission unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device in the form of a file or streaming via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0061] The decoding device can decode video / images by performing a series of procedures such as inverse quantization, inverse transform, and prediction corresponding to the operation of the encoding device.

[0062] The renderer can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0063] FIG. 2 is a schematic diagram illustrating an encoding device to which an embodiment according to the present disclosure can be applied.

[0064] Referring to FIG. 2, the encoding device (200) may be configured to include an image partitioner (210), a predictor (220), a residual processor (230), an entropy encoder (240), an adder (250), a filter (260), and a memory (270). The predictor (220) may include an inter-predictor (221) and an intra-predictor (222). The residual processor (230) may include a transformer (232), a quantizer (233), a dequantizer (234), and an inverse transformer (235). The residual processor (230) may further include a subtractor (231). The addition unit (250) may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit (210), prediction unit (220), residual processing unit (230), entropy encoding unit (240), addition unit (250), and filtering unit (260) may be configured by one or more hardware components (e.g., an encoder chipset or processor) according to the embodiment. Additionally, the memory (270) may include a DPB (Decoded Picture Buffer) and may be configured by a digital storage medium. The hardware component may further include the memory (270) as an internal / external component.

[0065] The image segmentation unit (210) can divide an input image (or picture, frame) input to an encoding device (200) into one or more processing units. For example, the processing unit may be called a coding unit (CU). A coding unit may be recursively divided into a coding tree unit (CTU) or a largest coding unit (LCU) according to a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, a single coding unit may be divided into multiple coding units of a deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. For example, a quad-tree structure may be applied first, and a binary-tree structure and / or a ternary-tree structure may be applied later. Alternatively, a binary-tree structure may be applied first. A coding procedure according to the present disclosure may be performed based on the final coding unit that is no longer divided. In this case, based on coding efficiency according to image characteristics, the maximum coding unit may be used directly as the final coding unit, or, if necessary, the maximum coding unit may be recursively divided into lower-depth coding units so that a coding unit of the optimal size is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may each be divided or partitioned from the final coding unit.The above prediction unit may be a unit of sample prediction, and the above transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0066] The term "unit" may be used interchangeably with terms such as "block" or "area" depending on the context. In general, an MxN block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chroma component. A sample may be used to refer to a single picture (or image) as a term corresponding to a pixel or pel.

[0067] The encoding device (200) can generate a residual signal (residual block, residual sample array) by subtracting a prediction signal (predicted block, prediction sample array) output from an inter prediction unit (221) or an intra prediction unit (222) from an input image signal (original block, original sample array), and the generated residual signal is transmitted to a conversion unit (232). In this case, as illustrated, the unit that subtracts the prediction signal (predicted block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device (200) may be called a subtraction unit (231). The prediction unit (220) can perform a prediction for a block to be processed (hereinafter, current block) and generate a predicted block (predicted block) containing prediction samples for said current block. The prediction unit (220) can determine whether intra prediction is applied or inter prediction is applied in units of the current block or CU. The prediction unit (220) can generate various information regarding prediction, such as prediction mode information, as described below in the description of each prediction mode, and transmit it to the entropy encoding unit (240). The information regarding prediction can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0068] The intra prediction unit (222) can predict the current block by referring to samples within the current picture. The referenced samples may be located near the current block or away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is merely an example, and depending on the settings, more or fewer directional prediction modes may be used. The intra prediction unit (222) may also determine the prediction mode applied to the current block by using the prediction mode applied to the surrounding blocks.

[0069] The inter prediction unit (221) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. The reference picture containing the reference blocks and the reference picture containing the temporal neighboring blocks may be the same or different from each other. The temporal neighboring blocks may be referred to by names such as collocated reference block, collocated CU (colCU), etc. A reference picture containing the aforementioned temporal surrounding blocks may be called a collocated picture (colPic). For example, the inter prediction unit (221) may construct a list of motion information candidates based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, for example, in the case of skip mode and merge mode, the inter prediction unit (221) may use the motion information of surrounding blocks as motion information of the current block. In the case of skip mode, unlike merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of surrounding blocks is used as a motion vector predictor, and the motion vector of the current block can be indicated by signaling the motion vector difference.

[0070] The prediction unit (220) may generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit (220) may apply intra prediction or inter prediction for the prediction of the current block, as well as apply intra prediction and inter prediction simultaneously. A prediction method that applies intra prediction and inter prediction simultaneously for the prediction of the current block may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit (220) may be based on an intra block copy (IBC) prediction mode or a palette mode for the prediction of the block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. Palette mode can be viewed as an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within a picture can be signaled based on information regarding palette tables and palette indices.

[0071] The prediction signal generated through the prediction unit (220) can be used to generate a restoration signal or to generate a residual signal. The subtraction unit (231) can generate a residual signal (residual signal, residual block, residual sample array) by subtracting the prediction signal (predicted block, prediction sample array) output from the prediction unit (220) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit (232).

[0072] The transformation unit (232) can generate transform coefficients by applying a transformation technique to a residual signal. For example, the transformation technique may include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transformation obtained from a graph when the relationship information between pixels is represented as a graph. CNT refers to a transformation obtained based on a prediction signal generated using all previously reconstructed pixels. The transformation process may be applied to a block of pixels of the same size in a square, or to a block of variable size that is not square.

[0073] The quantization unit (233) can quantize the transformation coefficients and transmit them to the entropy encoding unit (240). The entropy encoding unit (240) can encode the quantized signal (information regarding the quantized transformation coefficients) and output it as a bitstream. The information regarding the quantized transformation coefficients may be called residual information. The quantization unit (233) can rearrange the block-shaped quantized transformation coefficients into a one-dimensional vector form based on the coefficient scan order, and can also generate information regarding the quantized transformation coefficients based on the one-dimensional vector-shaped quantized transformation coefficients.

[0074] The entropy encoding unit (240) can perform various encoding methods such as, for example, exponential Golomb, CAVLC (context-adaptive variable length coding), CABAC (context-adaptive binary arithmetic coding), etc. The entropy encoding unit (190) may encode information required for video / image restoration (e.g., values ​​of syntax elements) together or separately, in addition to quantized transform coefficients. The encoded information (e.g., encoded video / image information) may be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The signaling information, transmitted information, and / or syntax elements mentioned in the present disclosure may be included in the video / image information. The video / image information may be encoded through the encoding procedure described above and included in the bitstream.

[0075] The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmitting a signal output from the entropy encoding unit (240) and / or a storage unit (not shown) for storing it may be provided as an internal / external element of the encoding device (200), or the transmission unit may be provided as a component of the entropy encoding unit (240).

[0076] The quantized transformation coefficients output from the quantization unit (233) can be used to generate a residual signal. For example, a residual signal (residual block or residual samples) can be restored by applying inverse quantization and inverse transformation to the quantized transformation coefficients through the inverse quantization unit (234) and the inverse transformation unit (235).

[0077] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding and / or restoration process.

[0078] The adder (250) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (221) or the intra prediction unit (222). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The adder (250) may be called a reconstructed unit or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0079] The filtering unit (260) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (260) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (270), specifically in the DPB of memory (170). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (260) can generate various information regarding filtering and transmit it to the entropy encoding unit (240), as described below in the description of each filtering method. The information regarding filtering can be encoded in the entropy encoding unit (240) and output in the form of a bitstream.

[0080] The modified restored picture transmitted to the memory (270) can be used as a reference picture in the inter-prediction unit (221). Through this, the encoding device (200) can avoid prediction mismatches between the encoding device (200) and the decoding device when inter-prediction is applied, and can also improve encoding efficiency.

[0081] The DPB in memory (270) can store a modified restored picture to be used as a reference picture in the inter prediction unit (221). Memory (270) can store motion information of blocks from which motion information is derived (or encoded) in the current picture and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter prediction unit (221) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory (270) can store restoration samples of restored blocks in the current picture and transmit them to the intra prediction unit (222).

[0082] FIG. 3 is a schematic diagram illustrating a decoding device to which an embodiment according to the present disclosure can be applied.

[0083] As illustrated in FIG. 3, the decoding device (300) may be configured to include an entropy decoder (310), a residual processor (320), a predictor (330), an adder (340), a filter (350), and a memory (360). The predictor (330) may include an inter-predictor (332) and an intra-predictor (331). The residual processor (320) may include a dequantizer (321) and an inverse transformer (321). The aforementioned entropy decoding unit (310), residual processing unit (320), prediction unit (330), addition unit (340), and filtering unit (350) may be configured by a single hardware component (e.g., a decoder chipset or a processor) according to an embodiment. Additionally, the memory (360) may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory (360) as an internal / external component.

[0084] When a bitstream containing video / image information is input, the decoding device (300) can restore the image by performing a process corresponding to the process performed by the encoding device (200) of FIG. 2. For example, the decoding device (300) can perform decoding using a processing unit applied in the encoding device (200). Thus, the processing unit for decoding may be, for example, a coding unit. The coding unit may be a coding tree unit, or a maximum coding unit may be obtained by dividing it according to a quad tree structure, a binary tree structure, and / or a binary tree structure. And, the restored image signal decoded and output through the decoding device (300) can be played back through a playback device (not shown).

[0085] The decoding device (300) can receive a signal output from the encoding device (200) of FIG. 2 in the form of a bitstream. The received signal can be decoded through an entropy decoding unit (310). For example, the entropy decoding unit (310) can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information regarding various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information may further include general constraint information. The decoding device (300) can decode the picture based on the information regarding the parameter sets and / or the general constraint information. The signaling / received information and / or syntax elements described below can be obtained from the bitstream by decoding through the decoding procedure. For example, the entropy decoding unit (310) can decode information within the bitstream based on coding methods such as exponential chord coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transformation coefficients regarding residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information on the syntax element to be decoded and decoding information of surrounding and decoding target blocks or information on symbols / bins decoded in the previous step, predict the probability of occurrence of the bin according to the determined context model, and perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin after determining the context model. Among the information decoded in the entropy decoding unit (310), information regarding prediction is provided to the prediction unit (330), and residual values ​​for which entropy decoding was performed in the entropy decoding unit (310), i.e., quantized transformation coefficients and related parameter information, can be input to the residual processing unit (320). The residual processing unit (320) can derive residual signals (residual blocks, residual samples, residual sample array). Additionally, among the information decoded in the entropy decoding unit (310), information regarding filtering can be provided to the filtering unit (350). Meanwhile, a receiving unit (not shown) that receives a signal output from an encoding device may be further configured as an internal / external element of the decoding device (300), or the receiving unit may be a component of the entropy decoding unit (310). Meanwhile, the decoding device according to the present document may be called a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit (310), and the sample decoder may include at least one of the inverse quantization unit (321), inverse transform unit (322), adder (340), filtering unit (350), memory (360), inter prediction unit (332), and intra prediction unit (331).

[0086] In the inverse quantization unit (321), the quantized transformation coefficients can be inversely quantized to output transformation coefficients. The inverse quantization unit (321) can rearrange the quantized transformation coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scan order performed in the encoding device (200). The inverse quantization unit (321) can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) and obtain transformation coefficients.

[0087] In the inverse conversion unit (322), the conversion coefficients can be inversely converted to obtain a residual signal (residual block, residual sample array).

[0088] The prediction unit (330) can generate a prediction signal based on various prediction methods described below. For example, the prediction unit may apply intra prediction or inter prediction for a single block, and may also apply intra prediction and inter prediction simultaneously. This may be called combined inter and intra prediction (CIIP). Additionally, the prediction unit may be based on an intra block copy (IBC) prediction mode or a palette mode for predicting a block. The IBC prediction mode or palette mode may be used for content video / video coding, such as in games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction in that it derives a reference block within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document. The palette mode can be viewed as an example of intra coding or intra prediction. When the palette mode is applied, information regarding the palette table and palette index can be included in the above video / image information and signaled.

[0089] The intra prediction unit (331) can predict the current block by referring to samples within the current picture. The description of the intra prediction unit (222) may be applied equally to the intra prediction unit (331). The referenced samples may be located in the neighborhood of the current block or located away from it, depending on the prediction mode. In intra prediction, the prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The intra prediction unit (331) may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0090] The inter prediction unit (332) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, to reduce the amount of motion information transmitted in the inter prediction mode, motion information can be predicted in blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, neighboring blocks may include spatial neighboring blocks existing within the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (332) may construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter-prediction can be performed based on various prediction modes (techniques), and information regarding the prediction may include information indicating the mode (technique) of inter-prediction for the current block.

[0091] The adder (340) can generate a restoration signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the prediction signal (predicted block, predicted sample array) output from the prediction unit (330) (including the inter prediction unit (332) and / or intra prediction unit (331)). In cases where there is no residual for the block to be processed, such as when a skip mode is applied, the predicted block can be used as the restoration block. The description of the adder (250) can be applied equally to the adder (340). The adder (340) may be called a restoration unit or a restoration block generation unit. The generated restoration signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after undergoing filtering as described below.

[0092] Meanwhile, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0093] The filtering unit (350) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (350) can generate a modified restored picture by applying various filtering methods to the restored picture, and can store the modified restored picture in memory (360), specifically in the DPB of memory (360). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0094] The (modified) restored picture stored in the DPB of the memory (360) can be used as a reference picture in the inter-prediction unit (332). The memory (360) can store motion information of blocks from which motion information within the current picture has been derived (or decoded) and / or motion information of blocks within the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit (332) to be used as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. The memory (360) can store restoration samples of blocks restored within the current picture and transmit them to the intra-prediction unit (331).

[0095] In this specification, the embodiments described in the filtering unit (260), inter prediction unit (221), and intra prediction unit (222) of the encoding device (200) may be applied to the filtering unit (350), inter prediction unit (332), and intra prediction unit (331) of the decoding device (300) in the same or corresponding manner.

[0096] Figure 4 illustrates an exemplary hierarchical structure for a coded video / image.

[0097] Referring to Figure 4, the coded image is divided into a Video Coding Layer (VCL) that handles the decoding processing of the image and the image itself, a subsystem that transmits and stores the encoded information, and a Network Abstraction Layer (NAL) that exists between the VCL and the subsystem and is responsible for network adaptation functions.

[0098] In VCL, VCL data containing compressed image data (slice data) can be generated, or parameter sets containing information such as Picture Parameter Set (PPS), Sequence Parameter Set (SPS), and Video Parameter Set (VPS), or SEI (Supplemental Enhancement Information) messages that are additionally required in the decoding process of the image can be generated.

[0099] In NAL, a NAL unit can be created by adding header information (NAL unit header) to the Raw Byte Sequence Payload (RBSP) generated in VCL. In this case, the RBSP refers to slice data, parameter sets, SEI messages, etc. generated in VCL. The NAL unit header may include NAL unit type information specified according to the RBSP data included in the NAL unit.

[0100] As shown in FIG. 4, NAL units can be classified into VCL NAL units and Non-VCL NAL units depending on the RBSP generated in VCL. A VCL NAL unit may refer to a NAL unit containing information about an image (slice data), and a Non-VCL NAL unit may refer to a NAL unit containing information necessary to decode an image (parameter set or SEI message).

[0101] The aforementioned VCL NAL unit and Non-VCL NAL unit can be transmitted over a network by attaching header information according to the data specifications of the underlying system. For example, the NAL unit can be transformed into a data format of a specified specification, such as H.266 / VVC file format, RTP (Real-time Transport Protocol), TS (Transport Stream), etc., and transmitted over various networks.

[0102] As described above, the NAL unit type can be determined according to the RBSP data structure included in the NAL unit, and information about this NAL unit type can be stored in the NAL unit header and signaled.

[0103] For example, NAL units can be broadly classified into VCL NAL unit types and Non-VCL NAL unit types depending on whether they contain information about the image (slice data). VCL NAL unit types can be classified according to the properties and types of the picture included in the VCL NAL unit, while Non-VCL NAL unit types can be classified according to the types of parameter sets.

[0104] The following is an example of a NAL unit type specified according to the type of parameter set included in the Non-VCL NAL unit type.

[0105] - APS (Adaptation Parameter Set) NAL unit: Type for the NAL unit containing the APS

[0106] - DPS(Decoding Parameter Set) NAL unit: Type for the NAL unit containing the DPS

[0107] - VPS (Video Parameter Set) NAL unit: Type for the NAL unit containing the VPS

[0108] - SPS (Sequence Parameter Set) NAL unit: Type for the NAL unit containing the SPS

[0109] - PPS(Picture Parameter Set) NAL unit: Type for the NAL unit containing the PPS

[0110] The above-described NAL unit types have syntax information for the NAL unit type, and said syntax information can be stored in the NAL unit header and signaled. For example, said syntax information may be nal_unit_type, and NAL unit types may be specified by the nal_unit_type value.

[0111] A slice header (slice header syntax, slice header information) may include information / parameters that can be commonly applied to the slice. The APS (APS syntax) or PPS (PPS syntax) may include information / parameters that can be commonly applied to one or more slices or pictures. The SPS (SPS syntax) may include information / parameters that can be commonly applied to one or more sequences. The VPS (VPS syntax) may include information / parameters that can be commonly applied to multiple layers. The DPS (DPS syntax) may include information / parameters that can be commonly applied to the entire video. The DPS may include information / parameters related to the concatenation of a CVS (coded video sequence). In the present disclosure, High Level Syntax (HLS) may include at least one of the APS syntax, PPS syntax, SPS syntax, VPS syntax, DPS syntax, or slice header syntax.

[0112] In the present disclosure, image / video information encoded by an encoding device and signaled in the form of a bitstream includes not only information related to picture partitioning, intra / inter prediction information, residual information, in-loop filtering information, etc., but may also include information included in the slice header, information included in the APS, information included in the PPS, information included in the SPS, information included in the VPS, and / or information included in the DPS.

[0113] In the present disclosure, the following descriptors may define the parsing process for each syntactic element.

[0114] - ae(v) is a context-adaptive arithmetic entropy-coded syntactic element.

[0115] - b(8) is a byte with an arbitrary bit string pattern of length 8 bits. The parsing process of this descriptor is defined by the return value of the function read_bits(8).

[0116] - f(n) is an n-bit fixed-pattern bit string where the left bits are written first. The parsing process of this descriptor is defined by the return value of the function read_bits(n).

[0117] - i(n) is a signed integer using n bits, where n is marked as "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process of this descriptor is defined by interpreting the return value of the function read_bits(n) into a two's complement integer representation where the most significant bit is written first.

[0118] - se(v) is a 0th-order Exp-Golomb encoded signed integer syntax element, written starting from the left bit. The parsing process of this descriptor is defined with order k set to 0.

[0119] - st(v) is a null-terminated string encoded in Universal Coded Character Set (UCS) transmission format 8 (UTF-8) characters as defined in ISO / IEC 10646. The parsing process is defined as follows:

[0120] st(v) starts at a byte-aligned position within the bitstream and reads and returns a sequence of consecutive bytes from the bitstream, starting from the current position and excluding the next byte-aligned byte with a value of 0x00. The bitstream pointer then advances by (stringLength + 1) * 8 bits, where stringLength is equal to the number of returned bytes.

[0121] Note - The st(v) syntax descriptor is used in this disclosure only when the current position in the bitstream is a byte-aligned position.

[0122] - tu(v) is a truncated unary encoding scheme that uses up to the number of bits defined by the maximum value (maxVal), and maxVal is defined in the semantics of the corresponding syntactic element.

[0123] - u(n) is an unsigned integer using n bits. If n is marked as "v" in the syntax table, the number of bits varies depending on the values ​​of other syntax elements. The parsing process of this descriptor is defined by interpreting the return value of the function read_bits(n) as the binary representation of an unsigned integer where the most significant bit is written first.

[0124] - ue(v) is a 0th-order Exp-Golomb encoded unsigned integer syntax element, written starting from the left bit. The parsing process of this descriptor is defined with order k set to 0.

[0125] Below, SEI messages related to the present invention will be described.

[0126] Table 1 shows an example of an Image Format Metadata (IFM) SEI message syntax according to one embodiment.

[0127] [Table 1]

[0128]

[0129] The Image Format Metadata (IFM) SEI message specifies an SEI message in which the payload of EXIF, JFIF, XMP, and ICC profile image format metadata can be transmitted within a video bitstream.

[0130] If the ifm_cancel_flag value is 1, it indicates that the SEI message cancels the persistence of all previous image format metadata SEI messages in the output order. If the ifm_cancel_flag value is 0, it indicates that image format information continues.

[0131] ifm_persistence_flag specifies the persistence of image format metadata SEI messages for the current layer. If the ifm_persistence_flag value is 0, it indicates that image format metadata SEI messages apply only to the currently decoded picture. If the ifm_persistence_flag value is 1, it indicates that IFM SEI messages apply to the currently decoded picture and persist to all subsequent pictures of the current layer in output order until one or more of the following conditions are met.

[0132] - When a new CLVS (Closed Layer Video Sequence) of the current layer starts.

[0133] - When the bitstream ends.

[0134] - When the picture of the current layer within the AU associated with the Exif metadata SEI message is output after the current picture in the output order.

[0135] ifm_num_metadata_payloads indicates the number of following metadata payloads.

[0136] ifm_bit_equal_to_zero must be equal to 0.

[0137] ifm_type_id[ i ] represents the type of the corresponding metadata payload as defined in Table 2 below.

[0138] [Table 2]

[0139]

[0140] If the value of ifm_type_id[ i ] is 0, the following image metadata payload is Exchangeable Image File (Exif) metadata. The Exif format for digital still cameras includes a set of metadata containing information about the digital photo processing used when the image was recorded. This metadata includes the camera acceleration vector at the time the image was taken, camera lens information, GPS data, the color space used, spectral sensitivity, maximum lens aperture (f-number), and various other information.

[0141] Note - Exif metadata is defined by one of the Interchangeable Image File (Exif) format standards for digital still cameras, jointly developed by the Camera & Imaging Products Association (CIPA) and the Japan Electronics and Information Technology Industries Association (JEITA). This metadata is applied to the relevant image source picture prior to encoding (e.g., content captured by a camera). To date, several versions of Exif have been distributed, and each version includes a unique identifier to signal the version number currently in use.

[0142] Note - CIPA or JEITA is not included in the list of organizations that can provide normative references to documents issued by CIPA or JEITA. To add either CIPA or JEITA to the list, the A.4 process must be performed on the organization, and their status as an A.5 organization must be approved by SG16. Additionally, rather than the concept of all standards produced by the organization, it will be necessary to explicitly specify a list of the specific documents being referenced.

[0143] If the value of ifi_type_id[ i ] is 1, 2 or 3, the following metadata payload is JFIF metadata as defined in ITU-T Recommendation T.871 | ISO / IEC International Standard 10918-5.

[0144] Note - JFIF metadata and its semantics are defined by ITU-T Recommendation T.871 | ISO / IEC International Standard 10918-5 (hereinafter, JFIF Standard). Of particular importance is that the JFIF Standard distinguishes between JFIF markers containing information describing the composition of image data and JFIF markers containing extended information.

[0145] If the value of ifm_type_id[ i ] is 1, the following image format information payload contains JFIF metadata excluding the APP0 marker segment identifiers themselves from one or more consecutive JFIF marker segments, as specified in ITU-T Recommendation T.871 | International Standard ISO / IEC 10918-5.

[0146] If the value of ifm_type_id[ i ] is 2, the following image format information payload contains bytes excluding the APP0 marker segment identifier itself from one or more JFIF extended markers, as specified in ITU-T Recommendation T.871 | International Standard ISO / IEC 10918-5.

[0147] If the value of ifm_type_id[ i ] is 3, the following image format information payload contains bytes obtained from the first JFIF marker found after the Start of Image (SOI), as specified in ITU-T Recommendation T.871 | International Standard ISO / IEC 10918-5.

[0148] If the value of ifm_type_id[ i ] is 4, the following image information payload contains Extensible Metadata Platform (XMP) metadata defined in ISO 16684-1 (XML format) or ISO 16684-3 (JSON format).

[0149] Note - Extended Metadata Platform (XMP) metadata is defined by the following standards.

[0150] - ISO 16684-1: Graphic Technology - Extensible Metadata Platform (XMP) Specification

[0151] - ISO 16684-3: Graphic Technology - Extensible Metadata Platform (XMP) Specification

[0152] - Part 3: JSON-LD Serialization of XMP

[0153] XMP is widely used in digital cameras and digital image editing software, and is utilized to record source information and editing history related to images. The most common set of metadata included in XMP corresponds to a vocabulary for storing information such as digital rights ownership and the names of software packages used to modify the image.

[0154] If the value of ifm_type_id[ i ] is 5, the following image information payload provides ICC profile metadata for a receiver using a color management workflow defined by the International Color Consortium (ICC). ICC profile metadata is defined by ISO 15076-1, ISO 15076-2, or ISO 20677.

[0155] If the value of ifm_type_id[ i ] is 5, the value of ifm_payload_len_minus1[ i ] plus 1 must not be less than 11, and the data payload corresponding to the value of ifm_type_id[ i ] is 5 must contain a complete ICC profile that conforms to the referenced ICC profile standard.

[0156] If the value of ifm_type_id[ i ] is 5, ifm_data_payload_byte[ i ][ j ] is a byte containing data that follows the syntax elements and semantics specified in ISO 15076-1, ISO 15076-2, or ISO 20677.

[0157] The variable ICCmajorVer is set to the value of ifm_data_payload_byte[ i ]

[0008] >> 4, and ICCminorVer is set to the value of ifm_data_payload_byte[ i ]

[0009] >> 4. The variables ICCmajorVer and ICCminorVer are interpreted as the major and minor versions of the ICC profile, respectively, and their definitions are specified in Table 3 and Table 4 below, respectively.

[0158] Values ​​of ICCmajorVer and ICCminorVer that are not listed in Tables 3 and 4 are reserved for future use by ITU-T | ISO / IEC. Values ​​not listed in Tables 3 and 4 must not exist in a bitstream suitable for the version of the present disclosure. The decoder must ignore ICC profile SEI messages containing said values.

[0159] [Table 3]

[0160]

[0161] [Table 4]

[0162]

[0163] ifm_uri_present_flag[ i ] signals whether metadata is obtained through a URI having syntax elements and semantics defined in IETF Internet Standard 66.

[0164] - If the value of ifm_uri_present_flag[ i ] is 0, the image information payload is obtained directly from the payload of the SEI message.

[0165] - If the value of ifm_uri_present_flag[ i ] is 1, the image information payload is obtained from the URI.

[0166] The value obtained by adding 1 to ifm_payload_len_minus1[ i ] represents the length of the image information payload determined by ifm_type_id[ i ] as described in Table 2 above.

[0167] ifm_data_payload_byte[ i ][ j ] must be a byte containing data having syntactic elements and semantics according to ifm_type_id[ i ] described in Table 2 above.

[0168] ifm_data_uri[ i ] must contain a URI with the syntax and semantics specified in IETF Internet Standard 66 to identify image information payloads according to ifm_type_id[ i ] described in Table 2.

[0169] The current design of the Image Format Metadata SEI message allows specifying the syntax and semantics of image information based on the image format type signaled by ifm_type_id. The image information payload can be signaled in bytes. It is stipulated that byte alignment must be performed before signaling the image information payload.

[0170] One embodiment provides a solution to the problem described above. Each item may be applied individually or in combination.

[0171] One embodiment adds a signaling that allows byte alignment to be applied before signaling the image format information payload in an image format metadata SEI message.

[0172] One embodiment relates to the above-described item. This embodiment is based on the VSEI specification.

[0173] According to one embodiment, the image format metadata SEI message syntax is as shown in Table 5 below.

[0174] [Table 5]

[0175]

[0176] The terms or names described below (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms, etc. described below. For example, the image information described below may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0177] The operations described below do not constitute an essential component of one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of one embodiment, and previously described operations may be added. Moreover, unless they contradict previously described operations, the operations described below form one embodiment integrally with previously described operations and do not form a separate embodiment distinct from previously described operations.

[0178] FIG. 5 is a diagram illustrating a method for decoding image information according to one embodiment of the present disclosure.

[0179] Terms or names (e.g., names of syntax elements or names of variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms, etc. described in FIG. 5. For example, the image information described in FIG. 5 may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0180] The decoding method (S500) may include operations described below. The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the decoding method according to one embodiment, and the previously described operations may be added. Moreover, unless the operations described below contradict the previously described operations, they form an embodiment integrally with the previously described operations and do not form a separate embodiment distinguished from the previously described operations.

[0181] The decoding method (S500) can be executed by a decoding device including a memory and a processor electrically connected to the memory, for example, by a processor.

[0182] The decoding device can acquire image information.

[0183] For example, a processor of a decoding device may acquire image information. The image information may include at least one (or multiple) main layer among multiple layers. Each of the at least one (or multiple) main layer may include a picture to be decoded. Here, the main layer is not limited to its name and may be referred to in various ways, such as a main picture layer.

[0184] The image information may further include at least one auxiliary layer associated with the current main layer among at least one (or multiple) main layers. Here, the auxiliary layer is not limited to its name and may be referred to in various ways, such as an auxiliary picture layer.

[0185] For example, at least one auxiliary layer can provide additional information (e.g., depth or alpha) of the main layer.

[0186] Additionally, for example, at least one auxiliary layer may include additional metadata such as a color profile corresponding to the picture of the main layer, capture parameters (EXIF information), or rendering characteristics (XMP information). The auxiliary layer is not limited to a specific name and may be referred to in various ways, such as an auxiliary picture layer.

[0187] The encoding device analyzes information contained in the main and auxiliary layers to generate image format metadata corresponding to each picture and can encode it in the form of SEI messages. This allows the main video data and the corresponding video format characteristics to be transmitted simultaneously within the bitstream, enabling optimal restoration and color representation based on the format characteristics during decoding.

[0188] Since the decoding device receives image format metadata generated by considering not only the picture information of the main layer but also the additional information of the associated auxiliary layer from the encoding device, it can restore the format information of the image without a separate post-processing process.

[0189] The decoding device can acquire a SEI (supplemental enhancement information) message (S510).

[0190] For example, the processor of the decoding device can acquire an SEI message.

[0191] SEI messages may convey specific types of information that assist in processes related to the decoding, display, or other purposes of image information. Here, SEI messages may not be necessary for the decoding process to determine the sample values ​​of the decoded picture.

[0192] According to one embodiment, the decoding device can obtain at least one SEI message associated with the current picture from the bitstream. For example, the processor of the decoding device can obtain at least one SEI message associated with the current picture.

[0193] Here, the SEI message may include an IFM SEI message that provides image format metadata associated with the current picture. Specifically, the SEI message may include an IFM SEI message that provides at least one (or multiple) image format metadata associated with each of at least one (or multiple) key layers. The IFM SEI message may include image format metadata associated with each of at least one (or multiple) key layers.

[0194] At least one (or multiple) IFM SEI messages may each be associated with at least one (or multiple) key layers. In other words, one of the at least one (or multiple) IFM SEI messages may be associated with the current key layer among the at least one (or multiple) key layers. Additionally, one IFM SEI message may exist in the current key layer.

[0195] IFM SEI messages can take various forms. For example, an IFM SEI message may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, an IFM SEI message may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. For example, an IFM SEI message may be represented as image_format_metadata(payloadSize), but is not limited thereto.

[0196] IFM SEI messages may have various names, such as IFM messages, IFM messages, IFM-related messages, and IFM-related information, and such names are not limited.

[0197] The decoding device can obtain image format metadata based on the SEI message (S520).

[0198] For example, the processor of the decoding device can process IFM SEI messages. The decoding device can obtain image format metadata based on processing the IFM SEI messages.

[0199] According to one embodiment, the image format metadata provided by the IFM SEI message may include information on whether a URI for the image format metadata exists, payload information for the image format metadata, and a zero bit for byte alignment within the SEI message.

[0200] The following describes the process by which a decoding device obtains image format metadata based on IFM SEI messages.

[0201] According to one embodiment, the decoding device can obtain information on whether a URI for image format metadata exists from an IFM SEI message.

[0202] Information regarding the existence of a URI for image format metadata may be a flag that signals whether metadata for image format data is obtained through a URI having syntax and semantics defined in IETF Internet Standard 66.

[0203] Information regarding the existence of a URI for image format metadata may take various forms and be represented by various names. For example, information regarding the existence of a URI for image format metadata may be a syntax element or a syntax structure containing one or more syntax elements. For example, information regarding the existence of a URI for image format metadata that is a syntax element may be represented as the syntax element ifm_uri_present_flag[ i ], but is not limited thereto.

[0204] Information regarding the existence of a URI for image format metadata may be a 1-bit flag or an indicator of 2 bits or more. For example, if information regarding the existence of a URI for image format metadata is represented by ifm_uri_present_flag[ i ]., ifm_uri_present_flag[ i ]. may indicate an existence flag that selects the delivery method of the i-th image format metadata. Here, index i may indicate an identifier that distinguishes the image format metadata. Information regarding the existence of a URI for image format metadata may indicate a 1-bit flag that distinguishes whether to directly include the image format metadata as bytes in the payload format within the bitstream or to reference it via a URI of an external resource. In other words, by referencing large amounts of image format metadata externally, the bitstream size can be reduced, and the reuse and version management of the same metadata can be facilitated. For example, image format metadata may include EXIF, JFIF, XMP, ICC, etc., but is not limited thereto.

[0205] In one embodiment, the decoding device can obtain a zero bit for byte alignment within an IFM SEI message based on information regarding the existence of a URI for the acquired image format metadata.

[0206] If the information regarding the existence of a URI for image format metadata indicates that a URI for image format metadata does not exist, the payload information for image format metadata can be obtained after obtaining a zero bit for byte alignment within the IFM SEI message. That is, if the information regarding the existence of a URI for image format metadata indicates that a URI for image format metadata does not exist, the decoding device can first obtain a zero bit for byte alignment within the IFM SEI message before obtaining the payload information for image format metadata.

[0207] For example, if the value of the information on whether a URI exists for image format metadata is 1, the corresponding image information payload can be obtained via the URI. In this case, it may indicate that URI data referencing external metadata exists, but payload information regarding metadata does not exist. Additionally, if the value of the information on whether a URI exists for image format metadata is 0, the corresponding image information payload can be obtained directly from the payload of the IFM SEI message. In this case, it may indicate that payload information regarding metadata exists, but URI data referencing external metadata does not exist. However, this is not limited thereto, and alternatively, specifying that the value of the information on whether a URI exists for image format metadata is 1 may be replaced with specifying that the value of the information on whether a URI exists for image format metadata is 0.

[0208] The zero bit for byte alignment within the SEI message may be a padding bit. The zero bit may be a specific bit inserted to align byte boundaries in 8-bit units. The zero bit for byte alignment within the SEI message may be set to a value of 0. The zero bit may be a single bit. For example, the zero bit for byte alignment within the SEI message may be the zero bit for byte alignment within the IFM SEI message.

[0209] The zero bit for byte alignment within an SEI message can take various forms and be represented by various names. For example, the zero bit for byte alignment within an SEI message may be a syntax element or a syntax structure containing one or more syntax elements. For example, the zero bit for byte alignment within an IFM SEI message, which is a syntax element, may be represented as the syntax element ifm_bit_equal_to_zero, but is not limited thereto.

[0210] In one embodiment, if information regarding the existence of a URI for image format metadata indicates that the URI for image format metadata does not exist, and the current bit position within the IFM SEI message is not aligned to a byte position, the decoding device may perform byte alignment based on the zero bit. Subsequently, the decoding device may obtain payload information for the image format metadata. Through this alignment process, consistency of the syntactic structure in which the payload information for the image format metadata is subsequently parsed in byte units is guaranteed, and the parsing stability of the bitstream can be improved.

[0211] In one embodiment, if the information regarding the existence of a URI for image format metadata indicates that a URI for image format metadata exists, the URI information for image format metadata can be obtained after acquiring a zero bit for byte alignment within the IFM SEI message. That is, if the information regarding the existence of a URI for image format metadata indicates that a URI for image format metadata exists, the decoding device can obtain the URI information for image format metadata after acquiring the zero bit. Accordingly, by performing byte alignment regardless of whether a URI for image format metadata exists, the conformance of the bitstream can be maintained, and data interpretation errors or parsing errors caused by bit boundary mismatches can be prevented.

[0212] For example, if the value of the information on whether a URI exists for image format metadata is 0, the decoding device may obtain a zero bit for byte alignment within the IFM SEI message and then obtain payload information for image format metadata. As another example, if the value of the information on whether a URI exists for image format metadata is 1, the decoding device may obtain a zero bit for byte alignment within the IFM SEI message and then obtain URI information for image format metadata instead of payload information for image format metadata. However, this is not limited thereto, and alternatively, specifying that the value of the information on whether a URI exists for image format metadata is 1 may be changed to specifying that the value of the information on whether a URI exists for image format metadata is 0.

[0213] According to one embodiment, the payload information for image format metadata and the URI information for image format metadata may have different data formats. For example, the payload information for image format metadata may be represented in a fixed-length data format (e.g., b(8) descriptor) consisting of bytes having an arbitrary bit string pattern of length of 8 bits. Alternatively, the URI information for image format metadata may be represented in a variable-length string format (e.g., st(v) descriptor) that starts at a byte-aligned position within the bitstream and parses consecutive bytes up to a byte with a value of 0x00 (null) into a string. However, the data formats described above are merely exemplary, and the payload information for image format metadata or the URI information for image format metadata may be implemented in a different format, and such variations are also included within the scope of the present invention.

[0214] The method of processing length information may vary depending on the data format of the above information. For example, in the case of the b(8) format, the decoding device can recognize the total number of bytes of the payload using explicit length information, such as the payload length of the image format metadata. Therefore, the decoding device determines the end point of the data by the length field rather than a separate end symbol, and the data interpretation is not affected even if the payload contains a 0x00 value.

[0215] On the other hand, the st(v) format does not indicate the end of the data with explicit length information, so the decoding device can distinguish the data through null termination. That is, the decoding device identifies the end of the string by searching for a termination byte with a value of 0x00, and due to this structure, a parsing method based on searching for a termination byte instead of length information can be applied.

[0216] For example, when the payload information for image format metadata has the data format of b(8), the payload information for image format metadata is processed in fixed 8-bit units, so strict alignment of bit boundaries is not required. On the other hand, when the URI information for image format metadata has the data format of st(v), the URI information for image format metadata must start at a byte-aligned position, and if there is a bit offset in the middle, parsing of the string is impossible. This is because the st(v) format has a structural characteristic in which the UTF-8 encoding method is defined in byte units.

[0217] According to the present disclosure, consistency in bit parsing can be maintained in both cases where payload information regarding image format metadata is directly included within the bitstream and cases where payload information does not exist but is referenced via an external URI. That is, by considering structural differences between data formats and commonly performing byte alignment using zero bits within an IFM SEI message, the present disclosure prevents data parsing errors caused by bit boundary inconsistencies and maintains syntactic consistency of the bitstream, not only when payload information is directly included but also when metadata is obtained via an external URI reference.

[0218] Accordingly, according to the present disclosure, the same parsing structure can be applied in the decoder regardless of the form in which image format metadata exists, thereby improving the conformance and interoperability of the bitstream.

[0219] FIG. 6 is a diagram illustrating a method for encoding image information according to one embodiment of the present disclosure.

[0220] The terms or names described in FIG. 6 (e.g., names of syntax elements or variables, etc.) are merely examples, and the technical features of the present disclosure are not limited to the terms, etc. described in FIG. 6. For example, the image information described in FIG. 6 may include various information according to the embodiments described in the present disclosure and may include information described in at least one of the tables described above.

[0221] The encoding method (S600) may include operations described below. The operations described below do not constitute an essential component of the decoding method according to one embodiment, and at least some of the operations described below may be omitted. Furthermore, the operations described below do not constitute a sufficient component of the encoding method according to one embodiment, and the previously described operations may be added. Moreover, unless the operations described below contradict the previously described operations, they form an embodiment integrally with the previously described operations and do not form a separate embodiment distinct from the previously described operations.

[0222] The encoding device can generate a SEI (supplemental enhancement information) message (S610).

[0223] For example, the processor of the encoding device can generate an SEI message to convey additional characteristics of the image information.

[0224] Here, the SEI message may include an IFM SEI message that provides image format metadata associated with the current picture. Specifically, the SEI message may include an IFM SEI message that provides at least one (or multiple) image format metadata associated with each of at least one (or multiple) key layers. The IFM SEI message may include image format metadata associated with each of at least one (or multiple) key layers.

[0225] In particular, at least one (or multiple) IFM SEI messages may each be associated with at least one (or multiple) major layers. In other words, one of the at least one (or multiple) IFM SEI messages may be associated with the current major layer among the at least one (or multiple) major layers. Additionally, one IFM SEI message may exist in the current major layer.

[0226] An encoding device can generate image format metadata based on a main layer and an auxiliary layer associated with the main layer. Here, the main layer may include a picture to be decoded. The main layer is not limited to a specific name and may be referred to in various ways, such as the main picture layer. Additionally, the auxiliary layer may include supplementary information of the main layer. For example, the auxiliary layer may include supplementary metadata such as a color profile corresponding to the main picture, capture parameters (EXIF information), or rendering characteristics (XMP information). The auxiliary layer is not limited to a specific name and may be referred to in various ways, such as the auxiliary picture layer.

[0227] The encoding device analyzes information contained in the main layer and auxiliary layer to generate image format metadata corresponding to each picture, and based on this, can generate at least one IFM (Image Format Metadata) SEI message associated with the current picture. The generated IFM SEI message can be transmitted within the same bitstream along with the main image data, and can enable optimal restoration and color representation according to format characteristics during decoding.

[0228] IFM SEI messages can take various forms. For example, an IFM SEI message may be a syntax element or a syntax structure containing one or more syntax elements. Additionally, an IFM SEI message may be a raw byte sequence payload (RBSP) containing one or more syntax elements or one or more syntax structures. For example, an IFM SEI message may be represented as image_format_metadata(payloadSize), but is not limited thereto.

[0229] IFM SEI messages may have various names, such as IFM messages, IFM messages, IFM-related messages, and IFM-related information, and such names are not limited.

[0230] The image format metadata provided by the IFM SEI message may include information on whether a URI for the image format metadata exists, payload information for the image format metadata, and zero bits for byte alignment within the SEI message.

[0231] The following describes the process by which an encoding device generates an IFM SEI message based on image format metadata.

[0232] According to one embodiment, the encoding device can encode payload information for image format metadata. The encoding device can generate and encode information regarding the existence of a URI for the image format metadata.

[0233] Information regarding the existence of a URI for image format metadata may be a flag that signals whether metadata for image format data is obtained through a URI having the syntax elements and semantics specified in IETF Internet Standard 66.

[0234] Information regarding the existence of a URI for image format metadata may take various forms and be represented by various names. For example, information regarding the existence of a URI for image format metadata may be a syntax element or a syntax structure containing one or more syntax elements. For example, information regarding the existence of a URI for image format metadata that is a syntax element may be represented as the syntax element ifm_uri_present_flag[ i ], but is not limited thereto.

[0235] Information regarding the existence of a URI for image format metadata may be a 1-bit flag or an indicator of 2 bits or more. For example, if information regarding the existence of a URI for image format metadata is represented by ifm_uri_present_flag[ i ]., ifm_uri_present_flag[ i ]. may indicate an existence flag that selects the delivery method of the i-th image format metadata. Here, index i may indicate an identifier that distinguishes the image format metadata. Information regarding the existence of a URI for image format metadata may indicate a 1-bit flag that distinguishes whether to directly include the image format metadata as bytes in the payload format within the bitstream or to reference it via a URI of an external resource. In other words, by referencing large amounts of image format metadata externally, the bitstream size can be reduced, and the reuse and version management of the same metadata can be facilitated. For example, image format metadata may include EXIF, JFIF, XMP, ICC, etc., but is not limited thereto.

[0236] For example, if the value of the information on whether a URI exists for image format metadata is 1, the corresponding image information payload can be obtained via the URI. In this case, it may indicate that URI data referencing external metadata exists, but payload information regarding metadata does not exist. Additionally, if the value of the information on whether a URI exists for image format metadata is 0, the corresponding image information payload can be obtained directly from the payload of the SEI message. In this case, it may indicate that payload information regarding metadata exists, but URI data referencing external metadata does not exist. However, this is not limited thereto, and alternatively, specifying that the value of the information on whether a URI exists for image format metadata is 1 may be replaced with specifying that the value of the information on whether a URI exists for image format metadata is 0.

[0237] In one embodiment, the encoding device can insert a zero bit for byte alignment within an IFM SEI message based on image format metadata.

[0238] If a URI for image format metadata does not exist and the current bit position within the IFM SEI message is not aligned to a byte position, the payload information for the image format metadata can be encoded after inserting a zero bit for byte alignment within the IFM SEI message. That is, if a URI for image format metadata does not exist and the current bit position within the IFM SEI message is not aligned to a byte position, the encoding device may first insert a zero bit for byte alignment within the SEI message before encoding the payload information for the image format metadata. Through this process, consistency of the syntactic structure can be ensured so that the payload information of the image format metadata can be reliably parsed byte by byte by a subsequent decoding device.

[0239] The zero bit for byte alignment within the SEI message may be a padding bit. The zero bit may be a specific bit inserted to align byte boundaries in 8-bit units. The zero bit for byte alignment within the SEI message may be set to a value of 0. Here, the zero bit may be a single bit. For example, the zero bit for byte alignment within the SEI message may be the zero bit for byte alignment within the IFM SEI message.

[0240] The zero bit for byte alignment within an SEI message can take various forms and be represented by various names. For example, the zero bit for byte alignment within an SEI message may be a syntax element or a syntax structure containing one or more syntax elements. For example, the zero bit for byte alignment within an IFM SEI message, which is a syntax element, may be represented as the syntax element ifm_bit_equal_to_zero, but is not limited thereto.

[0241] In one embodiment, if a URI for image format metadata exists, the URI information for the image format metadata can be encoded after inserting a zero bit for byte alignment within the IFM SEI message. That is, if a URI for image format metadata exists, the encoding device can encode the URI information for the image format metadata after inserting a zero bit. Accordingly, regardless of whether a URI for image format metadata exists, consistency of the syntactic structure of the bitstream can be ensured by aligning by inserting a zero bit on a byte-by-byte basis, and data interpretation errors or parsing errors caused by bit boundary mismatches can be prevented.

[0242] According to one embodiment, the payload information for image format metadata and the URI information for image format metadata may have different data formats. For example, the payload information for image format metadata may be encoded in a fixed-length data format (e.g., b(8) descriptor) consisting of bytes having an arbitrary bit string pattern of length of 8 bits. Alternatively, the URI information for image format metadata may be encoded in a variable-length string format (e.g., st(v) descriptor) that starts at a byte-aligned position within the bitstream and parses consecutive bytes up to a byte with a value of 0x00 (null) into a string. However, the data formats described above are merely exemplary, and the payload information for image format metadata or the URI information for image format metadata may be implemented in a different format, and such variations are also included within the scope of the present invention.

[0243] The method of processing length information may vary depending on the data format of the above information. For example, in the case of the b(8) format, the encoding device can recognize and encode the total number of bytes of the payload using explicit length information, such as the payload length of the image format metadata. Therefore, the encoding device determines the end point of the data by the length field rather than a separate end symbol, and does not affect the encoding process even if the payload contains a 0x00 value.

[0244] On the other hand, in the case of the st(v) format, the encoding device can indicate the end of the string by applying a null termination method instead of indicating the end of the data with explicit length information. That is, the encoding device specifies the end of the data by inserting a termination byte with a value of 0x00 at the end of the string data, thereby enabling a string parsing and encoding structure based on termination bytes instead of length information.

[0245] For example, if the payload information of image format metadata generated by the encoding device has the b(8) data format, the payload information is processed in fixed 8-bit units, so strict alignment of bit boundaries is not required. On the other hand, if the URI information of image format metadata has the st(v) data format, the URI information must start at a byte-aligned position, and if there is a bit offset in the middle, accurate encoding of the string is impossible. This is because the st(v) format has structural characteristics defined in byte units according to the UTF-8 encoding method.

[0246] According to the present disclosure, an encoding device can encode image format metadata to maintain consistency of the bit parsing structure for both cases where the payload information of the image format metadata is included in a bitstream or referenced via an external URI.

[0247] The encoding device can encode video information (S620).

[0248] For example, the processor of an encoding device can encode image information including a main layer, an auxiliary layer associated with the main layer, and an SEI message.

[0249] An encoding device can encode an image or image information. Here, the image may include a still image or a video, and the image information may refer to pixel data of the image, object information, additional information (e.g., SEI messages, etc.), or a combination thereof.

[0250] An encoding device can generate a bitstream by performing an encoding process on an input image or image information. At this time, the encoding process may include one or more steps such as prediction, transformation, quantization, and entropy encoding, and the generated bitstream can be used as data for reconstructing the image or image information by a decoding device.

[0251] Additionally, the encoding device according to the present disclosure may encode an Image Format Metadata (IFM) supplemental enhancement information (SEI) message for transmitting format characteristics of an image or additional metadata information. The IFM SEI message may include image format metadata of each picture, which may include format-related information such as color profiles, shooting parameters, and rendering characteristics, as well as information on the existence of a URI for the image format metadata, payload information, and alignment bits for byte alignment.

[0252] Accordingly, the present disclosure may be applied not only to general encoding structures for encoding images or image information, but also to encoding methods of IFM SEI messages including image format metadata and related additional information.

[0253] According to one embodiment, image information may include an SEI message. The SEI message may convey a specific type of information that assists in processes related to the decoding, display, or other purposes of the image information. Here, the SEI message may include an IFM SEI message that provides at least one (or multiple) image format metadata associated with at least one (or multiple) key layers. Specifically, the IFM SEI message may include image format metadata associated with at least one (or multiple) key layers.

[0254] In particular, at least one (or multiple) IFM SEI messages may each be associated with at least one (or multiple) major layers. In other words, one of the at least one (or multiple) IFM SEI messages may be associated with the current major layer among the at least one (or multiple) major layers. Additionally, one IFM SEI message may exist in the current major layer.

[0255] In this way, the processor of the encoding device can encode image information including a main layer, an auxiliary layer associated with the main layer, and image format metadata.

[0256] According to the present disclosure, an encoding device can encode to maintain consistency of the bit parsing structure not only when the payload information of image format metadata is directly included in the bitstream, but also when the payload information does not exist and is referenced via an external URI.

[0257] That is, by considering structural differences between data formats and commonly performing byte alignment using zero bits within SEI messages, the encoding device can prevent encoding errors caused by bit boundary mismatches and ensure syntactic consistency of the bitstream in both cases where payload information is directly included in the bitstream and where metadata is provided through external URI references.

[0258] Accordingly, the encoding device according to the present disclosure can apply the same encoding structure regardless of the form in which image format metadata exists, thereby improving the specification compliance of the generated bitstream and interoperability with the decoder.

[0259] A bitstream is generated based on video information encoded according to the encoding method (S600) described above, and the bitstream can be stored on a computer-readable storage medium.

[0260] In addition, a bitstream is generated based on video information encoded according to the encoding method (S600) described above, and the bitstream can be transmitted through a transmission unit and / or a transmission medium.

[0261] FIG. 7 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0262] As illustrated in FIG. 7, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0263] The above encoding server compresses content input from multimedia input devices, such as smartphones, cameras, and camcorders, into digital data to generate a bitstream and transmits it to the streaming server. As another example, if multimedia input devices, such as smartphones, cameras, and camcorders, generate the bitstream directly, the encoding server may be omitted.

[0264] The bitstream may be generated by a video encoding method and / or encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0265] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user requests a desired service from the web server, the web server transmits it to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0266] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0267] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0268] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0269] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0270] An embodiment according to the present disclosure can be used to encode / decode images.

Claims

1. In a video decoding method performed by a decoding device, A step of obtaining at least one IFM (Image Format Metadata) SEI (supplemental enhancement information) message associated with the current picture from a bitstream; and The method includes the step of obtaining image format metadata based on at least one IFM SEI message, and The above image format metadata is, An image decoding method characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

2. In Claim 1, The step of acquiring the above image format metadata is, A step of obtaining information on whether a URI for the image format metadata exists from the above IFM SEI message; A step of obtaining a zero bit for byte alignment within the IFM SEI message based on information regarding the existence of a URI for the image format metadata; and A video decoding method further comprising the step of obtaining payload information for the image format metadata above.

3. In Claim 2, A video decoding method characterized in that, when information on whether a URI exists for the image format metadata indicates that a URI for the image format metadata does not exist, the payload information for the image format metadata is obtained after obtaining the zero bit for byte alignment within the IFM SEI message.

4. In Claim 2, An image decoding method characterized by performing byte alignment based on the zero bit when information regarding the existence of a URI for the image format metadata indicates that the URI for the image format metadata does not exist, and when the current bit position within the IFM SEI message is not aligned to a byte unit position.

5. In Claim 2, A video decoding method characterized in that, when information on whether a URI exists for the image format metadata indicates that a URI for the image format metadata exists, the URI information for the image format metadata is obtained after obtaining the zero bit for byte alignment within the IFM SEI message.

6. In Claim 5, An image decoding method characterized in that the payload information for the image format metadata and the URI information for the image format metadata have different data formats.

7. In Claim 1, An image decoding method characterized in that the above zero bit is a single bit with a value set to 0.

8. In a video encoding method performed by an encoding device, A step of generating at least one IFM (Image Format Metadata) SEI (supplemental enhancement information) message associated with the current picture based on image format metadata; and The method includes the step of encoding image information including at least one IFM SEI message, and The above image format metadata is, A video encoding method characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

9. In Claim 8, The step of generating the above IFM SEI message is, A step of encoding payload information for the image format metadata above; A step of generating and encoding information regarding the existence of a URI for the image format metadata; and A video encoding method further comprising the step of inserting a zero bit for byte alignment within the IFM SEI message based on the image format metadata.

10. In Claim 9, A video encoding method characterized by the fact that, when a URI for the image format metadata does not exist and the current bit position within the IFM SEI message is not aligned to a byte unit position, the payload information for the image format metadata is encoded after inserting the zero bit for byte alignment within the IFM SEI message.

11. In Claim 9, A video encoding method characterized by, if a URI for the image format metadata exists, the URI information for the image format metadata is encoded after inserting the zero bit for byte alignment within the IFM SEI message.

12. In Claim 11, A video encoding method characterized in that the payload information for the image format metadata and the URI information for the image format metadata have different data formats.

13. In Claim 8, A video encoding method characterized in that the above zero bit is a single bit with a value set to 0.

14. In a computer-readable storage medium for storing a bitstream, At least one IFM (Image Format Metadata) SEI (supplemental enhancement information) message associated with the current picture is generated based on image format metadata, and a bitstream generated based on image information including said at least one IFM SEI message is stored in a computer-readable storage medium. The above image format metadata is, A storage medium characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

15. A method for transmitting data regarding an image, wherein the method comprises the step of acquiring image information, wherein the image information includes at least one IFM (Image Format Metadata) SEI (supplemental enhancement information) message associated with a current picture; and The method includes the step of transmitting the data including the above image information, Image format metadata is, A bitstream transmission method characterized by including information on whether a URI exists for the image format metadata, payload information for the image format metadata, and a zero bit for byte alignment within the IFM SEI message.

Images