Image encoding / decoding method and apparatus and recording medium storing bitstream

Abstract

The image encoding method and apparatus of this disclosure can receive a video frame to be encoded, encode the received video frame to generate a compressed video frame, generate encoder optimization information related to the compressed video frame, and generate a bitstream including the compressed video frame and the encoder optimization information.

Description

Technical Field

[0001] This disclosure relates to an image encoding / decoding method and apparatus, as well as a recording medium for storing bit streams. Background Technology

[0002] Recently, the demand for high-resolution and high-quality images, such as HD (high-definition) and UHD (ultra-high-definition) images, has been increasing in various application areas, and therefore, efficient image compression technologies are being discussed.

[0003] Various techniques exist, such as inter-frame prediction techniques that use video compression technology to predict pixel values ​​included in the current frame from frames before or after the current frame, intra-frame prediction techniques that use pixel information in the current frame to predict pixel values ​​included in the current frame, and entropy coding techniques that assign short codes to values ​​that occur frequently and long codes to values ​​that occur infrequently. These image compression techniques can be used to effectively compress image data and send or store it. Summary of the Invention

[0004] Technical issues

[0005] This disclosure provides methods and apparatus for configuring encoder optimization information.

[0006] This disclosure provides a method and apparatus for signaling encoder optimization information.

[0007] Technical solution

[0008] The image encoding method and apparatus of this disclosure can receive a video frame to be encoded, encode the received video frame to generate a compressed video frame, generate encoder optimization information related to the compressed video frame, and generate a bitstream including the compressed video frame and the encoder optimization information. Here, the encoder optimization information may include bit depth optimization related information. The image encoding method and apparatus of this disclosure...

[0009] In the image encoding method and apparatus according to this disclosure, bit depth optimization-related information may include information related to the shift size of the input sample.

[0010] In the image coding method and apparatus according to this disclosure, bit depth optimization related information may include at least one of information related to the input bit depth, information related to the optimized bit depth, information related to the bit depth difference, or information related to the task bit depth.

[0011] In the image encoding method and apparatus according to this disclosure, bit depth optimization-related information may include information for identifying the type of bit depth optimization.

[0012] In the image coding method and apparatus according to this disclosure, bit depth optimization-related information can be generated based on a spatial resampling flag indicating whether spatial resampling optimization is applied.

[0013] In the image coding method and apparatus according to this disclosure, bit depth optimization-related information can be generated based on a flag indicating whether spatial quality optimization is applied.

[0014] In the image coding method and apparatus according to this disclosure, bit depth optimization-related information can be generated based on a flag indicating whether bit depth optimization is applied.

[0015] In the image encoding method and apparatus according to this disclosure, bit depth optimization-related information can be generated based on a flag indicating whether bit depth truncation is applied.

[0016] In the image encoding method and apparatus according to this disclosure, bit depth optimization-related information may be generated based on a bit depth optimization identifier used to identify bit depth optimization.

[0017] In the image encoding method and apparatus according to this disclosure, the bit depth optimization identifier may be a spatial resampling type identifier used to identify the type of spatial resampling applied.

[0018] In the image encoding method and apparatus according to this disclosure, the bit depth optimization identifier may be a quality optimization type identifier used to identify the type of spatial quality optimization applied.

[0019] The image decoding method and apparatus according to this disclosure can receive a bitstream including encoded video frames and reconstruct the encoded video frames included in the bitstream. Here, the bitstream may include encoder optimization information related to the encoded video frames, and the encoder optimization information may include bit depth optimization related information.

[0020] A computer-readable digital storage medium is provided for storing encoded video / image information, which enables a decoding device according to the present disclosure to perform an image decoding method.

[0021] A computer-readable digital storage medium is provided for storing video / image information generated based on the image encoding method of this disclosure.

[0022] A method and apparatus are provided for transmitting video / image information generated according to the image encoding method of this disclosure.

[0023] Beneficial effects

[0024] According to this disclosure, the bit depth optimization characteristics of an application can be identified by configuring bit depth optimization-related information into the encoder optimization information.

[0025] According to this disclosure, detailed information related to bit depth optimization can be efficiently communicated using signals. Attached Figure Description

[0026] Figure 1 A video / image encoding system according to this disclosure is shown.

[0027] Figure 2 A schematic block diagram of an encoding apparatus to which embodiments of the present disclosure are applicable and which performs encoding of video / image signals is shown.

[0028] Figure 3 A schematic block diagram of a decoding apparatus to which embodiments of the present disclosure are applicable and which performs decoding of video / image signals is shown.

[0029] Figure 4 A method for generating a bit stream, performed by an encoding device 200 according to the present disclosure, is illustrated.

[0030] Figure 5 An illustrative configuration of an encoding device 200 for performing a method for generating a bitstream according to the present disclosure is shown.

[0031] Figure 6 An example is illustrated of a method for reconstructing a video frame performed by a decoding device 300 according to the present disclosure.

[0032] Figure 7 An illustrative configuration of a decoding device 300 for performing a method for reconstructing a video frame according to the present disclosure is shown.

[0033] Figure 8 Examples of content streaming systems to which embodiments of this disclosure can be applied are shown. Detailed Implementation

[0034] Because this disclosure can be modified in various ways and has multiple embodiments, specific embodiments will be shown in the accompanying drawings and described in detail in the specific embodiments. However, this disclosure is not intended to be limited to the specific embodiments, but should be understood to include all changes, equivalents, and substitutions included within the spirit and scope of this disclosure. Similar reference numerals are used for similar components in the description of the various figures.

[0035] Terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. Terms are used only to distinguish one component from others. For example, without departing from the scope of this disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. Terms include any combination of one or more of the associated terms.

[0036] When a component is described as "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to the other component, but the other component can exist in between. On the other hand, when a component is described as "directly connected" or "directly linked" to another component, it should be understood that there is no other component in between.

[0037] The terminology used in this application is for describing particular embodiments only and is not intended to limit this disclosure. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, it should be understood that terms such as “comprising” or “having” are intended to specify the presence of the features, quantities, steps, operations, components, portions, or combinations thereof described in this specification, but do not preclude the possibility of the presence or addition of one or more other features, quantities, steps, operations, components, portions, or combinations thereof.

[0038] This disclosure relates to video / image coding. For example, the methods / implementations disclosed herein can be applied to methods disclosed in the Multifunctional Video Coding (VVC) standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the Audio Video Coding 2 (AVS2) standard, or next-generation video / image coding standards (e.g., H.267 or H.268).

[0039] This specification sets forth various implementations of video / image encoding, and unless otherwise specified, these implementations may be combined with each other.

[0040] In this article, video can refer to a collection of images over time. A frame typically refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms part of a frame during encoding. A slice / tile can include at least one Code Tree Unit (CTU). A frame can consist of at least one slice / tile. A tile is a rectangular area composed of multiple CTUs within a specific tile column and a specific tile row of a frame. A tile column is a rectangular area of ​​CTUs with a height equal to the height of the frame and a width specified by the syntax requirements of the frame parameter set. A tile row is a rectangular area of ​​CTUs with a height specified by the frame parameter set and a width equal to the width of the frame. CTUs within a tile can be arranged continuously according to CTU raster scans, and tiles within a frame can be arranged continuously according to tile raster scans. A slice can include an integer number of complete tiles of a frame that can be exclusively included in a single NAL unit, or an integer number of consecutive complete CTU rows within a tile. Furthermore, a frame can be divided into at least two sub-frames. A sub-frame can be a rectangular area of ​​at least one slice within a frame.

[0041] A pixel, or pelin, can represent the smallest unit that makes up a frame (or image). Additionally, "sample" can be used as the term corresponding to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0042] A unit can represent the basic unit of image processing. A unit may include a specific region of the image and at least one of the information associated with that region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". In general, an M×N block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0043] In this document, “A or B” can mean “A only”, “B only”, or “both A and B”. In other words, “A or B” can be interpreted as “A and / or B”. For example, “A, B or C” can mean “A only”, “B only”, “C only”, or “any combination of A, B and C”.

[0044] The forward slash ( / ) or comma used in this article can indicate "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0045] In this document, "at least one of A and B" can mean "only A", "only B" or "both A and B". Furthermore, in this document, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same way as "at least one of A and B".

[0046] Additionally, in this document, "at least one of A, B, and C" can mean "A only", "B only", "C only" or "any combination of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" can mean "at least one of A, B, and C".

[0047] Additionally, the parentheses used in this document can indicate "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" in this document is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when the indication is "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0048] In this article, the technical features described individually in a single diagram can be implemented individually or simultaneously.

[0049] Figure 1 A video / image encoding system according to this disclosure is shown.

[0050] Reference Figure 1 A video / image encoding system may include a first device (source device) and a second device (receiving device).

[0051] A source device can transmit encoded video / image information or data to a receiving device in the form of a file or stream via a digital storage medium or network. The source device may include a video source, an encoding device, and a transmitting unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may consist of a separate device or external components.

[0052] A video source can acquire video / images through processes that capture, synthesize, or generate video / images. A video source may include means for capturing video / images and means for generating video / images. Means for capturing video / images may include at least one camera, a video / image archive containing previously captured video / images, etc. Means for generating video / images may include a computer, tablet computer, smartphone, etc., and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc., and in this case, the process of capturing video / images can be replaced by a process of generating related data.

[0053] Encoding devices can encode input video / images. They can perform a series of processes such as prediction, transformation, and quantization for compression and encoding efficiency. The encoded data (encoded video / image information) can be output as a bitstream.

[0054] The transmitting unit can send encoded video / image information or data, output in bitstream form, to the receiving unit of the receiving device in the form of a file or stream via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit can include elements for generating media files according to a predetermined file format, and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and send it to a decoding device.

[0055] Decoding devices can decode video / images by performing a series of processes, such as dequantization, inverse transform, and prediction, that correspond to the operations of encoding devices.

[0056] The renderer can render decoded video / images. The rendered video / images can be displayed through a display unit.

[0057] Figure 2 A schematic block diagram of an encoding apparatus that can be applied to embodiments of the present disclosure and perform encoding of video / image signals is shown.

[0058] Reference Figure 2 The encoding device 200 may consist of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). Additionally, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0059] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into at least one processing unit. As an example, a processing unit can be referred to as a coding unit (CU). In this case, the coding unit can be recursively segmented from coding tree unit (CTU) or maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure.

[0060] For example, a coding unit can be segmented into multiple deeper coding units based on a quadtree, binary tree, and / or ternary tree structure. In this case, for example, a quadtree structure can be applied first, followed by a binary tree and / or ternary tree structure. Alternatively, a binary tree structure can be applied before the quadtree structure. The coding process according to this specification can be performed based on the final coding unit that is no longer segmented. In this case, based on image characteristics, coding efficiency, etc., the largest coding unit can be directly used as the final coding unit, or, if necessary, the coding unit can be recursively segmented into deeper coding units, and the coding unit with the optimal size can be used as the final coding unit. Here, the coding process can include processes such as prediction, transformation, and reconstruction, as described later.

[0061] As another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be divided or segmented from the aforementioned final encoding unit, respectively. The prediction unit may be a unit for predicting samples, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0062] In some cases, a unit can be used interchangeably with terms such as block or region. Generally, an M×N block can represent a set of transform coefficients or samples consisting of M columns and N rows. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component, or only the pixel / pixel value of the chrominance component. Samples can be used as a term to form a frame (or image) corresponding to a pixel or cell.

[0063] Encoding device 200 can subtract the prediction signal (prediction block, prediction sample array) output from inter-frame predictor 221 or intra-frame predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is sent to converter 232. In this case, the unit in encoding device 200 that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) can be called subtractor 231.

[0064] Predictor 220 can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block that includes prediction samples of the current block. Predictor 220 can determine whether to apply intra-frame prediction or inter-frame prediction on a per-block or per-unit basis. Predictor 220 can generate various information about the prediction (e.g., prediction mode information) and send it to entropy encoder 240, as described later in the description of the various prediction modes. The information about the prediction can be encoded in entropy encoder 240 and output as a bitstream.

[0065] Intra-frame predictor 222 can predict the current block by referencing samples within the current frame. Depending on the prediction mode, the referenced samples can be located near the current block or positioned at a specific distance away from the current block. In intra-frame prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. The non-directional mode can include at least one of a DC mode or a planar mode. Depending on the granularity of the prediction direction, the directional modes can include 33 or 65 directional modes. However, this is just an example; more or fewer directional modes can be used depending on the configuration. Intra-frame predictor 222 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0066] Inter-frame predictor 221 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a co-located reference block, a co-located CU (colCU), etc., and the reference frame including the temporally neighboring block may be referred to as a co-located frame (colPic). For example, inter-frame predictor 221 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 221 can use the motion information of neighboring blocks as the motion information of the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, the motion vectors of neighboring blocks are used as motion vector predictors, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0067] Predictor 220 can generate a prediction signal based on various prediction methods described later. For example, the predictor can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply both intra-frame and inter-frame prediction simultaneously. This can be referred to as the Inter-intra-frame Combined Prediction (CIIP) mode. Alternatively, the predictor can predict blocks based on the Intra-Block Copy (IBC) prediction mode, or it can predict blocks based on a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding such as Screen Content Coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When applying a palette mode, sample values ​​within the frame can be signaled based on information about the palette table and palette index. The prediction signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal.

[0068] Transformer 232 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen–Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT represents the transform obtained from a graph when the relationship information between pixels is represented as a graph. CNT represents the transform obtained based on generating a prediction signal using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size, or it can be applied to non-square blocks of variable size.

[0069] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. This information about the quantized transform coefficients can be called residual information. Quantizer 233 can reorder the block-form quantized transform coefficients into a 1D vector form based on the coefficient scan order, and can generate information about the quantized transform coefficients based on this 1D vector form.

[0070] The entropy encoder 240 can perform various encoding methods such as Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 240 can encode information required for video / image reconstruction other than quantization transform coefficients (e.g., values ​​of syntax elements, etc.) together or separately.

[0071] Encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) unit level. The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. Information and / or syntax elements transmitted from the encoding device to / signaled to the decoding device may be included in the video / image information. The video / image information can be encoded and included in the bitstream through the encoding process described above. The bitstream can be transmitted over a network or stored in a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) for transmitting the signal output from the entropy encoder 240 and / or a storage unit (not shown) for storing the signal may be configured as internal / external components of the encoding device 200, or the transmitting unit may also be included in the entropy encoder 240.

[0072] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients via dequantizer 234 and inverse transformer 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from inter-frame predictor 221 or intra-frame predictor 222 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). When there is no residual for the block to be processed (similar to when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can also be used for inter-frame prediction of the next frame by filtering, as described later. Furthermore, luminance mapping and chroma scaling (LMCS) can be applied in frame encoding and / or reconstruction processing.

[0073] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270 (specifically, the DPB of memory 270). Various filtering methods can include deblocking filtering, sample adaptive offsetting, adaptive loop filtering, bilateral filtering, etc. Filter 260 can generate various information about the filtering and send it to entropy encoder 240. The information about the filtering can be encoded in entropy encoder 240 and output as a bitstream.

[0074] The modified reconstructed frame sent to memory 270 can be used as a reference frame in inter-frame predictor 221. When inter-frame prediction is applied through it, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can also improve encoding efficiency.

[0075] The DPB of memory 270 can store modified reconstructed frames for use as reference frames in inter-frame predictor 221. Memory 270 can store motion information of blocks from which motion information in the current frame is derived (or encoded) and / or motion information of blocks in previously reconstructed frames. The stored motion information can be sent to inter-frame predictor 221 as motion information for spatially or temporally neighboring blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current frame and send them to intra-frame predictor 222.

[0076] Figure 3 A schematic block diagram of a decoding device that can be implemented using embodiments of the present disclosure and performs decoding of video / image signals is shown.

[0077] Reference Figure 3 The decoding device 300 can be configured to include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322.

[0078] According to the implementation, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 described above can be configured by a single hardware component (e.g., a decoder chipset or processor). Additionally, the memory 360 may include a decoded screen buffer (DPB) and can be configured by a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0079] When the input includes a bitstream containing video / image information, the decoding device 300 can respond to... Figure 2 The encoding device processes video / image information to reconstruct the image. For example, the decoding device 300 can deduce units / blocks based on block segmentation information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied in the encoding device. Therefore, the decoding processing unit can be an encoding unit, and the encoding unit can be segmented from the encoding tree unit or a larger encoding unit according to a quadtree structure, binary tree structure, and / or ternary tree structure. At least one transform unit can be derived from the encoding unit. Furthermore, the reconstructed image signal decoded and output by the decoding device 300 can be played back by a playback device.

[0080] Decoding device 300 can receive data in bitstream form from... Figure 2 The signal output by the encoding device can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. The decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements that are signaled / received, as described later herein, can be decoded and obtained from the bitstream through the decoding process. For example, the entropy decoder 310 can decode the information in the bitstream based on encoding methods such as Exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element from the bitstream, determine a context model using information about the syntax element to be decoded, decoding information of adjacent blocks and the block to be decoded, or information about symbols / bins decoded in previous steps, perform arithmetic decoding of bins by predicting the occurrence probability of bins based on the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins for the context model of the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictors (inter-frame predictor 332 and intra-frame predictor 331), and the residual values ​​(i.e., quantization transform coefficients and related parameter information) from which entropy decoding has been performed in the entropy decoder 310 can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, information about filtering from the information decoded in the entropy decoder 310 can be provided to the filter 350. Furthermore, the receiving unit (not shown) that receives the signal output from the encoding device can be further configured as an internal / external component of the decoding device 300, or the receiving unit can be a component of the entropy decoder 310.

[0081] Furthermore, the decoding device according to this specification may be referred to as a video / image / screen decoding device, and the decoding device may be divided into an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0082] Dequantizer 321 can dequantize the quantized transform coefficients and output the transform coefficients. Dequantizer 321 can reorder the quantized transform coefficients into two-dimensional blocks. In this case, the reordering can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0083] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0084] Predictor 320 can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. Predictor 320 can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from entropy decoder 310, and determine the specific intra-frame / inter-frame prediction mode.

[0085] Predictor 320 can generate prediction signals based on various prediction methods described later. For example, predictor 320 can not only apply intra-frame prediction or inter-frame prediction to predict a block, but can also apply intra-frame prediction and inter-frame prediction simultaneously. This can be referred to as the Inter-Frame Intra-Frame Combined Prediction (CIIP) mode. Alternatively, the predictor can predict blocks based on the Intra-Frame Block Copy (IBC) prediction mode, or it can predict blocks based on a palette mode. The IBC prediction mode or palette mode can be used for content image / video coding such as Screen Content Coding (SCC) in games, etc. IBC essentially performs prediction within the current frame, but it can be performed similarly to inter-frame prediction because it derives a reference block within the current frame. In other words, IBC can use at least one of the inter-frame prediction techniques described herein. The palette mode can be considered an example of intra-frame coding or intra-frame prediction. When the palette mode is applied, information about the palette table and palette index can be included in the video / image information and signaled.

[0086] Intra-predictor 331 can predict the current block by referencing samples within the current frame. Depending on the prediction mode, the referenced samples can be located near the current block or at a specific distance away. In intra-prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. Intra-predictor 331 can determine the prediction mode applied to the current block by using prediction modes applied to neighboring blocks.

[0087] Inter-frame predictor 332 can deduce the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0088] Adder 340 can add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including inter-frame predictor 332 and / or intra-frame predictor 331) to generate a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array). When there is no residual for the block to be processed (similar to when a skip mode is applied), the prediction block can be used as a reconstruction block.

[0089] Adder 340 can be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, can be output through filtering as described later, or can be used for inter-frame prediction of the next frame. In addition, luminance mapping and chroma scaling (LMCS) can be applied in the frame decoding process.

[0090] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and send the modified reconstructed image to memory 360 (specifically, the DPB of memory 360). Various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0091] The (modified) reconstructed frame stored in the DPB of memory 360 can be used as a reference frame in inter-frame predictor 332. Memory 360 can store motion information of blocks in the current frame from which motion information is derived (or decoded) and / or motion information of blocks in previously reconstructed frames. The stored motion information can be sent to inter-frame predictor 332 as motion information of spatially or temporally neighboring blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current frame and send them to intra-frame predictor 331.

[0092] The embodiments described in this document in the filter 260, inter-frame predictor 221 and intra-frame predictor 222 of the encoding device 200 can also be applied equivalently or correspondingly to the filter 350, inter-frame predictor 332 and intra-frame predictor 331 of the decoding device 300, respectively.

[0093] Figure 4 A method for generating a bit stream, performed by an encoding device 200 according to the present disclosure, is illustrated.

[0094] It can receive video frames to be encoded using the S400.

[0095] The received video frame can be encoded to generate a compressed video frame S410.

[0096] It can generate encoder optimization information (EOI) S420.

[0097] The encoder optimization information disclosed herein can be related to the received or compressed video footage. The encoder optimization information can define the purpose, characteristics, and application scope (or objective) of the encoder optimization.

[0098] As an example, encoder optimization information may include an EOI cancellation flag (eoi_cancel_flag). eoi_cancel_flag can be associated with the persistence of encoder optimization information. When the value of eoi_cancel_flag is 1, it indicates that the persistence of previously applied encoder optimization information is cancelled. For example, when the value of eoi_cancel_flag is 1, it indicates that the persistence of encoder optimization information included in previous prediction units (PUs) in output order is cancelled. When the value of eoi_cancel_flag is 0, it indicates that optimization-related information applied during preprocessing or encoding follows.

[0099] Optimization-related information may include at least one of the following: EOI persistence flag (eoi_persistence_flag), EOI identifier for human viewing (eoi_for_human_viewing_idc), EOI identifier for machine analysis (eoi_for_machine_analysis_idc), or EOI type information (eoi_type). In other words, the above optimization-related information can be encoded based on the fact that the value of eoi_cancel_flag is 0.

[0100] `eoi_persistence_flag` relates to the persistence of optimization information. `eoi_persistence_flag` represents the persistence of optimization information as indicated by `eoi_type`. When the value of `eoi_persistence_flag` is 0, it indicates that the optimization information (based on the `eoi_type` identifier) ​​applies only to the current frame. When the value of `eoi_persistence_flag` is 1, it indicates that the optimization information (based on the `eoi_type` identifier) ​​applies to the current frame and all subsequent frames. Here, "subsequent frames" can refer to all subsequent frames of the current layer in output order.

[0101] `eoi_for_human_viewing_idc` represents information used to identify the purpose and objective of optimization. A value of 3 indicates that the optimization aims to include human viewing. A value of 2 indicates that the video is suitable for human viewing but is not specifically optimized for human viewing. A value of 1 indicates that the video is not suitable for human viewing. A value of 0 indicates that it is unknown whether the video is suitable for human viewing.

[0102] `eoi_for_machine_analysis_idc` represents information used to identify the purpose and objective of optimization. A value of 3 indicates that the optimization objective includes machine analysis. A value of 2 indicates that the video is suitable for machine analysis but not specifically optimized for it. A value of 1 indicates that the video is not suitable for machine analysis. A value of 0 indicates that it is unknown whether the video is suitable for machine analysis.

[0103] `eoi_type` can represent a characteristic of an optimization method. For example, `eoi_type` can be defined as shown in Table 1 below. `eoi_type` can identify the optimization characteristic (or optimization type) defined in Table 1. However, the optimization characteristics defined in Table 1 are merely examples, and new optimization characteristics can also be defined under the corresponding structure. Alternatively, `eoi_type` can identify only a portion of the optimization characteristics defined in Table 1. In addition to Table 1, other optimization characteristics not defined in Table 1 can also be defined.

[0104] [Table 1]

[0105] When (eoi_type & bitMask) is not 0, it indicates that the application has the optimization features with the bitmask values ​​in Table 1. When eoi_type is greater than 0 and (eoi_type & bitMask) is 0, it indicates that the optimization features with the corresponding bitmask values ​​are not applied. When eoi_type is 0, it indicates that the optimizations determined by the application are used. For example, the optimization features based on eoi_type can also be defined as shown in Table 2 below.

[0106] [Table 2]

[0107] The encoder optimization information related to the objectives, characteristics, and application scope of the optimizations described above can also be applied equivalently to the implementation methods described below. The encoder optimization information can be defined in the SEI message, as shown in Table 3 below.

[0108] [Table 3]

[0109] In the encoder optimization information according to this disclosure, bit depth optimization-related information can be defined. This bit depth optimization-related information may include at least one of the following: information related to the shift size of the input sample, information related to the bit depth, or information related to the bit depth optimization type.

[0110] 1. Information related to the shift size

[0111] Information related to the shift size of the input sample can be used to identify the shift size of the input sample. This shift size information can be applied equally to the luma and chroma components, regardless of the component type of the input sample. For example, the information used to identify the shift size of the input sample can be defined as eoi_num_bit_depth_shift, representing the shift size of the luma and chroma components.

[0112] Alternatively, information identifying the shift size of the input sample can be defined for each component type of the input sample. For example, the information identifying the shift size of the input sample can be defined as bit_depth_shift_luma representing the shift size of the luminance component and bit_depth_shift_chroma representing the shift size of the chrominance component. Alternatively, the information identifying the shift size of the input sample can be defined as bit_depth_shift_luma representing the shift size of the luminance component, bit_depth_shift_cb representing the shift size of the Cb component, and bit_depth_shift_cr representing the shift size of the Cr component.

[0113] 2. Information related to bit depth

[0114] Information related to bit depth can be defined as at least one of the following: information related to the input bit depth (or, the bit depth before shifting), information related to the optimized bit depth (or, the bit depth after shifting), information related to the bit depth difference, or information related to the task bit depth.

[0115] 2-1. Information related to input bit depth

[0116] Information related to the input bit depth can indicate the magnitude of the bit depth of the input sample.

[0117] For example, information related to the input bit depth can be defined as eoi_input_bit_depth_minus8, which represents the value obtained by subtracting 8 from the size of the bit depth of the input sample. In this case, the bit depth of the input sample can be derived as shown in Equation 1 below.

[0118] [Formula 1]

[0119] Bitdepth=8+eoi_input_bit_depth_minus8

[0120] Alternatively, information related to the input bit depth can also be defined as eoi_input_bit_depth, which represents a value with the same magnitude as the bit depth of the input sample. In this case, the bit depth of the input sample can be derived as shown in Equation 2 below.

[0121] [Equation 2]

[0122] Bitdepth=eoi_input_bit_deltt

[0123] The information related to the input bit depth described above can be applied equally to the luminance and chrominance components, regardless of the component type of the input sample.

[0124] Alternatively, information related to the input bit depth can be defined separately for each component type of the input sample. This information can be defined separately as information representing the magnitude of the bit depth for the luminance component and information representing the magnitude of the bit depth for the chrominance component.

[0125] For example, information representing the bit depth of the luminance component can be defined as eoi_input_luma_bit_depth_minus8, which represents the value obtained by subtracting 8 from the bit depth of the input sample associated with the luminance component. In this case, the bit depth of the input sample associated with the luminance component can be derived as shown in Equation 3 below.

[0126] [Formula 3]

[0127] Bitdepth=8+eoi_input_luma_bit_depth_minus8

[0128] Alternatively, information representing the size of the bit depth of the luminance component can also be defined as eoi_input_luma_bit_depth, which represents the same value as the bit depth of the input sample associated with the luminance component.

[0129] Information representing the bit depth of the chroma component can be defined as eoi_input_chroma_bit_depth_minus8, which represents the value obtained by subtracting 8 from the bit depth of the input sample associated with the chroma component. In this case, the bit depth of the input sample associated with the chroma component can be derived as shown in Equation 4 below.

[0130] [Formula 4]

[0131] Bitdepth=8+eoi_input_chroma_bit_depth_minus8

[0132] Alternatively, information representing the size of the bit depth of the chroma component can also be defined as eoi_input_chroma_bit_depth, which represents the same value as the bit depth of the input sample associated with the chroma component.

[0133] Alternatively, information representing the bit depth of the chrominance component can be defined separately for each Cb component and each Cr component. In this case, the information representing the bit depth of the chrominance component can be defined separately as information representing the bit depth of the Cb component and information representing the bit depth of the Cr component.

[0134] The information representing the bit depth of the Cb component can be defined as eoi_input_cb_bit_depth_minus8, which represents the value obtained by subtracting 8 from the bit depth of the input sample associated with the Cb component. In this case, the bit depth of the input sample associated with the Cb component can be derived as shown in Equation 5 below.

[0135] [Formula 5]

[0136] Bitdepth=8+eoi_input_cb_bit_depth_minus8

[0137] Information representing the size of the bit depth of the Cb component can also be defined as eoi_input_cb_bit_depth, which represents the same value as the bit depth of the input sample associated with the Cb component.

[0138] The information representing the bit depth of the Cr component can be defined as eoi_input_cr_bit_depth_minus8, which represents the value obtained by subtracting 8 from the bit depth of the input sample associated with the Cr component. The bit depth of the input sample associated with the Cr component can be derived as shown in Equation 6 below.

[0139] [Formula 6]

[0140] Bitdepth=8+eoi_input_cr_bit_depth_minus8

[0141] Information representing the size of the bit depth of the Cr component can also be defined as eoi_input_cr_bit_depth, which represents the same value as the bit depth of the input sample associated with the Cr component.

[0142] 2-2. Information related to the optimized bit depth

[0143] Information associated with the optimized bit depth can indicate the magnitude of the optimized bit depth.

[0144] For example, information related to the optimized bit depth can be defined as eoi_optimized_bit_depth_minus8, which represents the value obtained by subtracting 8 from the size of the optimized bit depth. In this case, the optimized bit depth (Optimized_Bitdepth) can be derived as shown in Equation 7 below.

[0145] [Formula 7]

[0146] Optimized_Bitdepth=8+eoi_optimized_bit_depth_minus8

[0147] Alternatively, the information associated with the optimized bit depth can also be defined as `eoi_optimized_bit_depth_minus6`, which represents the value obtained by subtracting 6 from the size of the optimized bit depth. This is because when it is defined as the value obtained by subtracting 8 in the same way as the information associated with the bit depth of the input sample, the range of the optimized bit depth can be constrained. For example, when the maximum bit depth of the input sample is 10, the optimized bit depth can be constrained to 8 or 9. In this case, the optimized bit depth (Optimized_Bitdepth) can be derived as shown in Equation 8.

[0148] [Formula 8]

[0149] Optimized_Bitdepth=6+eoi_optimized_bit_depth_minus8

[0150] Alternatively, information related to the optimized bit depth can also be defined as eoi_optimized_bit_depth, which represents a value with the same magnitude as the optimized bit depth. In this case, the optimized bit depth (Optimized_Bitdepth) can be derived as shown in Equation 9 below.

[0151] [Formula 9]

[0152] Optimized_Bitdepth=eoi_optimized_bit_depth

[0153] The information related to the optimized bit depth described above can be applied equally to the luminance and chrominance components, regardless of the optimized component type.

[0154] Alternatively, information related to the optimized bit depth can be defined separately for each optimized component type. In this case, the information related to the optimized bit depth can be defined separately as information representing the magnitude of the optimized bit depth for the luminance component and information representing the magnitude of the optimized bit depth for the chrominance component.

[0155] Information representing the size of the optimized bit depth for the luminance component can be defined as eoi_optimized_luma_bit_depth_minus8, which represents the value obtained by subtracting 8 from the optimized bit depth of the luminance component. In this case, the optimized bit depth of the luminance component can be derived as follows: Equation 10.

[0156] [Formula 10]

[0157] Optimized_Bitdepth=8+eoi_optimized_luma_bit_depth_minus8

[0158] Alternatively, information representing the size of the optimized bit depth for the luminance component can be defined as eoi_optimized_luma_bit_depth_minus6, which represents the value obtained by subtracting 6 from the optimized bit depth of the luminance component. In this case, the optimized bit depth of the luminance component can be derived as follows: Equation 11.

[0159] [Equation 11]

[0160] Optimized_Bitdepth=6+eoi_optimized_luma_bit_depth_minus6

[0161] Alternatively, information indicating the size of the optimized bit depth for the luminance component can also be defined as eoi_optimized_luma_bit_depth, which represents the same value as the optimized bit depth for the luminance component.

[0162] Information indicating the size of the optimized bit depth for the chroma component can be defined as eoi_optimized_chroma_bit_depth_minus8, which represents the value obtained by subtracting 8 from the optimized bit depth of the chroma component. In this case, the optimized bit depth of the chroma component can be derived as follows: Equation 12.

[0163] [Equation 12]

[0164] Optimized_Bitdepth=8+eoi_optimized_chroma_bit_depth_minus8

[0165] Alternatively, information representing the size of the optimized bit depth for the chroma component can be defined as eoi_optimized_chroma_bit_depth_minus6, which represents the value obtained by subtracting 6 from the optimized bit depth of the chroma component. In this case, the optimized bit depth of the chroma component can be derived as follows: Equation 13.

[0166] [Equation 13]

[0167] Optimized_Bitdepth=6+eoi_optimized_chroma_bit_depth_minus6

[0168] Alternatively, information indicating the size of the optimized bit depth for the chroma component can also be defined as eoi_optimized_chroma_bit_depth, which represents the same value as the optimized bit depth for the chroma component.

[0169] Information related to the optimized bit depth can also be defined as information representing the size of the optimized bit depth for the luminance component, information representing the size of the optimized bit depth for the Cb component, and information representing the size of the optimized bit depth for the Cr component.

[0170] The information representing the size of the optimized bit depth of the Cb component can be defined as eoi_optimized_cb_bit_depth_minus8, which represents the value obtained by subtracting 8 from the optimized bit depth of the Cb component. In this case, the optimized bit depth of the Cb component can be derived as follows: Equation 14.

[0171] [Formula 14]

[0172] Optimized_Bitdepth=8+eoi_optimized_cb_bit_depth_minus8

[0173] Alternatively, information representing the size of the optimized bit depth of the Cb component can be defined as eoi_optimized_cb_bit_depth_minus6, which represents the value obtained by subtracting 6 from the optimized bit depth of the Cb component. In this case, the optimized bit depth of the Cb component can be derived as follows: Equation 15.

[0174] [Formula 15]

[0175] Optimized_Bitdepth=6+eoi_optimized_cb_bit_depth_minus6

[0176] Alternatively, information representing the size of the optimized bit depth of the Cb component can also be defined as eoi_optimized_cb_bit_depth, which represents the same value as the optimized bit depth of the Cb component.

[0177] Information representing the size of the optimized bit depth of the Cr component can be defined as eoi_optimized_cr_bit_depth_minus8, which represents the value obtained by subtracting 8 from the optimized bit depth of the Cr component. In this case, the optimized bit depth of the Cr component can be derived as follows: Equation 16.

[0178] [Formula 16]

[0179] Optimized_Bitdepth=8+eoi_optimized_cr_bit_depth_minus8

[0180] Alternatively, information representing the size of the optimized bit depth of the Cr component can be defined as eoi_optimized_cr_bit_depth_minus6, which represents the value obtained by subtracting 6 from the optimized bit depth of the Cr component. In this case, the optimized bit depth of the Cr component can be derived as follows: Equation 17.

[0181] [Equation 17]

[0182] Optimized_Bitdepth=6+eoi_optimized_cr_bit_depth_minus6

[0183] Alternatively, information representing the size of the optimized bit depth of the Cr component can also be defined as eoi_optimized_cr_bit_depth, which represents the same value as the optimized bit depth of the Cr component.

[0184] 2-3. Information related to bit depth difference

[0185] Information related to the bit depth difference (eoi_bit_depth_difference_from_input) can be information about the size of the optimized bit depth. The bit depth difference can refer to the difference between the bit depth of the input sample and the optimized bit depth.

[0186] For example, when the information related to the bit depth of the input sample is defined as eoi_input_bit_depth_minus8, the optimized bit depth can be derived as shown in Equation 18.

[0187] [Formula 18]

[0188] Bitdepth=8+eoi_input_bit_depth_minus8

[0189] if(bitdepth optimization = Truncating bit depth)

[0190] Optimized_Bitdepth = Bitdepth - eoi_bit_depth_difference_from_input

[0191] if(bitdepth optimization = Increasing bit depth)

[0192] Optimized_Bitdepth = Bitdepth + eoi_bit_depth_difference_from_input

[0193] The information related to the bit depth difference described above can be applied equally to the luminance and chrominance components, regardless of the optimized component type.

[0194] Alternatively, information related to the bit depth difference can be defined separately for each optimized component type. In this case, the information related to the bit depth difference can be defined separately as information representing the bit depth difference for the luma component (eoi_luma_bit_depth_difference_from_input) and information representing the bit depth difference for the chroma component (eoi_chroma_bit_depth_difference_from_input).

[0195] For example, when the information representing the size of the bit depth of the luminance component is defined as eoi_input_luma_bit_depth_minus8, the optimized bit depth of the luminance component can be derived as follows: Equation 19.

[0196] [Formula 19]

[0197] Bitdepth=8+eoi_input_luma_bit_depth_minus8

[0198] if(bitdepth optimization = Truncating bit depth)

[0199] Optimized_Bitdepth = Bitdepth - eoi_luma_bit_depth_difference_from_input

[0200] if(bitdepth optimization = Increasing bit depth)

[0201] Optimized_Bitdepth = Bitdepth + eoi_luma_bit_depth_difference_from_input

[0202] When the information representing the size of the bit depth of the chroma component is defined as eoi_input_chroma_bit_depth_minus8, the optimized bit depth of the chroma component can be derived as follows: Equation 20.

[0203] [Formula 20]

[0204] Bitdepth=8+eoi_input_chroma_bit_depth_minus8

[0205] if(bitdepth optimization = Truncating bit depth)

[0206] Optimized_Bitdepth = Bitdepth - eoi_chroma_bit_depth_difference_from_input

[0207] if(bitdepth optimization = Increasing bit depth)

[0208] Optimized_Bitdepth = Bitdepth + eoi_chroma_bit_depth_difference_from_input

[0209] Alternatively, the information related to the bit depth difference can also be defined as the information representing the bit depth difference for the luminance component (eoi_luma_bit_depth_difference_from_input), the information representing the bit depth difference for the Cb component (eoi_input_cb_bit_depth_minus8), and the information representing the bit depth difference for the Cr component (eoi_input_cb_bit_depth_minus8).

[0210] When the information representing the size of the bit depth of the Cb component is defined as eoi_input_cb_bit_depth_minus8, the optimized bit depth of the Cb component can be derived as follows: Equation 21.

[0211] [Equation 21]

[0212] Bitdepth=8+eoi_input_cb_bit_depth_minus8

[0213] if(bitdepth optimization = Truncating bit depth)

[0214] Optimized_Bitdepth = Bitdepth - eoi_cb_bit_depth_difference_from_input

[0215] if(bitdepth optimization = Increasing bit depth)

[0216] Optimized_Bitdepth = Bitdepth + eoi_cb_bit_depth_difference_from_input

[0217] When the information representing the size of the bit depth of the Cr component is defined as eoi_input_cr_bit_depth_minus8, the optimized bit depth of the Cr component can be derived as follows: Equation 22.

[0218] [Equation 22]

[0219] Bitdepth=8+eoi_input_cr_bit_depth_minus8

[0220] if(bitdepth optimization = Truncating bit depth)

[0221] Optimized_Bitdepth = Bitdepth - eoi_cr_bit_depth_difference_from_input

[0222] if(bitdepth optimization = Increasing bit depth)

[0223] Optimized_Bitdepth = Bitdepth + eoi_cr_bit_depth_difference_from_input

[0224] 2-4. Information related to task bit depth

[0225] Information related to the task bit depth can indicate the size of the bit depth before a change or the size of the bit depth performed for the task. When the size of the bit depth of the encoded bitstream and the size of the bit depth based on the information related to the task bit depth are different from each other, it may be necessary to change the bit depth of the decoded image during the decoding process to the size of the bit depth based on the information related to the task bit depth to achieve a specific purpose.

[0226] Information related to the bit depth of a task can be defined as eoi_bit_depth_for_task, which can be applied equally to the luma and chroma components, regardless of the component type.

[0227] Alternatively, task bit depth related information can be defined separately for each component type. In this case, the task bit depth related information can be defined separately as task bit depth related information for the luma component (eoi_luma_bit_depth_for_task) and task bit depth related information for the chroma component (eoi_chroma_bit_depth_for_task). Alternatively, the task bit depth related information can be defined separately as task bit depth related information for the luma component (eoi_luma_bit_depth_for_task), task bit depth related information for the Cb component (eoi_cb_bit_depth_for_task), and task bit depth related information for the Cr component (eoi_cr_bit_depth_for_task).

[0228] 3. Information related to bit depth optimization type

[0229] Information related to the bit depth optimization type can be information used to identify the type of bit depth optimization. This information can indicate one of the predefined bit depth optimization types. Predefined bit depth optimization types can include at least one of truncating the bit depth or increasing the bit depth.

[0230] For example, information related to the bit depth optimization type can be defined by `eoi_bit_depth_optimization_type_flag`. When `eoi_bit_depth_optimization_type_flag` is 0, it indicates that the bit depth optimization type is truncated bit depth. When `eoi_bit_depth_optimization_type_flag` is 1, it indicates that the bit depth optimization type is increased bit depth. In other words, when `eoi_bit_depth_optimization_type_flag` is 0, a left shift can be applied, and when `eoi_bit_depth_optimization_type_flag` is 1, a right shift can be applied.

[0231] Conversely, a value of 0 for `eoi_bit_depth_optimization_type_flag` indicates that the bit depth optimization type is increasing the bit depth. A value of 1 for `eoi_bit_depth_optimization_type_flag` indicates that the bit depth optimization type is truncating the bit depth. In other words, a right shift can be applied when `eoi_bit_depth_optimization_type_flag` is 0, and a left shift can be applied when `eoi_bit_depth_optimization_type_flag` is 1.

[0232] Furthermore, the bit depth information of the video input to the encoding device can be included in the bitstream. In this case, all or part of the information related to the aforementioned input bit depth may not be defined as encoder optimization information in the bit depth optimization-related information. Instead, information related to whether the bit depth has changed (or whether bit depth optimization has been applied) or the optimized bit depth can be defined and used.

[0233] The aforementioned bit depth optimization related information can be generated and encoded into a bit stream based on at least one of a predetermined variable or a bit depth optimization identifier.

[0234] 1. Variable-based generation methods

[0235] The bit depth optimization according to this disclosure can be defined as a type of spatial resampling optimization. In this case, the predetermined variable can be a spatial resampling flag (EoiSpatialResamplingFlag) indicating whether spatial resampling optimization is applied. When the value of EoiSpatialResamplingFlag is 1, it indicates that spatial resampling has been applied. When the value of EoiSpatialResamplingFlag is 0, it indicates that spatial resampling has not been applied. The value of EoiSpatialResamplingFlag can be derived from Equation 23 or Equation 24 below.

[0236] [Equation 23]

[0237] EoiSpatialResamplingFlag = ((eoi_type & 0x04 ) > 0 ) ? 1 : 0

[0238] [Equation 24]

[0239] if ((eoi_type & 0x04 ) != 0)

[0240] EoiSpatialResamplingFlag= 1; else EoiSpatialResamplingFlag= 0; In this way, based on `eoi_type`, the application's optimization characteristics can be identified. Further details related to these optimization characteristics can be defined within the encoder optimization information. For example, with `EoiSpatialResamplingFlag` set to 1, bit depth optimization information can be generated and encoded into the bitstream. With `EoiSpatialResamplingFlag` set to 0, bit depth optimization information can be omitted and not encoded into the bitstream.

[0241] The bit depth optimization according to this disclosure can also be defined as a type of space quality optimization. In this case, the predetermined variable can be a flag (EoiQualityOptimizationFlag) indicating whether space quality optimization is applied. When the value of EoiQualityOptimizationFlag is 0, it indicates that space quality optimization is not applied. When the value of EoiQualityOptimizationFlag is 1, it indicates that space quality optimization is applied. The value of EoiQualityOptimizationFlag can be derived from Equation 25 or Equation 26 below.

[0242] [Equation 25]

[0243] EoiQualityOptimizationFlag = ((eoi_type & 0x10 ) > 0 ) ? 1 : 0

[0244] [Equation 26]

[0245] if ((eoi_type & 0x10 ) != 0)

[0246] EoiQualityOptimizationFlag = 1; else EoiQualityOptimizationFlag = 0; In this way, based on `eoi_type`, the application's optimization characteristics can be identified. Further details related to these optimization characteristics can be defined within the encoder's optimization information. For example, if `EoiQualityOptimizationFlag` is 1, bit depth optimization information can be generated and encoded into the bitstream. If `EoiQualityOptimizationFlag` is 0, bit depth optimization information can be omitted and not encoded into the bitstream.

[0247] The bit depth optimization according to this disclosure can also be defined as a new eoi_type in the encoder optimization information.

[0248] As examples, Tables 4 and 5 are additional examples of bit depth optimization.

[0249] [Table 4]

[0250] [Table 5]

[0251] In Table 5, when (eoi_type & bitMask) is not 0, it indicates that the application has the optimization features with the bitmask values ​​in Table 4. When eoi_type is greater than 0 and (eoi_type & bitMask) is 0, it indicates that the optimization features with the corresponding bitmask values ​​are not applied. When eoi_type is 0, it indicates that the optimizations determined by the application are used.

[0252] When bit depth optimization is defined as a new `eoi_type`, a predefined variable can be a flag indicating whether bit depth optimization is applied (`EoiBitdepthOptimizationFlag`). A value of 0 for `EoiBitdepthOptimizationFlag` indicates that bit depth optimization is not applied. A value of 1 for `EoiBitdepthOptimizationFlag` indicates that bit depth optimization is applied. The value of `EoiBitdepthOptimizationFlag` can be derived using equation 27 or equation 28 below.

[0253] [Equation 27]

[0254] EoiBitdepthOptimizationFlag = ((eoi_type & 0x40 ) > 0 ) ? 1 : 0

[0255] [Equation 28]

[0256] if ((eoi_type & 0x40 ) != 0)

[0257] EoiBitdepthOptimizationFlag = 1; else EoiBitdepthOptimizationFlag = 0; In this way, based on `eoi_type`, the application's optimization characteristics can be identified. Further details related to the application's optimization characteristics can be defined within the encoder's optimization information. For example, if `EoiBitdepthOptimizationFlag` is 1, bit depth optimization information can be generated and encoded into the bitstream. In this case, it might be necessary to generate information related to the aforementioned bit depth optimization type as bit depth optimization information and encode it into the bitstream. If `EoiBitdepthOptimizationFlag` is 0, bit depth optimization information can be omitted, and it can be left unencoded into the bitstream.

[0258] Alternatively, Tables 6 and 7 provide examples of bit depth truncation defined for eoi_type.

[0259] [Table 6]

[0260] [Table 7]

[0261] In Table 7, when (eoi_type & bitMask) is not 0, it indicates that the application has the optimization features with the bitmask values ​​in Table 6. When eoi_type is greater than 0 and (eoi_type & bitMask) is 0, it indicates that the optimization features with the corresponding bitmask values ​​are not applied. When eoi_type is 0, it indicates that the optimizations determined by the application are used.

[0262] When bit depth truncation is defined as a new `eoi_type`, a predefined variable can be a flag indicating whether bit depth truncation is applied (`EoiBitdepthTruncationFlag`). A value of 0 for `EoiBitdepthTruncationFlag` indicates that bit depth truncation is not applied. A value of 1 for `EoiBitdepthTruncationFlag` indicates that bit depth truncation is applied. The value of `EoiBitdepthTruncationFlag` can be derived using equation 29 or equation 30 below.

[0263] [Equation 29]

[0264] EoiBitdepthTruncationFlag = ((eoi_type & 0x40 ) > 0 ) ? 1 : 0

[0265] [Formula 30]

[0266] if ((eoi_type & 0x40 ) != 0)

[0267] EoiBitdepthTruncationFlag = 1; else EoiBitdepthTruncationFlag = 0; In this way, based on `eoi_type`, the optimization characteristics of the application can be identified. Detailed information related to the application's optimization characteristics can be further defined in the encoder optimization information. For example, based on a value of `EoiBitdepthTruncationFlag` of 1, bit depth optimization-related information can be generated and encoded into the bitstream. In this case, information related to the aforementioned bit depth optimization type may not be generated as bit depth optimization-related information and may not be encoded into the bitstream. In other words, information related to the bit depth optimization type can be adaptively generated and encoded into the bitstream based on the optimization characteristics added to `eoi_type`. Based on a value of `EoiBitdepthTruncationFlag` of 0, bit depth optimization-related information may not be generated and may not be encoded into the bitstream.

[0268] The variables according to this disclosure may be one of the above-mentioned EoiSpatialResamplingFlag, EoiQualityOptimizationFlag, EoiBitdepthOptimizationFlag or EoiBitdepthTruncationFlag, or may include at least two of them.

[0269] 2. A method for generating identifiers based on bit depth optimization

[0270] The bit depth optimization identifier can be information used to identify bit depth optimization. The bit depth optimization identifier can be defined as at least one of the spatial resampling type identifier (eoi_spatial_resampling_type_idc) or the quality optimization type identifier (eoi_quality_optimization_type_idc).

[0271] As an example, the spatial resampling type identifier (eoi_spatial_resampling_type_idc) can be used as a bit depth optimization identifier. eoi_spatial_resampling_type_idc can be information used to identify the type of spatial resampling used by the application.

[0272] When the value of `eoi_spatial_resampling_type_idc` is 0, it indicates that the spatial resampling type is upsampling. When the value of `eoi_spatial_resampling_type_idc` is 1, it indicates that the spatial resampling type is downsampling. When the value of `eoi_spatial_resampling_type_idc` is 2, it indicates that the spatial resampling type is truncated bit depth. When the value of `eoi_spatial_resampling_type_idc` is 3, it indicates that the spatial resampling type is increased bit depth. Alternatively, when the value of `eoi_spatial_resampling_type_idc` is 2, it indicates that the spatial resampling type is increased bit depth, and when the value of `eoi_spatial_resampling_type_idc` is 3, it also indicates that the spatial resampling type is truncated bit depth.

[0273] eoi_spatial_resampling_type_idc can be represented using 2 bits as shown in Table 8 below.

[0274] [Table 8]

[0275] Alternatively, Table 9 can correspond to cases where the value of `eoi_spatial_resampling_type_idc` is between 0 and 2. Truncated bit depth or increased bit depth may not be defined as the type of spatial resampling. In other words, when the value of `eoi_spatial_resampling_type_idc` is 2, it can indicate that the type of spatial resampling is truncated bit depth or increased bit depth. In Table 9, truncated bit depth can be replaced with increased bit depth.

[0276] [Table 9]

[0277] Upsampling, downsampling, truncating bit depth, and increasing bit depth are examples of spatial resampling, and other types of spatial resampling can also be identified based on eoi_spatial_resampling_type_idc.

[0278] Alternatively, the quality optimization type identifier (eoi_quality_optimization_type_idc) can be used as a bit depth optimization identifier. eoi_quality_optimization_type_idc can be information used to identify the type of spatial quality optimization applied. Specifically, eoi_quality_optimization_type_idc can be defined as follows.

[0279] For example, a value of 0 for `eoi_quality_optimization_type_idc` indicates that the spatial quality optimization type is a denoising filter. A value of 1 for `eoi_quality_optimization_type_idc` indicates that the spatial quality optimization type is an edge-preserving filter. A value of 2 for `eoi_quality_optimization_type_idc` indicates that the spatial quality optimization type is image enhancement. A value of 3 for `eoi_quality_optimization_type_idc` indicates that the spatial quality optimization type is truncated bit depth.

[0280] In this case, eoi_quality_optimization_type_idc can be represented using 2 bits as shown in Table 10 below.

[0281] [Table 10]

[0282] Alternatively, when the value of `eoi_quality_optimization_type_idc` is 0, it indicates that the spatial quality optimization type is a denoising filter. When the value of `eoi_quality_optimization_type_idc` is 1, it indicates that the spatial quality optimization type is an edge-preserving filter. When the value of `eoi_quality_optimization_type_idc` is 2, it indicates that the spatial quality optimization type is image enhancement processing. When the value of `eoi_quality_optimization_type_idc` is 3, it indicates that the spatial quality optimization type is bit depth optimization.

[0283] When `eoi_quality_optimization_type_idc` indicates bit depth optimization (e.g., `eoi_quality_optimization_type_idc=3`), information may be needed to identify whether the bit depth optimization type is truncated bit depth. Based on the bit depth optimization indicated by `eoi_quality_optimization_type_idc`, information related to the bit depth optimization type can be generated and encoded into the bitstream.

[0284] eoi_quality_optimization_type_idc can be represented using 2 bits as shown in Table 11.

[0285] [Table 11]

[0286] In addition to the types of spatial quality optimization mentioned above, other types of spatial quality optimization can be identified based on eoi_quality_optimization_type_idc.

[0287] The following section will describe an example of the encoder optimization information syntax that defines bit depth optimization-related information.

[0288] [Table 12]

[0289] Table 12 shows the case where bit depth optimization-related information is defined as eoi_num_bit_depth_shift.

[0290] [Table 13]

[0291] Table 13 shows the cases where bit depth optimization-related information is defined as bit_depth_shift_luma and bit_depth_shift_chroma.

[0292] [Table 14]

[0293] Table 14 shows the cases where bit depth optimization related information is defined as bit_depth_shift_luma, bit_depth_shift_cb, and bit_depth_shift_cr.

[0294] [Table 15]

[0295] Table 15 shows the cases where bit depth optimization information is defined as eoi_input_bit_depth_minus8 and eoi_optimized_bit_depth_minus8.

[0296] [Table 16]

[0297] Table 16 shows the cases where bit depth optimization information is defined as eoi_input_bit_depth_minus8 and eoi_optimized_bit_depth_minus6.

[0298] [Table 17]

[0299] Table 17 shows the cases where bit depth optimization related information is defined as eoi_input_bit_depth_minus8 and eoi_bit_depth_difference_from_input.

[0300] [Table 18]

[0301] Table 18 shows the cases where bit depth optimization information is defined as eoi_input_bit_depth and eoi_optimized_bit_depth.

[0302] [Table 19]

[0303] Table 19 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, and eoi_optimized_chroma_bit_depth_minus8.

[0304] [Table 20]

[0305] Table 20 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus6, and eoi_optimized_chroma_bit_depth_minus6.

[0306] [Table 21]

[0307] Table 21 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_luma_bit_depth_difference_from_input, and eoi_chroma_bit_depth_difference_from_input.

[0308] [Table 22]

[0309] Table 22 shows the cases where bit depth optimization-related information is defined as eoi_input_luma_bit_depth, eoi_input_chroma_bit_depth, eoi_optimized_luma_bit_depth, and eoi_optimized_chroma_bit_depth.

[0310] [Table 23]

[0311] Table 23 shows the cases where the bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, eoi_optimized_cb_bit_depth_minus8, and eoi_optimized_cr_bit_depth_minus8.

[0312] [Table 24]

[0313] Table 24 shows the cases where the bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus6, eoi_optimized_cb_bit_depth_minus6, and eoi_optimized_cr_bit_depth_minus6.

[0314] [Table 25]

[0315] Table 25 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_luma_bit_depth_difference_from_input, eoi_cb_bit_depth_difference_from_input, and eoi_cr_bit_depth_difference_from_input.

[0316] [Table 26]

[0317] Table 26 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth, eoi_input_cb_bit_depth, eoi_input_cr_bit_depth, eoi_optimized_luma_bit_depth, eoi_optimized_cb_bit_depth, and eoi_optimized_cr_bit_depth.

[0318] The bit depth optimization information according to Tables 12 to 26 can be generated and encoded into the bitstream based on the value of EoiSpatialResamplingFlag being 1. On the other hand, if the value of EoiSpatialResamplingFlag is 0, the bit depth optimization information according to Tables 12 to 26 can be omitted from encoding into the bitstream.

[0319] When eoi_spatial_resampling_type_idc is 2 (i.e., when the bit depth is truncated), relevant bit depth optimization information based on Tables 12 to 26 can be generated and encoded into the bit stream.

[0320] Alternatively, when the value of eoi_spatial_resampling_type_idc is the same as 2 and a truncated bit depth is applied, the bit depth optimization information can represent the size of a right shift. When the value of eoi_spatial_resampling_type_idc is the same as 3 and an increased bit depth is applied, the bit depth optimization information can represent the size of a left shift. In this way, the bit depth optimization information can represent the size of the shift in the shift direction according to eoi_spatial_resampling_type_idc. In this case, the condition if(eoi_spatial_resampling_type_idc==2) in Tables 12 to 14 can also be deleted, and the corresponding condition can also be changed to the case where the value of eoi_spatial_resampling_type_idc is 2 or 3 (i.e., if(eoi_spatial_resampling_type_idc==2||3)).

[0321] The values ​​0, 1, 2, 3, 4, 5, 6, and 7 in the bit depth optimization information in Tables 12 to 14 represent input samples shifted by 1 bit, 2 bits, 3 bits, 4 bits, 5 bits, 6 bits, 7 bits, and 8 bits, respectively. The shift direction can be determined by `eoi_spatial_resampling_type_idc`. Since there is no reason to define a corresponding value when there is no shift, the case of a 0-bit shift can be omitted.

[0322] When the state of shifting to 0 is not defined, it can be represented as eoi_num_bit_depth_shift_minus1 to more clearly represent the meaning of the syntax, and when the value is 0, 1, 2, 3, 4, 5, 6 and 7, it can represent that the input sample is shifted by 1 bit, 2 bits, 3 bits, 4 bits, 5 bits, 6 bits, 7 bits and 8 bits respectively.

[0323] [Table 27]

[0324] Table 27 shows the case where bit depth optimization-related information is defined as eoi_num_bit_depth_shift.

[0325] [Table 28]

[0326] Table 28 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag and eoi_num_bit_depth_shift.

[0327] [Table 29]

[0328] Table 29 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, bit_depth_shift_luma, and bit_depth_shift_chroma.

[0329] [Table 30]

[0330] Table 30 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, bit_depth_shift_luma, bit_depth_shift_cb, and bit_depth_shift_cr.

[0331] [Table 31]

[0332] Table 31 shows the cases where bit depth optimization related information is defined as eoi_input_bit_depth_minus8 and eoi_optimized_bit_depth_minus8.

[0333] [Table 32]

[0334] Table 32 shows the cases where bit depth optimization related information is defined as eoi_input_bit_depth_minus8 and eoi_optimized_bit_depth_minus6.

[0335] [Table 33]

[0336] Table 33 shows the cases where bit depth optimization related information is defined as eoi_input_bit_depth_minus8 and eoi_bit_depth_difference_from_input.

[0337] [Table 34]

[0338] Table 34 shows the cases where bit depth optimization information is defined as eoi_input_bit_depth and eoi_optimized_bit_depth.

[0339] [Table 35]

[0340] Table 35 shows the case where bit depth optimization information is defined as eoi_bit_depth_for_task.

[0341] [Table 36]

[0342] Table 36 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, and eoi_optimized_chroma_bit_depth_minus8.

[0343] [Table 37]

[0344] Table 37 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus6, and eoi_optimized_chroma_bit_depth_minus6.

[0345] [Table 38]

[0346] Table 38 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_luma_bit_depth_difference_from_input, and eoi_chroma_bit_depth_difference_from_input.

[0347] [Table 39]

[0348] Table 39 shows the cases where bit depth optimization-related information is defined as eoi_input_luma_bit_depth, eoi_input_chroma_bit_depth, eoi_optimized_luma_bit_depth, and eoi_optimized_chroma_bit_depth.

[0349] [Table 40]

[0350] Table 40 shows the cases where bit depth optimization related information is defined as eoi_luma_bit_depth_for_task and eoi_chroma_bit_depth_for_task.

[0351] [Table 41]

[0352] Table 41 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, eoi_optimized_cb_bit_depth_minus8, and eoi_optimized_cr_bit_depth_minus8.

[0353] [Table 42]

[0354] Table 42 shows the cases where the bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, eoi_optimized_cb_bit_depth_minus6, and eoi_optimized_cr_bit_depth_minus6.

[0355] [Table 43]

[0356] Table 43 shows the cases where bit depth optimization related information is defined as eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_luma_bit_depth_difference_from_input, eoi_cb_bit_depth_difference_from_input, and eoi_cr_bit_depth_difference_from_input.

[0357] [Table 44]

[0358] Table 44 shows the cases where bit depth optimization-related information is defined as eoi_input_luma_bit_depth, eoi_input_cb_bit_depth, eoi_input_cr_bit_depth, eoi_optimized_luma_bit_depth, eoi_optimized_cb_bit_depth, and eoi_optimized_cr_bit_depth.

[0359] [Table 45]

[0360] Table 45 shows the cases where bit depth optimization related information is defined as eoi_luma_bit_depth_for_task, eoi_cb_bit_depth_for_task, and eoi_cr_bit_depth_for_task.

[0361] The bit depth optimization information from Tables 27 to 45 can be generated and encoded into the bitstream based on an EoQualityOptimizationFlag value of 1. Conversely, if the EoQualityOptimizationFlag value is 0, the bit depth optimization information from Tables 27 to 45 may not be encoded into the bitstream.

[0362] When eoi_quality_optimization_type_idc is 3 (i.e., when the type of spatial quality optimization is bit depth optimization), bit depth optimization related information according to Tables 27 to 45 can be generated and encoded into the bit stream.

[0363] Among the bit depth optimization-related information in Tables 27 to 45, the information related to the shift size of the input sample can represent the size of the right shift when applying truncated bit depth, and the size of the left shift when applying increased bit depth. The values ​​0, 1, 2, 3, 4, 5, 6, and 7 related to the shift size of the input sample can represent shifts of 1 bit, 2 bits, 3 bits, 4 bits, 5 bits, 6 bits, 7 bits, and 8 bits, respectively. The shift direction can be determined based on at least one of `eoi_quality_optimization_type_idc` or `eoi_bit_depth_optimization_type_flag`. Since there is no reason to define a corresponding value when there is no shift, the case where the shift is 0 can be left undefined.

[0364] [Table 46]

[0365] Table 46 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag and eoi_num_bit_depth_shift.

[0366] [Table 47]

[0367] Table 47 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, bit_depth_shift_luma, and bit_depth_shift_chroma.

[0368] [Table 48]

[0369] Table 48 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, bit_depth_shift_luma, bit_depth_shift_cb, and bit_depth_shift_cr.

[0370] [Table 49]

[0371] Table 49 shows the cases where bit depth optimization information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_bit_depth_minus8, and eoi_optimized_bit_depth_minus8.

[0372] [Table 50]

[0373] Table 50 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_bit_depth_minus8, and eoi_optimized_bit_depth_minus6.

[0374] [Table 51]

[0375] Table 51 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_bit_depth_minus8, and eoi_bit_depth_difference_from_input.

[0376] [Table 52]

[0377] Table 52 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_bit_depth, and eoi_optimized_bit_depth.

[0378] [Table 53]

[0379] Table 53 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag and eoi_bit_depth_for_task.

[0380] [Table 54]

[0381] Table 54 shows the cases where bit depth optimization related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, and eoi_optimized_chroma_bit_depth_minus8.

[0382] [Table 55]

[0383] Table 55 shows the cases where bit depth optimization related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus6, and eoi_optimized_chroma_bit_depth_minus6.

[0384] [Table 56]

[0385] Table 56 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth_minus8, eoi_input_chroma_bit_depth_minus8, eoi_luma_bit_depth_difference_from_input, and eoi_chroma_bit_depth_difference_from_input.

[0386] [Table 57]

[0387] Table 57 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth, eoi_input_chroma_bit_depth, eoi_optimized_luma_bit_depth, and eoi_optimized_chroma_bit_depth.

[0388] [Table 58]

[0389] Table 58 shows the cases where bit depth optimization information is defined as eoi_bit_depth_optimization_type_flag, eoi_luma_bit_depth_for_task, and eoi_bit_depth_for_task.

[0390] [Table 59]

[0391] Table 59 shows the cases where bit depth optimization related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus8, eoi_optimized_cb_bit_depth_minus8, and eoi_optimized_cr_bit_depth_minus8.

[0392] [Table 60]

[0393] Table 60 shows the cases where bit depth optimization related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_optimized_luma_bit_depth_minus6, eoi_optimized_cb_bit_depth_minus6, and eoi_optimized_cr_bit_depth_minus6.

[0394] [Table 61]

[0395] Table 61 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth_minus8, eoi_input_cb_bit_depth_minus8, eoi_input_cr_bit_depth_minus8, eoi_luma_bit_depth_difference_from_input, eoi_cb_bit_depth_difference_from_input, and eoi_cr_bit_depth_difference_from_input.

[0396] [Table 62]

[0397] Table 62 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_input_luma_bit_depth, eoi_input_cb_bit_depth, eoi_input_cr_bit_depth, eoi_optimized_luma_bit_depth, eoi_optimized_cb_bit_depth, and eoi_optimized_cr_bit_depth.

[0398] [Table 63]

[0399] Table 63 shows the cases where bit depth optimization-related information is defined as eoi_bit_depth_optimization_type_flag, eoi_luma_bit_depth_for_task, eoi_cb_bit_depth_for_task, and eoi_cr_bit_depth_for_task.

[0400] The bit depth optimization information from Tables 46 to 63 can be generated and encoded into the bitstream based on an EoiBitdepthOptimizationFlag value of 1. Conversely, if the EoiBitdepthOptimizationFlag value is 0, the bit depth optimization information from Tables 46 to 63 may not be encoded into the bitstream.

[0401] Alternatively, the bit depth optimization information from Tables 46 to 63 can be generated and encoded into the bitstream based on an EoiBitdepthTruncationFlag value of 1. On the other hand, if the EoiBitdepthTruncationFlag value is 0, the bit depth optimization information from Tables 46 to 63 may not be encoded into the bitstream.

[0402] Among the bit depth optimization-related information in Tables 46 to 63, the information related to the shift size of the input sample can indicate the size of the right shift when applying a truncated bit depth, and the size of the left shift when applying an increased bit depth. The values ​​0, 1, 2, 3, 4, 5, 6, and 7 related to the shift size of the input sample can represent shifts of 1 bit, 2 bits, 3 bits, 4 bits, 5 bits, 6 bits, 7 bits, and 8 bits, respectively. The shift direction can be determined based on at least one of `eoi_quality_optimization_type_idc` or `eoi_bit_depth_optimization_type_flag`. Since there is no reason to define a corresponding value when there is no shift, the case where the shift is 0 can be left undefined.

[0403] Figure 5 An illustrative configuration of an encoding device 200 for performing a method for generating a bitstream according to the present disclosure is shown.

[0404] Reference Figure 5 The encoding device 200 may include a receiver 500, a video compressor 510, an EOI generator 520, and a bitstream generator 530.

[0405] Receiver 500 can receive one or more encoded video frames.

[0406] The video compressor 510 can encode one or more received video frames to generate compressed video frames.

[0407] The EOI generator 520 can generate encoder optimization information. This encoder optimization information can be related to compressed video frames, and the methods and references used to generate it are... Figure 4 The descriptions are the same.

[0408] Bitstream generator 530 can generate bitstreams that include compressed video footage and encoder optimization information.

[0409] Figure 6 An example is illustrated of a method for reconstructing a video frame performed by a decoding device 300 according to the present disclosure.

[0410] It can receive bitstreams of encoded video footage from the S600.

[0411] It can reconstruct encoded video images from S610 bitstreams.

[0412] Video information related to encoded video frames can be extracted from the bitstream. The extracted video information can then be used to reconstruct the encoded video frames.

[0413] Additionally, SEI messages can be included in the NAL units of the bitstream. Encoder optimization information (EOI) S420 can be defined in the SEI message. Encoder optimization information can be associated with encoded video frames. Encoder optimization information can be included in the SEI message and signaled via the bitstream.

[0414] Encoder optimization information can include the EOI cancellation flag (eoi_cancel_flag). Based on eoi_cancel_flag being 0, optimization-related information can be included in the encoder optimization information, which is consistent with the reference... Figure 4 The descriptions are the same.

[0415] Additionally, bit depth optimization-related information can be defined in the encoder optimization information. This bit depth optimization-related information may include at least one of the following: information related to the shift size of the input sample, information related to the bit depth, or information related to the bit depth optimization type. This is consistent with the reference... Figure 4 The descriptions are the same.

[0416] Bit depth optimization information can be included in the encoder optimization information and signaled based on at least one of a predetermined variable or a bit depth optimization identifier.

[0417] Here, the predetermined variables may include at least one of the following: an 'EoiSpatialResamplingFlag' indicating whether spatial resampling optimization is applied; an 'EoiQualityOptimizationFlag' indicating whether spatial quality optimization is applied; an 'EoiBitdepthOptimizationFlag' indicating whether bit depth optimization is applied; or an 'EoiBitdepthTruncationFlag' indicating whether bit depth truncation is applied. Methods and references for deriving variables are provided. Figure 4 The descriptions are the same.

[0418] Additionally, the bit depth optimization identifier may include at least one of the spatial resampling type identifier (eoi_spatial_resampling_type_idc) or the quality optimization type identifier (eoi_quality_optimization_type_idc), which is consistent with the reference. Figure 4 The descriptions are the same.

[0419] Signaling and reference based on bit depth optimization-related information from at least one of predetermined variables or bit depth optimization identifiers. Figure 4 The descriptions are the same.

[0420] Figure 7 An illustrative configuration of a decoding device 300 for performing a method for reconstructing a video frame according to the present disclosure is shown.

[0421] Reference Figure 7 The decoding device 300 may include a receiver 700, a video information extractor 710, and a video reconstructor 720.

[0422] Receiver 700 can receive bitstreams including encoded video images.

[0423] The video information extractor 710 can extract video information related to encoded video frames from the bitstream. Additionally, the video information extractor 710 can extract SEI messages from the NAL units of the bitstream. The extracted SEI messages may include encoder optimization information.

[0424] The video reconstructor 720 can reconstruct encoded video footage based on extracted video information.

[0425] In the above embodiments, the method is described based on a flowchart as a series of steps or blocks. However, the corresponding embodiments are not limited to this order of steps. Some steps may occur simultaneously with other steps or in a different order, as described above. In addition, those skilled in the art will understand that the steps shown in the flowchart are not exclusive. Other steps may be included, or one or more steps in the flowchart may be deleted, without affecting the scope of the embodiments of this disclosure.

[0426] The methods described above according to embodiments of the present disclosure can be implemented in software, and the encoding and / or decoding devices according to the present disclosure can be included in an apparatus for performing image processing, such as a TV, computer, smartphone, set-top box, display device, etc.

[0427] In this disclosure, when the implementation is implemented as software, the above-described method can be implemented as a module (process, function, etc.) performing the above-described functions. The module can be stored in memory and can be executed by a processor. The memory can be internal or external to the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, logic circuitry, and / or data processing devices. The memory may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and / or other storage devices. In other words, the implementations described herein can be executed by implementation on a processor, microprocessor, controller, or chip. For example, the functional units shown in the various figures can be executed by implementation on a computer, processor, microprocessor, controller, or chip. In this case, information for implementation (e.g., information about instructions) or algorithms can be stored in a digital storage medium.

[0428] Furthermore, decoding and encoding devices employing embodiments of this disclosure can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices similar to video communication, mobile streaming devices, storage media, cameras, devices for providing video-on-demand (VOD) services, OTT (over-the-top) video devices, devices for providing internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle (including autonomous vehicle) terminals, aircraft terminals, ship terminals, etc.), and medical video devices, and can be used to process video signals or data signals. For example, OTT (over-the-top) video devices can include game consoles, Blu-ray players, internet-connected televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0429] Furthermore, the processing methods applying the embodiments of this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data with data structures according to the embodiments of this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical media storage devices. Additionally, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmission via the Internet). Furthermore, bitstreams generated by encoding methods can be stored in computer-readable recording media or transmitted via wired / wireless communication networks.

[0430] Furthermore, the embodiments of this disclosure can be implemented by a computer program product using program code, and the program code can be executed on a computer using the embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0431] Figure 8 Examples of content streaming systems to which embodiments of this disclosure can be applied are shown.

[0432] Reference Figure 8 A content streaming system that applies embodiments of the present disclosure may generally include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0433] An encoding server generates a bitstream by compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data and then sends it to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0434] A bitstream can be generated by an encoding method or bitstream generation method that applies the embodiments of this disclosure, and the streaming server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0435] A streaming server sends multimedia data to a user device via a web server based on a user request, and the web server acts as a medium to inform the user what services are available. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server, which controls the commands / responses between the various devices in the content streaming system.

[0436] A streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a specific time period.

[0437] Examples of user devices may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, touchscreen PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays (HMDs)), digital TVs, desktop computers, digital signage, etc.

[0438] In a content streaming system, each server can operate as a distributed server, and in this case, the data received from each server can be distributed and processed.

[0439] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined and implemented as an apparatus, and the technical features of the apparatus claims of this disclosure can be combined and implemented as a method. Furthermore, the technical features of the method claims and the apparatus claims of this disclosure can be combined and implemented as an apparatus, and the technical features of the method claims and the apparatus claims of this disclosure can be combined and implemented as a method.

Claims

1. A method comprising the following steps: Receive the video frame to be encoded; The received video footage is encoded to generate compressed video footage; Generate encoder optimization information related to the compressed video frame; as well as Generate a bitstream that includes the compressed video footage and the encoder optimization information. The encoder optimization information includes bit depth optimization related information.

2. The method according to claim 1, wherein, The bit depth optimization-related information includes information related to the shift size of the input sample.

3. The method according to claim 1, wherein, The bit depth optimization-related information includes at least one of the following: information related to the input bit depth, information related to the optimized bit depth, information related to the bit depth difference, or information related to the task bit depth.

4. The method according to claim 1, wherein, The bit depth optimization-related information includes information used to identify the type of bit depth optimization.

5. The method according to claim 1, wherein, The bit depth optimization-related information is generated based on a spatial resampling flag indicating whether spatial resampling optimization is applied.

6. The method according to claim 1, wherein, The bit depth optimization information is generated based on a flag indicating whether spatial quality optimization is applied.

7. The method according to claim 1, wherein, The bit depth optimization information is generated based on a flag indicating whether bit depth optimization is applied.

8. The method according to claim 1, wherein, The bit depth optimization information is generated based on a flag indicating whether bit depth truncation is applied.

9. The method according to claim 1, wherein, The bit depth optimization-related information is generated based on a bit depth optimization identifier used to identify bit depth optimization.

10. The method according to claim 9, wherein, The bit depth optimization identifier is a space resampling type identifier used to identify the type of spatial resampling used in the application.

11. The method according to claim 9, wherein, The bit depth optimization identifier is a quality optimization type identifier used to identify the type of spatial quality optimization applied to an application.

12. A method comprising the following steps: Receives a bitstream including encoded video frames; as well as The reconstructed video footage included in the bitstream The bitstream includes encoder optimization information related to the encoded video frames, and The encoder optimization information includes bit depth optimization related information.

13. A computer-readable storage medium for storing a bit stream generated by the method according to claim 12.

14. A method comprising the following steps: A bitstream is generated based on the following steps: receiving a video frame to be encoded; encoding the received video frame to generate a compressed video frame; generating encoder optimization information related to the compressed video frame; and generating the bitstream including the compressed video frame and the encoder optimization information; and Send data including the bit stream. The encoder optimization information includes bit depth optimization related information.

Images