Image decoding method and device
By deducing and adjusting the number of predicted symbols in the image encoding system, the problem of low encoding efficiency of high-resolution videos is solved, and more efficient symbol prediction and encoding effects are achieved, which are suitable for applications such as virtual reality and holograms.
Patent Information
- Application Number
- CN202380029112.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-08
- Filing Date
- 2023-01-19
- Publication Date
- 2024-11-08
AI Technical Summary
The prior art is difficult to effectively compress and transmit high-resolution, high-quality video/image data, especially in applications of immersive media such as virtual reality, artificial reality and holograms, resulting in increased transmission and storage costs.
By deducing the number of predicted symbols for the current block in the image encoding system and performing symbol prediction, in order to reduce the complexity of symbol prediction and improve encoding efficiency, the number of predicted symbols is adjusted according to the characteristics of the video and sliced intra-frame , inter-slice, luminance block, chroma block and slice header syntax elements that individually notify the number of predicted symbols.
It improves the accuracy and encoding efficiency of symbol prediction, reduces transmission and storage costs, and is suitable for the encoding requirements of high-resolution videos.
Smart Images

Figure CN118923109A_ABST
Abstract
Description
Technical Field
[0001] The present document relates to an image encoding technology, and more particularly, to an image decoding method and apparatus for encoding image information by predicting a sign of a transform coefficient in an image encoding system. Background Art
[0002] In recent years, there has been an increasing demand for high-resolution, high-quality video / images such as 4K or 8K or higher UHD (ultra-high definition) video / images in various fields. As video / image data becomes higher resolution and higher quality, the amount of information or bit volume transmitted increases relative to existing video / image data, and when image data is transmitted using a medium such as an existing wired or wireless broadband line or when video / image data is stored using an existing storage medium, transmission and storage costs increase.
[0003] In addition, interest and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality) content, and holograms are increasing, and broadcasting of videos / images having image characteristics different from real images such as game images is increasing.
[0004] Therefore, efficient image compression technology is needed to effectively compress, transmit, store and reproduce high-resolution, high-quality video / image information with the above-mentioned various characteristics. Summary of the invention
[0005] Technical issues
[0006] The present disclosure provides a method and apparatus for improving video / image coding efficiency.
[0007] Another technical task of this document is to provide a method and device for performing sign prediction on a current block.
[0008] Another technical task of this document is to provide a method and apparatus for adaptively deriving the number of symbols used for symbol prediction.
[0009] Technical Solution
[0010] According to an embodiment of the present document, there is provided an image decoding method performed by a decoding device. The method comprises: deriving a number of prediction symbols for a current block, deriving a sign of a residual coefficient of the number of prediction symbols by performing symbol prediction on the current block, deriving a residual sample of the current block based on the sign, and generating a reconstructed picture based on the residual sample.
[0011] According to another embodiment of the present document, a decoding device for performing image decoding is provided. The decoding device includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to: derive a number of prediction symbols for a current block, perform symbol prediction on the current block, derive a sign of a residual coefficient based on the number of prediction symbols, derive a residual sample of the current block based on the sign, and generate a reconstructed picture based on the residual sample.
[0012] According to another embodiment of the present document, a video encoding method performed by an encoding device is provided. The method includes: deriving a prediction symbol quantity for a current block, performing symbol prediction on the current block to derive a sign of a residual coefficient based on the prediction symbol quantity, deriving a residual sample of the current block based on the sign, and encoding image information including residual information of the current block and information about the prediction symbol quantity.
[0013] According to another embodiment of the present document, a video encoding device is provided. The encoding device includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to: derive a number of prediction symbols for a current block, perform symbol prediction on the current block to derive a sign of a residual coefficient based on the number of prediction symbols, derive a residual sample of the current block based on the sign, and encode image information including residual information of the current block and information about the number of prediction symbols.
[0014] According to another embodiment of the present document, a digital storage medium is provided, which can store a bit stream including residual information encoded by the image encoding method described in the present document and information about the number of prediction symbols.
[0015] According to another embodiment of the present document, a data transmission method for a video is provided. The data transmission method includes: obtaining a bit stream of image information including information about the number of prediction symbols and residual information of a current block, and transmitting data of the bit stream including the image information, wherein the image information includes information about the number of prediction symbols and the residual information.
[0016] According to another embodiment of the present document, a device for transmitting video data is provided. The device includes: at least one processor, the at least one processor is configured to obtain a bit stream of image information including information about the number of prediction symbols and residual information of a current block; and a transmitting unit, the transmitting unit is configured to transmit data of the bit stream including the image information, the image information including information about the number of prediction symbols and the residual information.
[0017] According to this document, the number of prediction symbols can be adaptively derived based on the last significant coefficient of the current block. This allows the symbol prediction accuracy to be improved by reflecting the characteristics of each block in the video, and the coding efficiency can be improved by reducing the complexity of symbol prediction.
[0018] According to this document, syntax elements for the number of prediction symbols can be signaled separately for intra slices and inter slices. This allows improving symbol prediction accuracy by reflecting the characteristics of the video, and can improve coding efficiency by reducing the complexity of symbol prediction.
[0019] According to this document, syntax elements for the number of prediction symbols can be signaled separately for luma blocks and chroma blocks. This allows improving symbol prediction accuracy by reflecting the characteristics of the video, and can improve coding efficiency by reducing the complexity of symbol prediction.
[0020] According to this document, a syntax element for the number of prediction symbols can be signaled in a slice header or a picture header. This allows the symbol prediction accuracy to be improved by reflecting the characteristics of each slice or picture in the video, and the coding efficiency can be improved by reducing the complexity of symbol prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 An example of a video / image encoding device to which an embodiment of the present disclosure can be applied is briefly illustrated.
[0022] Figure 2 is a schematic diagram illustrating a configuration of a video / image encoding device to which an embodiment of the present disclosure can be applied.
[0023] Figure 3 : is a schematic diagram illustrating a configuration of a video / image decoding device to which an embodiment of the present disclosure can be applied.
[0024] Figure 4 Illustrate quantization in a quantizer of a video / image coding standard.
[0025] Figure 5 Neighboring samples used to derive the cost of a symbol combination hypothesis are exemplarily illustrated.
[0026] Figure 6 The figure shows an implementation of deriving the number of prediction symbols based on a symbol prediction region of an N×N transform block.
[0027] Figure 7 An actual symbol prediction area of a 16×4 transform block is exemplarily illustrated when the symbol prediction area is derived as 8×8.
[0028] Figure 8An 8×8 transform block to which a diagonal scan order is applied is exemplarily illustrated.
[0029] Fig. 9 An embodiment in which the number of prediction symbols is derived based on the position index of the last significant coefficient in the 8×8 transform block when the specific threshold is 16 is illustrated.
[0030] Fig.10 The diagram illustrates an embodiment of deriving the number of prediction symbols based on the position index of the last significant coefficient in a 16×4 transform block.
[0031] Fig.11 An embodiment of deriving the number of prediction symbols based on the position index of the last significant coefficient in the 8×16 transform block when the specific threshold is 64 is illustrated.
[0032] Fig.12 A video encoding method performed by an encoding device according to the present document is schematically illustrated.
[0033] Fig.13 An encoding device that performs a video encoding method according to the present document is schematically illustrated.
[0034] Fig.14 A video decoding method performed by a decoding device according to this document is schematically illustrated.
[0035] Fig.15 A decoding device that performs a video decoding method according to the present document is schematically illustrated.
[0036] Fig.16 The content streaming system architecture to which the embodiments of this document are applied is exemplarily illustrated. DETAILED DESCRIPTION
[0037] The present disclosure can be modified in various forms, and its specific embodiments will be described and illustrated in the accompanying drawings. However, the embodiments are not intended to limit the present disclosure. The terms used in the following description are only used to describe specific embodiments and are not intended to limit the present disclosure. As long as it is clearly understood in different ways, the expression of the singular includes the expression of the plural. Terms such as "including" and "having" are intended to indicate the presence of features, numbers, steps, operations, elements, components, or combinations thereof used in the following description, and therefore it should be understood that the possibility of the presence or addition of one or more different features, numbers, steps, operations, elements, components, or combinations thereof is not excluded.
[0038] In addition, the elements in the drawings described in the present disclosure are drawn independently for the convenience of explaining different specific functions, and do not mean that these elements are embodied by independent hardware or independent software. For example, two or more elements in the elements can be combined to form a single element, or an element can be divided into multiple elements. The embodiments of combining elements and / or dividing elements belong to the present disclosure without departing from the concept of the present disclosure.
[0039] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In addition, throughout the accompanying drawings, like reference numerals may be used to indicate like elements, and the same description of the like elements will be omitted.
[0040] Figure 1 An example of a video / image encoding device to which an embodiment of the present disclosure can be applied is briefly illustrated.
[0041] refer to Figure 1 The video / image coding system may include a first device (source device) and a second device (receiving device). The source device may send coded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or a network.
[0042] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be configured as a separate device or an external component.
[0043] The video source can obtain the video / image by capturing, synthesizing or generating the process of the video / image. The video source may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer and a smart phone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a process that generates relevant data.
[0044] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation and quantization to achieve compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.
[0045] The transmitter may transmit the encoded image / image information or data output in the form of a bit stream to the receiver of the receiving device in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter may include an element for generating a media file in a predetermined file format, and may include an element for transmitting through a broadcast / communication network. The receiver may receive / extract a bit stream and transmit the received bit stream to a decoding device.
[0046] The decoding device may decode the video / image by performing a series of processes such as dequantization, inverse transformation, and prediction corresponding to the operations of the encoding device.
[0047] The renderer may render the decoded video / image. The rendered video / image may be displayed by a display.
[0048] The present disclosure relates to video / image coding. For example, the methods / implementations disclosed in the present disclosure may be applied to methods disclosed in Versatile Video Coding (VVC), EVC (Basic Video Coding) standard, AOMedia Video 1 (AV1) standard, 2nd generation Audio Video Coding standard (AVS2) or next generation video / image coding standard (e.g., H.267, or H.268, etc.).
[0049] The present disclosure presents various embodiments of video / image encoding, and unless otherwise mentioned, the embodiments may be performed in combination with each other.
[0050] In the present disclosure, a video may refer to a series of images in time. Generally, a picture refers to a unit representing an image in a specific time zone, and a sub-picture / slice / tile is a unit that constitutes a part of a picture in coding. A sub-picture / slice / tile may include one or more coding tree units (CTUs). A picture may be composed of one or more sub-pictures / slices / tiles. A picture may be composed of one or more tile groups. A tile group may include one or more tiles. A brick may represent a rectangular area of a CTU row within a tile in a picture. A tile may be partitioned into a plurality of tiles, each tile consisting of one or more CTU rows within a tile. Tiles that are not partitioned into a plurality of tiles may also be referred to as tiles. A tile scan may sort the CTUs partitioned in a picture in a specific order, wherein the CTUs are sorted continuously in the tile by a CTU raster scan, the tiles within the tile are sorted continuously by a raster scan of the tile, and the tiles in the picture are sorted continuously by a raster scan of the tile of the picture. In addition, a sub-picture may represent a rectangular area of one or more slices within a picture. That is, the sub-picture contains one or more slices that cover a rectangular area of the picture together. A tile is a rectangular area of a CTU within a specific tile column and a specific tile row in a picture. A tile column is a rectangular area of a CTU whose height is equal to the height of the picture and whose width is specified by a syntax element in a picture parameter set. A tile row is a rectangular area of a CTU whose height is specified by a syntax element in a picture parameter set and whose width is equal to the width of the picture. Tile scanning is a specific order sorting of CTUs that partition a picture, wherein CTUs can be sorted continuously in tiles by CTU raster scanning, and tiles in a picture can be sorted continuously by raster scanning of tiles of a picture. A slice includes an integer number of tiles of a picture that can be exclusively contained in a single NAL unit. A slice may consist of multiple complete tiles or only of a complete sequence of tiles of a continuous sequence. In the present disclosure, tile groups and slices may be used interchangeably. For example, in the present disclosure, a tile group / tile group header may be referred to as a slice / slice header.
[0051] A pixel or a picture element (pel) may represent the smallest unit constituting a picture (or image). In addition, a "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.
[0052] A unit may represent a basic unit of image processing. A unit may include at least one of a specific area of a picture and information related to the area. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or an area. In general, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients of M columns and N rows.
[0053] In this specification, "A or B" may mean "only A", "only B", or "A and B". In other words, in this specification, "A or B" may be interpreted as "A and / or B". For example, "A, B or C" herein means "only A", "only B", "only C", or "any one and any combination of A, B, and C".
[0054] A slash ( / ) or a comma used in this specification may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "A and B". For example, "A,B,C" may mean "A, B, or C".
[0055] In the present specification, "at least one of A and B" may mean "only A", "only B", or "both A and B". In addition, in the present specification, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted as being the same as "at least one of A and B".
[0056] In addition, in the present specification, "at least one of A, B, and C" means "only A", "only B", "only C", or "any combination of A, B, and C". In addition, "at least one of A, B, or C" or "at least one of A, B and / or C" may mean "at least one of A, B, and C".
[0057] In addition, brackets used in this specification may refer to "for example". Specifically, when "prediction (intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction". In other words, "prediction" in this specification is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when "prediction (i.e., intra-frame prediction)" is indicated, "intra-frame prediction" may be proposed as an example of "prediction".
[0058] In this specification, technical features described separately in one figure may be implemented separately or simultaneously.
[0059] The following figures are created to explain specific examples of this specification. Since the names of specific devices or the names of specific signals / messages / fields described in the figures are presented by way of example, the technical features of this specification are not limited to the specific names used in the following figures.
[0060] Figure 2 is a schematic diagram illustrating a configuration of a video / image encoding device to which an embodiment of the present disclosure can be applied. Hereinafter, a video encoding device may include an image encoding device.
[0061] refer to Figure 2 , the encoding device 200 includes an image segmenter 210, a predictor 220, a residual processor 230 and an entropy encoder 240, an adder 250, a filter 260 and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234 and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be called a reconstructor or a reconstructed block generator. According to an embodiment, the image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250 and the filter 260 may be composed of at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) or may be composed of a digital storage medium. The hardware component may also include a memory 270 as an internal / external component.
[0062] The image divider 210 may divide the input image (or picture or frame) input to the encoding device 200 into one or more processors. For example, the processor may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure. For example, a coding unit may be divided into a plurality of coding units with a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure may be applied first, and then a binary tree structure and / or a ternary structure may be applied. Alternatively, a binary tree structure may be applied first. The encoding process according to the present disclosure may be performed based on a final coding unit that is no longer divided. In this case, the maximum coding unit may be used as the final coding unit based on coding efficiency according to image characteristics, or if necessary, the coding unit may be recursively divided into coding units with a deeper depth and a coding unit with an optimal size may be used as the final coding unit. Here, the encoding process may include a process of prediction, transformation, and reconstruction, which will be described later. As another example, the processor may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be split or divided from the above-mentioned final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.
[0063] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an M×N block may represent a set of samples or transform coefficients consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, may represent a pixel / pixel value of a luminance component only, or may represent a pixel / pixel value of a chrominance component only. A sample may be used as a term corresponding to one picture (or image) of a pixel or a pixel element.
[0064] In the encoding device 200, the prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 is subtracted from the input image signal (original block, original sample array) to generate a residual signal (residual block, residual sample array) and the generated residual signal is sent to the transformer 232. In this case, as shown in the figure, the unit for subtracting the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) in the encoder 200 can be called a subtractor 231. The predictor can perform prediction on the block to be processed (hereinafter referred to as the current block) and generate a prediction block including the prediction samples of the current block. The predictor can determine whether to apply intra prediction or inter prediction in units of the current block or CU. As described later in the description of each prediction mode, the predictor can generate various information related to the prediction, such as prediction mode information, and send the generated information to the entropy encoder 240. The information about the prediction can be encoded in the entropy encoder 240 and output in the form of a bitstream.
[0065] The intra-frame predictor 222 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block, or can be far away from the current block. In intra-frame prediction, the prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the setting. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.
[0066] The inter-frame predictor 221 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. Here, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a co-located CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 221 may use the motion information of the neighboring block as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be indicated by signaling a motion vector difference.
[0067] The predictor 220 may generate a prediction signal based on various prediction methods described below. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply both intra prediction and inter prediction at the same time. This may be referred to as intra-frame combined prediction (CIIP). In addition, the predictor may predict a block based on an intra-block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode may be used for content image / video encoding of games, etc., such as screen content coding (SCC). IBC basically performs prediction in the current picture, but may be performed similarly to inter prediction because the reference block is derived in the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure. The palette mode may be considered as an example of intra coding or intra prediction. When the palette mode is applied, the sample values within the picture may be signaled based on information about the palette table and the palette index.
[0068] The prediction signal generated by the predictor (including the inter-frame predictor 221 and / or the intra-frame predictor 222) can be used to generate a reconstruction signal or to generate a residual signal. The transformer 232 can generate a transform coefficient by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT represents a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform generated based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process can be applied to square pixel blocks of the same size, or can be applied to blocks of variable size other than square.
[0069] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the block type quantized transform coefficients into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form. Information about the transform coefficients may be generated. The entropy encoder 240 may perform various encoding methods, such as, for example, exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 240 may encode information required for video / image reconstruction (e.g., values of syntax elements, etc.) other than the quantized transform coefficients together or separately. Encoded information (e.g., encoded video / image information) may be transmitted or stored in units of NAL (network abstraction layer) in the form of a bitstream. The video / image information may also include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. In the present disclosure, information and / or syntax elements sent / signaled from the encoding device to the decoding device may be included in the video / picture information. The video / image information may be encoded by the above-mentioned encoding process and included in the bitstream. The bitstream may be sent through a network, or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that sends a signal output from the entropy encoder 240 and / or a storage unit (not shown) that stores the signal may be included as an internal / external element of the encoding device 200, alternatively, the transmitter may be included in the entropy encoder 240.
[0070] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients using the dequantizer 234 and the inverse transformer 235. The adder 250 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). If the block to be processed has no residual (such as the case where the skip mode is applied), the prediction block can be used as a reconstructed block. The adder 250 can be called a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.
[0071] Furthermore, during picture encoding and / or reconstruction, luma mapping and chroma scaling (LMCS) may be applied.
[0072] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 270 (specifically, the DPB of the memory 270). Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information related to filtering, and send the generated information to the entropy encoder 240, as described later in the description of various filtering methods. The information related to filtering can be encoded by the entropy encoder 240 and output in the form of a bit stream.
[0073] The modified reconstructed picture sent to the memory 270 may be used as a reference picture in the inter-frame predictor 221. When the inter-frame prediction is applied by the encoding device, prediction mismatch between the encoding device 200 and the decoding device may be avoided, and encoding efficiency may be improved.
[0074] The DPB of the memory 270 may store a modified reconstructed picture used as a reference picture in the inter-frame predictor 221. The memory 270 may store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of a reconstructed block in the picture. The stored motion information may be sent to the inter-frame predictor 221 and used as motion information of a spatially adjacent block or motion information of a temporally adjacent block. The memory 270 may store reconstructed samples of a reconstructed block in the current picture and may transmit the reconstructed samples to the intra-frame predictor 222.
[0075] Figure 3is a schematic diagram illustrating a configuration of a video / image decoding device to which an embodiment of the present disclosure can be applied.
[0076] refer to Figure 3 , the decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to an embodiment, the entropy decoder 310, the residual processor 320, the predictor 330, the adder 340, and the filter 350 may be composed of hardware components (e.g., a decoder chipset or a processor). In addition, the memory 360 may include a decoded picture buffer (DPB), or may be composed of a digital storage medium. The hardware component may also include a memory 360 as an internal / external component.
[0077] When a bit stream including video / image information is input, the decoding apparatus 300 can be used with Figure 2 The image is reconstructed correspondingly to the processing of the video / image information in the encoding device. For example, the decoding device 300 can derive the unit / block based on the block segmentation related information obtained from the bit stream. The decoding device 300 can perform decoding using a processor applied in the encoding device. Therefore, the decoding processor can be, for example, a coding unit, and the coding unit can be divided from the coding tree unit or the maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. One or more transform units can be derived from the coding unit. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced by a reproduction device.
[0078] The decoding device 300 may receive the bit stream from Figure 2The received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The decoding device may also decode the picture based on the information about the parameter set and / or the general constraint information. The signaled / received information and / or syntax elements described later in this disclosure may be decoded by a decoding process and obtained from the bitstream. For example, the entropy decoder 310 decodes the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and outputs syntax elements required for image reconstruction and quantized values of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method may receive a bin (binary bit) corresponding to each syntax element in the bitstream, use the decoding target syntax element information, the decoding information of the decoding target block, or the information of the symbol / bin decoded in the previous stage to determine the context model, and arithmetically decode the bin by predicting the probability of occurrence of the bin according to the determined context model, and generate a symbol corresponding to the value of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method may update the context model by using the decoded symbol / bin information for the context model of the next symbol / bin. Information related to prediction among the information decoded by the entropy decoder 310 may be provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value (that is, quantized transform coefficient and related parameter information) on which entropy decoding is performed in the entropy decoder 310 may be input to the residual processor 320. The residual processor 320 may derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded by the entropy decoder 310 may be provided to the filter 350. Meanwhile, a receiver (not shown) for receiving a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300, or the receiver may be a component of the entropy decoder 310. Meanwhile, the decoding device according to the present disclosure may be referred to as a video / image / picture decoding device, and the decoding device may be classified into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of the dequantizer 321, the inverse transformer 322, the adder 340, the filter 350, the memory 360, the inter-frame predictor 332, and the intra-frame predictor 331.
[0079] The dequantizer 321 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 321 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding device. The dequantizer 321 may dequantize the quantized transform coefficients by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficients.
[0080] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).
[0081] The predictor may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information on prediction output from the entropy decoder 310, and may determine a specific intra / inter prediction mode.
[0082] The predictor 320 may generate a prediction signal based on various prediction methods described below. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. This may be referred to as combined inter and intra prediction (CIIP). In addition, the predictor may predict a block based on an intra block copy (IBC) prediction mode or a palette mode. The IBC prediction mode or the palette mode may be used for content image / video encoding of games, etc., such as screen content coding (SCC). IBC basically performs prediction in the current picture, but may be performed similarly to inter prediction because a reference block is derived in the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, the sample values within the picture may be signaled based on information about the palette table and the palette index.
[0083] The intra-frame predictor 331 can predict the current block by referring to samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block, or can be far away from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.
[0084] The inter-frame predictor 332 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on neighboring blocks, and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating a mode of inter-frame prediction for the current block.
[0085] The adder 340 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331). If the block to be processed has no residual (for example, when the skip mode is applied), the prediction block can be used as the reconstructed block.
[0086] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering as described below, or may be used for inter prediction of the next picture.
[0087] In addition, luma mapping and chroma scaling (LMCS) can be applied during picture decoding.
[0088] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 360 (specifically, the DPB of the memory 360). Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.
[0089] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame predictor 332. The memory 360 can store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the reconstructed block in the picture. The stored motion information can be sent to the inter-frame predictor 332 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 can store the reconstructed samples of the reconstructed block in the current picture, and can transmit the reconstructed samples to the intra-frame predictor 331.
[0090] In the present disclosure, the embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may be the same as the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300 or may be respectively applied to correspond to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300. The same contents may also be applied to the inter-frame predictor 332 and the intra-frame predictor 331.
[0091] In addition, the encoding device and the decoding device can perform sign prediction on the residual coefficient. For example, the encoding device and the decoding device can predict the signs of a limited number of residual coefficients for each block. Here, the block can be a CU or a TU, and the residual coefficient can be referred to as a transform coefficient.
[0092] For example, when predicting n symbols in the current block, the encoding device and the decoding device are able to calculate the costs of 2n symbol combination hypotheses, among which the n symbols can be used to predict the symbol combination hypothesis with the minimum cost.
[0093] In addition, for example, the encoding device may determine a symbol to be predicted among the symbols of the residual coefficients in the current block and predict the determined symbol. The encoding device may determine the symbol to be predicted according to the RDO decision made among the symbols of the residual coefficients of the current block.
[0094] Generally, since the sign error of the transform coefficient with a larger value has a relatively large impact on the reconstructed block, the sign of the transform coefficient with a larger value can be predicted relatively easily. A sorting-based method can be applied to determine the sign to be predicted based on the above logic. For example, one way to sort the transform coefficients of the transform block can be based on the absolute value of the transform coefficient level. However, since two quantizers are used in video / image coding standards such as the VVC standard and ECM-2.0, the transform coefficient level may not accurately reflect the quantization. Therefore, the best performance cannot be obtained by sorting by transform coefficient level.
[0095] Figure 4 Quantization in a quantizer of a video / image coding standard is shown. Figure 4 The horizontal axis of represents the dequantized transform coefficient, and the vertical axis represents the level of the dequantized transform coefficient. Figure 4 , for the same level value, the dequantized transform coefficients may be different due to two quantizers ( Figure 4 The Q0 and Q1 in the figure are different as shown below.
[0096] - Case 1: Level = 2, quantizer Q 0 . Dequantized transform coefficient = 4△ k
[0097] - Case 2: Level = 2, quantizer Q 1 . Dequantized transform coefficient = 3△ k
[0098] The above cases are all level 2, but the dequantized transform coefficients (4△ k ) can be greater than the dequantized transform coefficient (3△ k ). In a method of deriving coefficients to be sign predicted which will be described later based on the above observation, transform coefficient sorting may be performed based on qIdx values (dequantized transform coefficient=qIdx×Δk).
[0099] The qIdx at the transform coefficient level varies depending on the DQ state and can be calculated as follows:
[0100] [Formula 1]
[0101] qIdx=(abs(level)<<1)-(state&1)
[0102] After the absolute values of the transform coefficient levels of the current block are decoded, the current block can be sorted based on the qIdx value. For example, the transform coefficient with the highest qIdx value can be located at the beginning of the current sorted block. The symbols of the first maxNumPredSigns symbols in the sorted current block can be predicted, and the remaining symbols can be EP encoded. That is, the transform coefficients of the current block can be sorted based on the qIdx value, and the n transform coefficients can be determined as the transform coefficients to be symbol predicted in the order of the qIdx value. Here, the current block can be a transform block (TB).
[0103] Then, for example, if n (n<=maxNumPredSigns) symbols are predicted in the current block, the encoding device and the decoding device can use the boundary cost measurement of each of the 2n symbol combination hypotheses to perform boundary reconstruction corresponding to the 2n symbol combination hypotheses 2n times. For example, if n (n<=maxNumPredSigns) symbols are predicted in the current block, 2n simplified boundary reconstructions can be performed as described below. One reconstruction can be performed for each specific combination of symbols of the n residual coefficients. For example, for a specific reconstruction, the inverse transform added to the block prediction can be used to regenerate only the leftmost pixel and the topmost pixel of the block. The first inversion (vertical inversion) is completed, but the second inversion (horizontal inversion) can be faster because it only needs to generate the leftmost pixel and the topmost pixel output. In addition, for example, the number of inversion operations performed can be reduced by using a system of 'templates'. When the symbol is assumed to be 1, the template can be defined as an inverse transform block and inversely transformed from the coordinates of the coefficient to predict the symbol. The operation using the template can be performed as follows.
[0104] 1. Inverse transform can be performed by setting one of the coefficients of the symbol to be predicted to 1 and the remaining coefficients to 0. That is, for the symbol of n coefficients to be predicted, the kth (k is 0 to n-1) prediction coefficient can be set to 1 and the remaining coefficients can be set to 0, thereby setting it as a template. A block when all n templates are added can be defined as the first symbol combination hypothesis.
[0105] 2. A symbol combination hypothesis in which only one symbol represents a negative number can be constructed using the first symbol combination hypothesis and each template. In addition, other symbol combination hypotheses can also be constructed using the already constructed symbol combination hypotheses and templates.
[0106] For example, when predicting three symbols, the storage / recovery and template application can be as shown in the following table. That is, the template and symbol combination assumptions when predicting three symbols can be as shown in the following table.
[0107] [Table 1]
[0108] Template Name How to create T001 inv xform single +ve first sign - hidden coefficient T010 inv xform single +ve second sign - hidden coefficient T100 inv xform single +ve third sign - hidden coefficient
[0109] [Table 2]
[0110] Assumptions How to create Store for later reuse as H000 inv xform all coefficients of added prediction H000 H001 H000-2*T001 H010 H000-2*T010 H010 H011 H010-2*T001 H100 H000-2*T100 H100 H101 H100-2*T001 H110 H100-2*T010 H110 H111 H110-2*T001
[0111] Alternatively, the symbol combination hypothesis can be constructed using a Gray code method. That is, the kth symbol combination hypothesis can be composed of the k-1th symbol combination hypothesis and one of n templates (where k is 1 to n-1). The first symbol combination hypothesis (where all symbols are positive) can be constructed as the sum of all templates, as previously described. For example, when predicting four symbols, the storage / recovery and template application can be as shown in the following table. That is, the template and symbol combination hypothesis when predicting four symbols can be as shown in the following table.
[0112] [Table 3]
[0113]
[0114]
[0115] Referring to Table 3, the boundary for the next symbol combination hypothesis reconstruction can be started by taking the previous hypothesis, and only a single predicted symbol needs to be changed to reconstruct the desired current hypothesis. This symbol change can then be approximated by doubling and adding / subtracting on the symbol combination hypothesis boundary of the template corresponding to the predicted symbol. The approximation disclosed in the above table can only be used for the symbol prediction process, not for the final reconstruction process. That is, the approximation can be a reconstruction value generated for the symbol prediction process rather than the reconstruction sample generation process of the current block.
[0116] The cost of the symbol combination hypothesis can be calculated as follows.
[0117] There may be a cost associated with each signature combination hypothesis that corresponds to a notion of image continuity across block boundaries. That is, the cost of a sign combination hypothesis may correspond to a notion of image continuity at block boundaries for the sign combination hypothesis. By minimizing the cost, a sign prediction value can be derived.
[0118] Figure 5 Neighboring samples used to derive the cost of a symbol combination hypothesis are shown exemplarily.
[0119] Figure 5 The previously restored neighboring pixels and the leftmost pixel and the topmost pixel of the current block for cost calculation are disclosed. The neighboring pixels may include left neighboring pixels p-2,y and p-1,y and top neighboring pixels px,-2 and px,-1. Here, when the width and height of the current block are W and H, y may be 0 to H-1 and x may be 0 to W-1. For example, the cost may be derived as follows:
[0120] [Formula 2]
[0121]
[0122] As in the above formula, the cost function for deriving the cost can be defined as the sum of the absolute second-order derivatives of the residual domain for the upper row and left column of the current block. Here, R can be the reconstructed neighboring pixels, P can be the predicted pixels of the current block, and r can be the residual hypothesis. Specifically, w can represent the width of the current block, h can represent the height of the current block, and R x,-2 It can represent the upper neighboring pixel at the position (x, -2), R x,-1 It can represent the upper neighboring pixel at the position (x, -1), P x,0 It can represent the predicted pixel at position (x,0), r x,0 It can represent the hypothetical residual pixel at the position (x,0), R -2,y It can represent the left neighboring pixel at the position (-2, y), R -1,y It can represent the left neighboring pixel at the (-1, y) position, P 0,y It can represent the predicted pixel at position (0, y), r 0,y It can represent the hypothetical residual pixel at the position (0, y). In addition, (-R -2 +2R -1 -P 0 ) term can be computed only once per block, and then the cost of the symbol combination assumption can be calculated by subtracting only the residual assumption.
[0123] In addition, the sign prediction of the residual coefficients can be applied only partially in the area of the TU. The sign prediction area of the transform block can be performed up to 32×32 blocks. For example, the sign of the upper left M×N block of the transform block can be predicted. The values of M and N can be calculated as follows:
[0124] [Formula 3]
[0125] M = min(width, 32)
[0126] N = min(height,32)
[0127] Here, the width may represent the width of the transform block, and the height may represent the height of the transform block. In addition, the symbol prediction may be applied not only to the coefficients after the main transform (e.g., MTS, DCT2, etc.), but also to the coefficients after the LFNST mode. In addition, in order to achieve a better gain / complexity tradeoff, for example, for one LFNST transform block, up to 4 (i.e., n=4) transform coefficients. Symbols may be predicted.
[0128] For each symbol to be predicted, the encoding device can search for the lowest cost symbol combination hypothesis that matches the true value of the transmitted symbol. On the other hand, for example, the symbol residual may not be initially transmitted, and the symbol combination hypothesis with the lowest cost may be simply searched. The predicted value of the current symbol of the symbol can be obtained from the searched symbol combination hypothesis. That is, for example, the values of n symbols can be predicted by the symbol combination hypothesis with the lowest cost among the symbol combination hypotheses.
[0129] If the prediction corresponds to the true value of the symbol, the symbol residual may be sent as "0", otherwise it may be sent as "1". That is, if the predicted value of the symbol is correct, a syntax element for the symbol with a value of 0 may be sent, and if the predicted value of the symbol is incorrect, a syntax element for the symbol with a value of 1 may be sent. This syntax element may also be referred to as a symbol residual.
[0130] The decoding device can parse coefficients, symbols, and symbol residuals as part of the parsing process. The symbol residual can be parsed at the end of the TU, and then the decoding device can know the absolute values of all coefficients. Therefore, the decoding device can determine which symbol is expected, and based on the dequantized coefficient values for each predicted symbol, the context model for parsing the symbol prediction residual can be determined.
[0131] Furthermore, the knowledge of whether the prediction was "correct" or "incorrect" can simply be stored as part of the CU data for the block being parsed.The actual sign of the residual coefficients may not be known at this point.
[0132] During the reconstruction process, the decoding device may perform operations similar to those of the encoding device. That is, the decoding device may perform the above-mentioned symbol combination hypothesis generation and cost calculation operations for symbol prediction.
[0133] For example, for n symbols predicted in a TU, the decoding device can determine the cost of the symbol combination hypothesis by performing boundary reconstruction 2n times. Then, the actual symbol to be applied to the residual coefficient with the predicted symbol can be determined by an XOR operation of the following values.
[0134] 1. The predicted value of the symbol.
[0135] 2. The "correct" or "incorrect" data is stored in the CU through bitstream parsing.
[0136] Furthermore, in each TU in which the symbol of a coefficient is “hidden” using the existing symbol data hiding mechanism, symbol prediction can be performed by considering the corresponding coefficient (ie, the coefficient of the hidden symbol) as “unavailable” in the symbol prediction process and using only other coefficients.
[0137] In addition, symbol prediction can be performed on the n transform coefficients as described above. As described above, in symbol prediction, the residual coefficients within the transform block are sorted from largest to smallest based on the quantization index (qIdx) value, and the signs of the n residual coefficients can be predicted starting from the residual coefficient with the maximum value. The value of n can be determined based on the maxNumPredSigns value signaled at the SPS level. maxNumPredSigns can represent the maximum number of predicted symbols that can be predicted in the symbol prediction technique. The actual number n of predicted symbols can be determined taking into account the number of non-zero coefficients within the symbol prediction region, as shown in the following formula.
[0138] [Formula 4]
[0139] n=min(numNzCoeffSPArea,maxNumPredSigns)
[0140] Here, n may represent the number of coefficients for performing symbol prediction, numNzCoeffSPArea may represent the number of non-zero coefficients within the symbol prediction region, and maxNumPredSigns may represent the maximum number of prediction symbols that may be predicted in symbol prediction.
[0141] For n coefficients, there may be 2n symbol combination hypotheses, where each coefficient has a positive or negative sign. The simplified boundary reconstruction described above may be performed for these 2n symbol combination hypotheses, and the cost of the symbol combination hypothesis may be calculated. The symbol combination hypothesis with the minimum cost may be determined as the best symbol prediction value.
[0142] Here, the symbol prediction area can have a fixed value, or the width (SPAreaWidth) and height (SPAreaHeight) of the symbol prediction area can be derived based on factors such as intraPeriod, QP, resolution, and whether the IBC mode is applied. In addition, for example, even if the symbol prediction area is derived, the actual symbol prediction area may vary according to the size of the current block, as shown in the following formula.
[0143] [Formula 5]
[0144] EffectiveSPAreaWidth=min(TBWidthSPAreaWidth)
[0145] EffectiveSPAreaHeight=min(TBHeight,SPAreaHeight)
[0146] Here, EffectiveSPAreaWidth may represent the width of the actual symbol prediction region of the transform block, TBWidth may represent the width of the transform block, and SPAreaWidth may represent the width of the symbol prediction region. Similarly, EffectiveSPAreaHeight may represent the height of the actual symbol prediction region of the transform block, TBHeight may represent the height of the transform block, and SPAreaHeight may represent the height of the symbol prediction region.
[0147] For example, referring to Formula 5, even if the symbol prediction area is set to 8×8, the actual symbol prediction area of the 16×4 transform block can be derived as 8×4 according to Formula 5, where EffectiveSPAreaWidth is 8 and EffectiveSPAreaHeight is 4.
[0148] However, if symbol prediction is performed for all blocks with the same number of predictions without considering characteristics of each block, computational complexity may increase and encoding efficiency may decrease.
[0149] Therefore, this document proposes an implementation method for improving symbol prediction. For example, this document suggests an implementation method for adaptively adjusting the number of prediction symbols applied in residual coding based on the position of the last significant coefficient in the current transform block. For example, one implementation method proposes a method for using different numbers of prediction symbols based on whether the position of the last significant coefficient in the current transform block is within a specific area. Here, for example, an arbitrary area used as a criterion for changing the number of prediction symbols can have a size with an arbitrary width and an arbitrary height (ThWidth, ThHeight). In other words, an arbitrary threshold size can be used as a criterion for changing the number of prediction symbols. Alternatively, for example, the actual symbol prediction area (EffectiveSPAreaWidth and EffectiveSPAreaHeight) can be used as an area that acts as a criterion for changing the number of prediction symbols. This document does not limit the method for determining an arbitrary area.
[0150] Figure 6 The figure shows an implementation method for deriving the number of prediction symbols based on the symbol prediction region of the N×N transform block. Figure 6 As shown in (a), when the last significant coefficient in the current transform block is located in the actual symbol prediction area based on the (0,0) coordinate of the current transform block (that is, when the x coordinate of the last significant coefficient is less than EffectiveSPAreaWidth and the y coordinate is less than EffectiveSPAreaHeight), the number of prediction symbols for the current transform block can be derived as n 1 On the contrary, Figure 6As shown in (b), when the last significant coefficient is outside the actual symbol prediction area, the number of prediction symbols for the current transform block can be derived as n 2 .
[0151] Specifically, for example, when the symbol prediction region is derived as 8×8, the actual symbol prediction region of the 16×4 transform block may be derived as 8×4.
[0152] Figure 7 The actual symbol prediction area of the 16×4 transform block is exemplarily illustrated when the symbol prediction area is derived as 8×8. Figure 7 As shown, when the actual symbol prediction area is derived, if the last significant coefficient of the current transform block is located in the 8×4 actual symbol prediction area, the number of prediction symbols can be derived as n 1 If the last significant coefficient is outside this region, the number of prediction symbols can be derived as n 2 .
[0153] Subsequently, the actual number of prediction symbols can be derived by considering the number of non-zero coefficients in the actual symbol prediction region of the current transform block. For example, if the number of prediction symbols for the current transform block is derived as n 1 , then it can be derived as shown below.
[0154] [Formula 6]
[0155] n=min(numNzCoeffSPArea,n)
[0156] Here, n represents the actual number of prediction symbols, and numNzCoeffSPArea represents the number of non-zero coefficients in the actual symbol prediction region. Referring to Formula 6, if the number of prediction symbols for the current transform block is derived as n 1 , then n 1 The smaller value between the number of non-zero coefficients in the actual symbol prediction region will be determined as the actual number of prediction symbols for the current transform block.
[0157] In addition, for example, if the number of prediction symbols for the current transform block is derived as n 2 , then it can be derived as shown below.
[0158] [Formula 7]
[0159] n=min(numNzCoeffSPArea,n 2 )
[0160] Here, n represents the actual number of prediction symbols, and numNzCoeffSPArea represents the number of non-zero coefficients in the actual symbol prediction region. Referring to Formula 7, if the number of prediction symbols for the current transform block is derived as n 2 , then n 2 The smaller value between the number of non-zero coefficients in the actual symbol prediction region will be determined as the actual number of prediction symbols for the current transform block.
[0161] Through the proposed implementation, by using different numbers of prediction symbols for transform blocks with different residual characteristics, the complexity caused by symbol prediction can be reduced while maintaining encoding performance. In addition, for example, in the case of a block to which LFNST is applied, the proposed implementation may not be used.
[0162] In addition, in another embodiment, this document proposes an embodiment of adaptively adjusting the number of prediction symbols applied in residual coding based on the position index of the last significant coefficient in the current transform block. For example, in one embodiment, a method of using different numbers of prediction symbols based on whether the position index value of the last significant coefficient in the current transform block is within a specified threshold is proposed.
[0163] Figure 8 An 8×8 transform block to which a diagonal scan order is applied is exemplarily illustrated. For example, when a diagonal scan order is used, the transform block may be divided into coefficient groups (CGs) that are scanned sequentially. For example, referring to Figure 8 , in the case of an 8×8 transform block, the position index of the last significant coefficient can be indexed in the order of the first CG, the second CG, the third CG and the fourth CG of the 4×4 CG, such as Figure 8 That is, the pixel positions in the first CG may have position indices from 0 to 15 in diagonal order, the pixel positions in the second CG may have position indices from 16 to 31 in diagonal order, the pixel positions in the third CG may have position indices from 32 to 47 in diagonal order, and the pixel positions in the fourth CG may have position indices from 48 to 63 in diagonal order.
[0164] For example, if the position index of the last significant coefficient in the current transform block is less than a certain threshold based on the (0,0) coordinate, the number of prediction symbols for the current transform block can be derived as n 1 On the contrary, if the position index of the last significant coefficient is equal to or greater than a certain threshold, the number of prediction symbols for the current transform block can be derived as n 2 .
[0165] Fig. 9An embodiment in which the number of prediction symbols is derived based on the position index of the last significant coefficient in the 8×8 transform block when the specific threshold is 16 is illustrated.
[0166] For example, Fig. 9 As shown in (a), if the position index of the last significant coefficient of the current block is less than 16 (meaning that the position index of the last significant coefficient is between 0 and 15), the number of prediction symbols for the current block can be derived as n 1 On the contrary, Fig. 9 As shown in (b), if the position index of the last significant coefficient is 16 or greater (meaning the position index is between 16 and 63), the number of prediction symbols for the current block can be derived as n 2 .
[0167] Fig.10 The diagram illustrates an embodiment of deriving the number of prediction symbols based on the position index of the last significant coefficient in a 16×4 transform block. Fig.10 (a) may represent a 16×4 transform block with a specific threshold of 16. Fig.10 (b) may represent a 16×4 transform block with a specific threshold of 64. Fig.10 As shown in (a), when the specific threshold is 16, if the position index of the last significant coefficient of the current block is less than 16 (meaning that the last significant coefficient is located in the first CG with a position index between 0 and 15), the number of prediction symbols for the current block can be derived as n 1 In all other cases, the number of prediction symbols for the current block can be derived as n 2 In addition, if Fig.10 As shown in (b), when the specific threshold is 64, the number of prediction symbols for the current block can be derived as n 1 , regardless of where the last significant coefficient is located within the current block.
[0168] Fig.11 An embodiment of deriving the number of prediction symbols based on the position index of the last significant coefficient in the 8×16 transform block when the specific threshold is 64 is illustrated.
[0169] Fig.11 (a) may represent an 8×16 transform block with a diagonal scan order. Fig.11As shown in (a), when a diagonal scan order is used, an 8×16 transform block may be divided into 4×4 coefficient groups (CGs) and indexed sequentially from the first CG to the eighth CG. That is, the pixel positions in the first CG may have position indexes from 0 to 15 in diagonal order, the pixel positions in the second CG may have position indexes from 16 to 31 in diagonal order, the pixel positions in the third CG may have position indexes from 32 to 47 in diagonal order, the pixel positions in the fourth CG may have position indexes from 48 to 63 in diagonal order, the pixel positions in the fifth CG may have position indexes from 64 to 79 in diagonal order, the pixel positions in the sixth CG may have position indexes from 80 to 95 in diagonal order, the pixel positions in the seventh CG may have position indexes from 96 to 111 in diagonal order, and the pixel positions in the eighth CG may have position indexes from 112 to 127 in diagonal order.
[0170] Fig.11 (b) of can represent an 8×16 transform block with a specific threshold of 64. Fig.11 As shown in (b), when the specific threshold is 64, if the position index of the last significant coefficient of the current block is less than 64 (which means that it is located at Fig.11 In one of the CGs shown in (b), where the position index is between 0 and 63), the number of prediction symbols for the current block can be derived as n 1 In all other cases, the number of prediction symbols for the current block can be derived as n 2 .
[0171] Subsequently, the actual number of prediction symbols can be derived by considering the number of non-zero coefficients in the actual symbol prediction region of the current transform block. For example, if the number of prediction symbols for the current transform block is derived as n 1 , it can be derived as described in Equation 6. Referring to Equation 6, if the number of prediction symbols for the current transform block is derived as n 1 , then n 1 The smaller value between the number of non-zero coefficients in the actual symbol prediction region will be determined as the actual number of prediction symbols for the current transform block.
[0172] In addition, for example, if the number of prediction symbols for the current transform block is derived as n 2 , it can be derived as described in Equation 7. Referring to Equation 7, if the number of prediction symbols for the current transform block is derived as n 2 , then n 2 The smaller value between the number of non-zero coefficients in the actual symbol prediction region will be determined as the actual number of prediction symbols for the current transform block.
[0173] Through the proposed implementation, by using different numbers of prediction symbols for transform blocks with different residual characteristics, the complexity caused by symbol prediction can be reduced while maintaining encoding performance. In addition, for example, in the case of a block to which LFNST is applied, the proposed implementation may not be used.
[0174] In addition, in another embodiment, this document proposes a method for signaling the number of symbols used in symbol prediction in a residual coding symbol prediction technique. Specifically, for example, this document proposes an embodiment of separately signaling information indicating the number of symbols used for symbol prediction in intra-frame slices and information indicating the number of symbols used for symbol prediction in inter-frame slices. According to this embodiment, the information signaled in a sequence parameter set (SPS) can be shown in the following table.
[0175] [Table 4]
[0176]
[0177] Table 4 illustrates an implementation of signaling sps_num_pred_signs_intra and sps_num_pred_signs_inter in an SPS. sps_num_pred_signs_intra may be a syntax element indicating the maximum number of prediction symbols that can be used for symbol prediction within a given symbol prediction unit region (e.g., a transform block or a prediction region) of an intra slice when symbol prediction is applied. In other words, sps_num_pred_signs_intra indicates the maximum number of prediction symbols that can be used for symbol prediction within a symbol prediction unit region (e.g., a transform block or a prediction region) of an intra slice. Additionally, sps_num_pred_signs_inter may be a syntax element indicating the maximum number of prediction symbols that can be used for symbol prediction within a given symbol prediction unit region (e.g., a transform block or a prediction region) of an inter slice when symbol prediction is applied. In other words, sps_num_pred_signs_inter indicates the maximum number of prediction symbols that can be used for symbol prediction within a symbol prediction unit region (e.g., a transform block or a prediction region) of an inter slice. For example, if there are more non-zero residual coefficients than the maximum number of prediction symbols in a unit region, symbol prediction may be applied only for as many residual coefficients as the maximum number of prediction symbols signaled in a given order (eg, a predetermined sorting order or scanning order).
[0178] In addition, for example, a method for signaling a syntax element indicating whether symbol prediction is enabled through an SPS may be proposed. The syntax element may be referred to as a symbol prediction enable flag. For example, if a syntax element indicating whether symbol prediction is enabled is present in an SPS, the proposed syntax elements (sps_num_pred_signs_intra, sps_num_pred_signs_inter) may depend on the syntax element indicating whether symbol prediction is enabled. For example, according to this embodiment, the information signaled in a sequence parameter set (SPS) may be as shown in the following table.
[0179] [Table 5]
[0180]
[0181] Referring to Table 5, a syntax element sps_sign_prediction_enabled_flag indicating whether symbol prediction is enabled may be signaled by the SPS. If a value of sps_sign_prediction_enabled_flag is 1, it may indicate that symbol prediction is enabled, and if a value of sps_sign_prediction_enabled_flag is 0, it may indicate that symbol prediction is not enabled. In addition, for example, only when sps_sign_prediction_enabled_flag is 1, that is, when symbol prediction is activated, information indicating the number of symbols for intra slice prediction (e.g., sps_num_pred_signs_intra) and information indicating the number of symbols for inter slice prediction (e.g., sps_num_pred_signs_inter) may be signaled.
[0182] In addition, for example, the proposed syntax element may also be modified in the form of sps_num_pred_signs_intra_minusX and sps_num_pred_signs_inter_minusX. In this case, the encoding device may signal a value obtained by subtracting X from the actual maximum number of prediction symbols as a syntax element, and the decoding device may parse these syntax elements and use the value obtained by adding X to the syntax element as the actual maximum number of prediction symbols. For example, sps_num_pred_signs_intra_minusX may represent a value obtained by subtracting X from the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of an intra slice, and sps_num_pred_signs_inter_minusX may represent a value obtained by subtracting X from the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of an inter slice. The proposed syntax element representing the number of prediction symbols may be encoded using an n (e.g., 4)-bit binary representation of an unsigned integer.
[0183] If the proposed implementation is applied, instead of using a fixed number of prediction symbols for all slices, different numbers of prediction symbols may be used according to the characteristics of intra slices and inter slices. This may result in improved coding performance and / or reduced complexity of the symbol prediction technique. In general, since intra slices contain a larger amount of residuals than inter slices, it may be more efficient to differentiate the number of prediction symbols between intra slices and inter slices.
[0184] In addition, in another embodiment, this document proposes an embodiment of separately signaling information indicating the number of symbols for luminance block prediction and information indicating the number of symbols for chrominance block prediction. According to this embodiment, the information signaled in the sequence parameter set (SPS) can be shown in the following table.
[0185] [Table 6]
[0186]
[0187] Table 6 illustrates an implementation of signaling sps_num_pred_signs_luma and sps_num_pred_signs_chroma in an SPS. sps_num_pred_signs_luma may be a syntax element indicating the maximum number of prediction symbols that can be symbol predicted within a given symbol prediction unit region (e.g., a transform block or a prediction region) of a luma block when symbol prediction is applied. In other words, sps_num_pred_signs_luma indicates the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a luma block. Additionally, sps_num_pred_signs_chroma may be a syntax element indicating the maximum number of prediction symbols that can be symbol predicted within a given symbol prediction unit region (e.g., a transform block or a prediction region) of a chroma block when symbol prediction is applied. In other words, sps_num_pred_signs_chroma indicates the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a chroma block. For example, if there are more non-zero residual coefficients than the maximum number of prediction symbols in a unit region, symbol prediction may be applied only for as many residual coefficients as the maximum number of prediction symbols signaled in a given order (eg, a predetermined sorting order or scanning order).
[0188] In addition, for example, a method for signaling a syntax element indicating whether symbol prediction is enabled through SPS may be proposed. The syntax element may be referred to as a symbol prediction enable flag. For example, if a syntax element indicating whether symbol prediction is enabled is present in SPS, the proposed syntax element (sps_num_pred_signs_luma, sps_num_pred_signs_chroma) may depend on the syntax element indicating whether symbol prediction is enabled. For example, according to this embodiment, the information signaled in the sequence parameter set (SPS) may be as shown in the following table.
[0189] [Table 7]
[0190]
[0191] Referring to Table 7, the syntax element sps_sign_prediction_enabled_flag indicating whether symbol prediction is enabled in the SPS may be signaled. If the value of sps_sign_prediction_enabled_flag is 1, it may indicate that symbol prediction is enabled, and if the value of sps_sign_prediction_enabled_flag is 0, it may indicate that symbol prediction is not enabled. In addition, for example, information indicating the number of symbols predicted for the luma block (e.g., sps_num_pred_signs_luma) and information indicating the number of symbols predicted for the chroma block (e.g., sps_num_pred_signs_chroma) may be signaled only when sps_sign_prediction_enabled_flag is 1, that is, when symbol prediction is activated.
[0192] In addition, for example, the proposed syntax elements may also be modified in the form of sps_num_pred_signs_luma_minusX and sps_num_pred_signs_chroma_minusX. In this case, the encoding device may signal a value obtained by subtracting X from the actual maximum number of prediction symbols as a syntax element, and the decoding device may parse these syntax elements and use the value obtained by adding X to the syntax element as the actual maximum number of prediction symbols. For example, sps_num_pred_signs_luma_minusX may represent a value obtained by subtracting X from the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a luma block, and sps_num_pred_signs_chroma_minusX may represent a value obtained by subtracting X from the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a chroma block. The proposed syntax element representing the number of prediction symbols may be encoded using an n (e.g., 4)-bit binary representation of an unsigned integer.
[0193] If the proposed embodiment is applied, instead of using a fixed number of prediction symbols for all slices, different numbers of prediction symbols can be used according to the characteristics of the luma block and the chroma block. This can lead to improved coding performance and / or reduced complexity of the symbol prediction technique.
[0194] Additionally, in another embodiment, this document proposes a method of signaling the number of symbols used in symbol prediction in a symbol prediction technique for residual coding in a slice header or a picture header.
[0195] For example, an implementation of signaling a syntax element indicating the number of prediction symbols in a slice header may be as shown in the following table.
[0196] [Table 8]
[0197]
[0198] sh_num_pred_signs may be a syntax element indicating the maximum number of prediction symbols that may be used for symbol prediction within a given symbol prediction unit region (e.g., a transform block or a prediction region) of each slice when symbol prediction is applied. In other words, sh_num_pred_signs may indicate the maximum number of prediction symbols that may be used for symbol prediction within a symbol prediction unit region (e.g., a transform block or a prediction region) of a slice. For example, if there are more non-zero residual coefficients than the maximum number of prediction symbols in the unit region, symbol prediction may be applied only for as many residual coefficients as the maximum number of prediction symbols that have been signaled in a given order (e.g., a predetermined sorting order or a scanning order).
[0199] In addition, for example, a method of signaling a syntax element indicating whether symbol prediction is enabled through SPS may be proposed. For example, if the syntax element indicating whether symbol prediction is enabled is present in SPS, the proposed slice header level syntax element (sh_num_pred_signs) may depend on the syntax element indicating whether symbol prediction is enabled. For example, according to this embodiment, the information signaled in the slice header (SH) may be as shown in the following table.
[0200] [Table 9]
[0201]
[0202] For example, information (e.g., sh_num_pred_sign) indicating the maximum number of prediction symbols that can be used for symbol prediction within a given symbol prediction unit region (e.g., transform block or prediction region) of a slice may be signaled only when sps_sign_prediction_enabled_flag is 1, that is, when symbol prediction is activated.
[0203] In addition, in another embodiment, this document proposes an embodiment of separately signaling information indicating the number of symbols for luminance block prediction and information indicating the number of symbols for chrominance block prediction in a slice header. According to this embodiment, the information signaled in the slice header may be as shown in the following table.
[0204] [Table 10]
[0205]
[0206] Table 10 illustrates an implementation of signaling sh_num_pred_signs_luma and sh_num_pred_signs_chroma in a slice header. sh_num_pred_signs_luma may be a syntax element indicating the maximum number of prediction symbols that can be symbol predicted within a given symbol prediction unit region (e.g., a transform block or a prediction region) of a luma block in a slice when symbol prediction is applied. In other words, sh_num_pred_signs_luma indicates the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a luma block in a slice. Similarly, sh_num_pred_signs_chroma may be a syntax element indicating the maximum number of prediction symbols that can be symbol predicted within a given symbol prediction unit region (e.g., a transform block or a prediction region) of a chroma block in a slice when symbol prediction is applied. In other words, sh_num_pred_signs_chroma indicates the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a chroma block in a slice.
[0207] In addition, for example, if the syntax element indicating whether symbol prediction is enabled is present in the SPS, the proposed slice header level syntax elements (sh_num_pred_signs_luma, sh_num_pred_signs_chroma) may depend on the syntax element indicating whether symbol prediction is enabled. For example, according to this embodiment, the information signaled in the slice header (SH) may be as shown in the following table.
[0208] [Table 11]
[0209]
[0210] For example, information indicating the maximum number of prediction symbols that can be used for symbol prediction within a given symbol prediction unit area (e.g., a transform block or a prediction area) of a luminance block in a slice (e.g., sh_num_pred_signs_luma) and information indicating the maximum number of prediction symbols that can be used for symbol prediction within a given symbol prediction unit area (e.g., a transform block or a prediction area) of a chroma block in a slice (e.g., sh_num_pred_signs_chroma) can be signaled only when sps_sign_prediction_enabled_flag is 1, that is, when symbol prediction is activated.
[0211] In addition, for example, the proposed syntax elements may also be modified in the form of sh_num_pred_signs_luma_minusX and sh_num_pred_signs_chroma_minusX. In this case, the encoding device may signal a value obtained by subtracting X from the actual maximum number of prediction symbols as a syntax element, and the decoding device may parse these syntax elements and use the value obtained by adding X to the syntax element as the actual maximum number of prediction symbols. For example, sh_num_pred_signs_luma_minusX may represent a value obtained by subtracting X from the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a luminance block in a slice, and sh_num_pred_signs_chroma_minusX may represent a value obtained by subtracting X from the maximum number of prediction symbols that can be symbol predicted within a symbol prediction unit region (e.g., a transform block or a prediction region) of a chroma block in a slice. The proposed syntax element representing the number of prediction symbols may be encoded using an n (e.g., 4)-bit binary representation of an unsigned integer.
[0212] If the proposed implementation is applied, instead of using a fixed number of prediction symbols throughout the sequence, different numbers of prediction symbols may be used on a slice-by-slice or picture-by-picture basis. This may result in improved coding performance and / or reduced complexity of the symbol prediction technique.
[0213] In addition, as another embodiment, this document proposes a method for signaling a symbol prediction area in a slice header or a picture header when the number of symbols to be symbol predicted is signaled in a slice header or a picture header in a symbol prediction technique of residual coding. For example, according to this embodiment, information indicating the symbol prediction area signaled in a slice header (SH) can be as shown in the following table.
[0214] [Table 12]
[0215]
[0216] For example, Table 12 may illustrate a method of signaling information representing a symbol prediction area at a slice header level when the maximum number of prediction symbols for symbol prediction is signaled at a slice header level, as proposed in the embodiment. sh_sign_pred_area may be a syntax element representing a unit area within a transform block to which symbol prediction is applied. In other words, sh_sign_pred_area represents a unit area within a transform block to which symbol prediction is applied. For example, if the value of sh_sign_pred_area is 4, it may represent that a 4×4 area at the upper left corner of the transform block is a symbol prediction area, and if the value of sh_sign_pred_area is 8, it may represent that an 8×8 area at the upper left corner of the transform block is a symbol prediction area.
[0217] In addition, for example, if a syntax element indicating whether symbol prediction is enabled exists in the SPS, the proposed syntax element (sh_sign_pred_area) indicating the symbol prediction area may depend on the syntax element indicating whether symbol prediction is enabled. For example, according to this embodiment, the information signaled in the slice header (SH) may be as shown in the following table.
[0218] [Table 13]
[0219]
[0220] For example, information indicating the symbol prediction area (eg, sh_sign_prediction_enabled_flag) may be signaled only when sps_sign_prediction_enabled_flag is 1, that is, when symbol prediction is activated.
[0221] In addition, for example, an embodiment in which the information indicating the number of prediction symbols and the information indicating the symbol prediction area are not separately signaled in the slice header may be proposed. For example, the proposed syntax element (sh_sign_pred_area) indicating the symbol prediction area may depend on the syntax element indicating the number of prediction symbols. According to this embodiment, the information signaled in the slice header (SH) may be as shown in the following table.
[0222] [Table 14]
[0223]
[0224] In addition, for example, if information indicating the number of prediction symbols for a luminance block and information indicating the number of prediction symbols for a chrominance block are signaled separately at the slice level, such as in the other embodiments described above, the proposed syntax element indicating the symbol prediction area may be signaled accordingly.
[0225] In addition, for example, the proposed syntax element may also be modified in the form of sh_sign_pred_areaX. In this case, the encoding device may signal a value obtained by subtracting X from the actual symbol prediction area value as a syntax element, and the decoding device may parse the syntax element and then use the value obtained by adding X to the value of the syntax element as the actual symbol prediction area value. For example, sh_sign_pred_areaX may represent a value obtained by subtracting X from a symbol prediction area value within a symbol prediction unit area (e.g., a transform block or a prediction area) of a corresponding slice. The proposed syntax element may be encoded as an n-bit (e.g., 4) unsigned integer in a binarized format.
[0226] When applying the proposed implementation, instead of using a fixed number of prediction symbols in the entire sequence, different symbol prediction regions can be used based on the characteristics of the video on a slice-by-slice or picture-by-picture basis. This allows improving the coding performance and / or reducing the complexity of the symbol prediction technique.
[0227] Fig.12 A video encoding method performed by an encoding device according to the present document is schematically illustrated. Fig.12 The method disclosed in can be Figure 2 Specifically, for example, Fig.12 Steps S1200 to S1220 in the above may be performed by the residual processing unit of the encoding device, and step S1230 may be performed by the entropy encoding unit of the encoding device. In addition, although not shown, the process of deriving the prediction sample for the current block may be performed by the prediction unit of the encoding device, and the process of generating the reconstructed sample and the reconstructed picture based on the prediction sample and the residual sample may be performed by the adder unit of the encoding device.
[0228] The encoding device derives the number of prediction symbols for the current block (S1200). The encoding device can derive the number of symbols to be symbol-predicted for the current block. For example, the encoding device can derive the number of prediction symbols for the current block based on information about the last significant coefficient and / or the number of prediction symbols of the current block.
[0229] For example, the number of prediction symbols may be derived based on whether the last significant coefficient of the current block is located in a specific region of the current block. For example, if the last significant coefficient of the current block is located in a specific region, the number of prediction symbols may be derived as a first value (n 1 ), and if the last significant coefficient is not located in a specific area, the prediction symbol number can be derived as a second value (n 2). For example, the specific area may be a predetermined area having a specific size. The predetermined size may be a Th×Th size. Alternatively, for example, the specific area may be a valid symbol prediction area. The valid symbol prediction area may represent the actual symbol prediction area described above. The valid symbol prediction area may be derived based on the size of the current block and the symbol prediction area. For example, the symbol prediction area may be an area having a predetermined size. Alternatively, for example, the symbol prediction area may be an area whose size is derived based on intraPeriod, QP, resolution, and / or whether the IBC mode is applied to the current block. The width of the valid symbol prediction area may be derived as the smaller value between the width of the symbol prediction area and the width of the current block, and the height of the valid symbol prediction area may be derived as the smaller value between the height of the symbol prediction area and the height of the current block. The valid symbol prediction area may be derived based on the aforementioned formula 5.
[0230] Alternatively, for example, the number of prediction symbols may be derived based on whether the position index of the last significant coefficient of the current block is lower than a specific threshold. For example, if the last significant coefficient of the current block is lower than a specific threshold, the number of prediction symbols may be derived as a first value (n 1 ), and if the last significant coefficient of the current block is equal to or greater than a specific threshold, the number of prediction symbols can be derived as a second value (n 2 ). The specific threshold value may be a predetermined value. For example, the specific threshold value may be 16 or 64.
[0231] Alternatively, for example, the encoding device may encode information about the symbol prediction number, and may derive the predicted symbol number based on the information about the symbol prediction number.
[0232] For example, the information about the symbol prediction number may include information indicating the symbol prediction number for a transform block in an intra slice and information indicating the symbol prediction number for a transform block in an inter slice, and the information about the symbol prediction number may be signaled through a sequence parameter set (SPS). The information indicating the symbol prediction number for a transform block in an intra slice may be represented as the aforementioned sps_num_pred_signs_intra, and the information indicating the symbol prediction number for a transform block in an inter slice may be represented as the aforementioned sps_num_pred_signs_inter. For example, if the current block is a transform block in an intra slice, the number of prediction symbols for the current block may be derived based on the information indicating the symbol prediction number for the transform block in the intra slice. Alternatively, for example, if the current block is a transform block in an inter slice, the number of prediction symbols for the current block may be derived based on the information indicating the symbol prediction number for the transform block in the inter slice.
[0233] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag, which indicates whether symbol prediction is available. For example, the encoding device may encode a symbol prediction enable flag indicating whether symbol prediction is available, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may be represented as the aforementioned sps_sign_prediction_enabled_flag.
[0234] Alternatively, for example, the information about the symbol prediction number may include information indicating the symbol prediction number for the luma block and information indicating the symbol prediction number for the chroma block, and the information about the symbol prediction number may be signaled through a sequence parameter set (SPS). The information indicating the symbol prediction number for the luma block may represent the aforementioned sps_num_pred_signs_luma, and the information indicating the symbol prediction number for the chroma block may represent the aforementioned sps_num_pred_signs_chroma. For example, if the current block is a luma block, the number of predicted symbols for the current block may be derived based on the information indicating the number of symbol predictions for the luma block. Alternatively, for example, if the current block is a chroma block, the number of predicted symbols for the current block may be derived based on the information indicating the number of symbol predictions for the chroma block.
[0235] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag, which indicates whether symbol prediction is available. For example, the encoding device may encode a symbol prediction enable flag indicating whether symbol prediction is available, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may be represented as the aforementioned sps_sign_prediction_enabled_flag.
[0236] Alternatively, for example, the information about the symbol prediction number may be signaled through a slice header, and the information about the symbol prediction number may represent the symbol prediction number for a transform block within the slice. The information about the symbol prediction number may represent the aforementioned sh_num_pred_signs.
[0237] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag, which indicates whether symbol prediction is available. For example, the encoding device may encode a symbol prediction enable flag indicating whether symbol prediction is available, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may be represented as the aforementioned sps_sign_prediction_enabled_flag.
[0238] Alternatively, for example, information about the number of symbol predictions may be signaled through a slice header, and the information about the number of symbol predictions may include information indicating the number of symbol predictions for a luma block within the slice and information indicating the number of symbol predictions for a chroma block within the slice. The information indicating the number of symbol predictions for the luma block may represent the aforementioned sh_num_pred_signs_luma, and the information indicating the number of symbol predictions for the chroma block may represent the aforementioned sh_num_pred_signs_chroma. For example, if the current block is a luma block within the current slice, the number of predicted symbols for the current block within the current slice may be derived based on the information indicating the number of symbol predictions for the luma block. Alternatively, for example, if the current block is a chroma block within the current slice, the number of predicted symbols for the current block within the current slice may be derived based on the information indicating the number of symbol predictions for the chroma block.
[0239] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag indicating whether symbol prediction is available. For example, the encoding device may encode a symbol prediction enable flag indicating whether symbol prediction is available, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may represent the aforementioned sps_sign_prediction_enabled_flag.
[0240] Alternatively, for example, the encoding device may encode information about the symbol prediction quantity and information about the symbol prediction region. For example, the symbol prediction quantity for the current block may be derived based on the information about the symbol prediction quantity and information about the symbol prediction region.
[0241] For example, the encoding device may encode information about the symbol prediction region. The information about the symbol prediction region and the information about the symbol prediction quantity may be signaled through a slice header, wherein the information about the symbol prediction region may indicate the symbol prediction region for a transform block within the slice, and the information about the symbol prediction quantity may indicate the symbol prediction quantity for the transform block.
[0242] Alternatively, for example, the encoding device may encode the information about the symbol prediction region, and may signal the information about the symbol prediction number based on the value of the information about the symbol prediction region.
[0243] The encoding apparatus derives signs of the residual coefficients of the prediction symbol number by performing sign prediction on the current block (S1210). The encoding apparatus may predict the signs of the residual coefficients corresponding to the prediction symbol number.
[0244] For example, the encoding device may predict the sign of the residual coefficient by performing the sign prediction process described above. For example, the encoding device may calculate the cost of the sign combination hypothesis of the residual coefficient and predict the sign of the residual coefficient with the lowest cost. For example, the encoding device may perform a partial inverse transform and boundary reconstruction on the sign combination hypothesis to derive the hypothesized reconstructed sample of the current block. The cost of the sign combination hypothesis may be calculated based on the left neighboring samples, the upper neighboring samples and the hypothesized reconstructed samples of the current block. The cost may be derived based on the aforementioned formula 1. In addition, the sign combination hypothesis may also be referred to as a sign combination candidate.
[0245] The encoding apparatus derives residual samples of the current block based on the symbol (S1220). The encoding apparatus may derive residual samples of the current block based on the symbol.
[0246] The encoding apparatus encodes image information including residual information of the current block and information on the number of prediction symbols (S1230). The encoding apparatus may encode residual information for residual samples.
[0247] For example, the encoding device may encode the residual information of the current block including the syntax element for the prediction symbol. For example, the syntax element for the prediction symbol may indicate whether the prediction symbol is accurate. That is, for example, the target syntax element among the syntax elements may indicate whether the prediction symbol for the transform coefficient of the target syntax element is accurate. The syntax element for the prediction symbol of the transform coefficient of the current block may represent the above-mentioned symbol residual. The image information may include the residual information.
[0248] In addition, for example, the encoding device may generate and encode prediction mode information for the current block. For example, the prediction mode information may indicate an intra-frame prediction mode or an inter-frame prediction mode applied to the current block. In addition, the encoding device may generate and encode prediction-related information about motion information of the current block. The motion information may include a motion vector and a reference picture index.
[0249] In addition, for example, the encoding device may encode residual information about the residual sample. The residual information may include a syntax element for a prediction symbol of a transform coefficient of the current block.
[0250] In addition, for example, the encoding device may encode a symbol prediction enabling flag, information about the number of prediction symbols, and / or information about a symbol prediction area.
[0251] The image information may include prediction mode information of the current block, prediction related information, a symbol prediction enabling flag, information about the number of prediction symbols, information about the symbol prediction area, and / or residual information. In addition, for example, the encoding device may encode the image information and output the image information in the form of a bitstream.
[0252] In addition, the bit stream containing the image information can be sent to the decoding device via a network or a (digital) storage medium. Here, the network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD.
[0253] Fig.13 The encoding device that performs the video encoding method according to this document is schematically illustrated. Fig.12 The method disclosed in can be Fig.13 The encoding device shown is executed. Specifically, for example, Fig.13 The residual processing unit of the encoding device in may perform steps S1200 to S1220, and the entropy encoding unit of the encoding device may perform step S1230. In addition, although not shown, the process of generating reconstructed samples and reconstructed pictures based on residual samples and prediction samples may be performed by an adder unit of the encoding device.
[0254] Fig.14 A video decoding method performed by a decoding device according to this document is schematically illustrated. Fig.14 The method disclosed in can be Figure 3 The decoding device shown is executed. Specifically, for example, Fig.14Steps S1400 to S1430 in the above may be performed by a residual processing unit of a decoding device. In addition, although not shown, the process of obtaining image information including prediction related information and residual information of the current block through a bitstream may be performed by an entropy decoding unit of a decoding device. The process of deriving a prediction sample for the current block based on the prediction related information may be performed by a prediction unit of a decoding device.
[0255] The decoding device derives the number of prediction symbols for the current block (S1400). The decoding device may derive the number of symbols to be symbol-predicted for the current block. For example, the decoding device may derive the number of prediction symbols for the current block based on information about the last significant coefficient and / or information about the number of prediction symbols for the current block.
[0256] For example, the number of prediction symbols may be derived based on whether the last significant coefficient of the current block is located in a specific region of the current block. If the last significant coefficient of the current block is located in a specific region, the number of prediction symbols may be derived as a first value (n 1 ), and if the last significant coefficient is not located in a specific region, the number of prediction symbols may be derived as a second value (n 2 ). The specific area may be a predefined area having a specific size (such as a Th×Th size). Alternatively, for example, the specific area may be a valid symbol prediction area, which represents the actual symbol prediction area described above. The valid symbol prediction area may be derived based on the size of the current block and the symbol prediction area. Alternatively, for example, the symbol prediction area may be an area whose size is derived based on factors such as intraPeriod, QP, resolution, and / or whether the IBC mode is applied to the current block. The width of the valid symbol prediction area may be derived as the smaller value between the width of the symbol prediction area and the width of the current block, and the height of the valid symbol prediction area may be derived as the smaller value between the height of the symbol prediction area and the height of the current block. The valid symbol prediction area may be derived based on the aforementioned formula 5.
[0257] Alternatively, for example, the number of prediction symbols may be derived based on whether the position index of the last significant coefficient of the current block is less than a specific threshold. For example, if the last significant coefficient of the current block is less than a specific threshold, the number of prediction symbols may be derived as a first value (n 1 ), and if the last significant coefficient of the current block is equal to or greater than a specific threshold, the number of prediction symbols can be derived as a second value (n 2 ). The specific threshold value may be a predetermined value. For example, the specific threshold value may be 16 or 64.
[0258] Alternatively, for example, the decoding device may obtain information about the predicted number of symbols through a bit stream, and may derive the predicted symbol number based on the information about the predicted number of symbols.
[0259] For example, the information about the number of symbol predictions may include information indicating the number of symbol predictions for transform blocks in intra slices and information indicating the number of symbol predictions for transform blocks in inter slices. The information about the number of symbol predictions may be signaled by a sequence parameter set (SPS). The information indicating the number of symbol predictions for transform blocks in intra slices may represent the aforementioned sps_num_pred_signs_intra, and the information indicating the number of symbol predictions for transform blocks in inter slices may represent the aforementioned sps_num_pred_signs_inter. For example, if the current block is a transform block in an intra slice, the number of prediction symbols for the current block may be derived based on the information indicating the number of symbol predictions for transform blocks in the intra slice. Alternatively, for example, if the current block is a transform block in an inter slice, the number of prediction symbols for the current block may be derived based on the information indicating the number of symbol predictions for transform blocks in the inter slice.
[0260] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag indicating whether symbol prediction is available. For example, the decoding device may obtain a symbol prediction enable flag indicating whether symbol prediction is available through a bitstream, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may represent the aforementioned sps_sign_prediction_enabled_flag.
[0261] Alternatively, for example, the information about the symbol prediction number may include information indicating the symbol prediction number for the luma block and information indicating the symbol prediction number for the chroma block, and the information about the symbol prediction number may be signaled through a sequence parameter set (SPS). The information indicating the symbol prediction number for the luma block may represent the aforementioned sps_num_pred_signs_luma, and the information indicating the symbol prediction number for the chroma block may represent the aforementioned sps_num_pred_signs_chroma. For example, if the current block is a luma block, the number of predicted symbols for the current block may be derived based on the information indicating the number of symbol predictions for the luma block. Alternatively, for example, if the current block is a chroma block, the number of predicted symbols for the current block may be derived based on the information indicating the number of symbol predictions for the chroma block.
[0262] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag indicating whether symbol prediction is available. For example, the decoding device may obtain a symbol prediction enable flag indicating whether symbol prediction is available through a bitstream, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may represent the aforementioned sps_sign_prediction_enabled_flag.
[0263] Alternatively, for example, the information about the symbol prediction number may be signaled through a slice header, and the information about the symbol prediction number may represent the symbol prediction number for a transform block within the slice. The information about the symbol prediction number may represent the aforementioned sh_num_pred_signs.
[0264] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag indicating whether symbol prediction is available. For example, the decoding device may obtain a symbol prediction enable flag indicating whether symbol prediction is available through a bitstream, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may represent the aforementioned sps_sign_prediction_enabled_flag.
[0265] Alternatively, for example, information about the number of symbol predictions may be signaled through a slice header and may include information indicating the number of symbol predictions for a luma block within the slice and the number of symbol predictions for a chroma block within the slice. The information indicating the number of symbol predictions for the luma block may represent the aforementioned sh_num_pred_signs_luma, and the information indicating the number of symbol predictions for the chroma block may represent the aforementioned sh_num_pred_signs_chroma. For example, if the current block is a luma block within the current slice, the number of predicted symbols for the current block within the current slice may be derived based on the information indicating the number of symbol predictions for the luma block. Alternatively, for example, if the current block is a chroma block within the current slice, the number of predicted symbols for the current block within the current slice may be derived based on the information indicating the number of symbol predictions for the chroma block.
[0266] In addition, for example, information about the number of symbol predictions may be signaled according to a symbol prediction enable flag indicating whether symbol prediction is available. For example, the decoding device may obtain a symbol prediction enable flag indicating whether symbol prediction is available through a bitstream, and may signal information about the number of symbol predictions based on the symbol prediction enable flag indicating that symbol prediction is available. The symbol prediction enable flag may be signaled through a sequence parameter set (SPS). The symbol prediction enable flag may represent the aforementioned sps_sign_prediction_enabled_flag.
[0267] Alternatively, for example, the decoding device may obtain information about the symbol prediction quantity and information about the symbol prediction region. For example, the symbol prediction quantity for the current block may be derived based on the information about the symbol prediction quantity and information about the symbol prediction region.
[0268] For example, the decoding device may obtain information about the symbol prediction region through a bitstream. The information about the symbol prediction region and the information about the symbol prediction quantity may be signaled through a slice header, wherein the information about the symbol prediction region may indicate a symbol prediction region for a transform block within a slice, and the information about the symbol prediction quantity may indicate a symbol prediction quantity for a transform block.
[0269] Alternatively, for example, the decoding device may obtain the information about the symbol prediction region through the bit stream, and may signal the information about the symbol prediction number based on the value of the information about the symbol prediction region.
[0270] The decoding apparatus derives the signs of the residual coefficients of the prediction symbol number by performing sign prediction on the current block (S1410). The decoding apparatus may predict the signs of the residual coefficients corresponding to the prediction symbol number.
[0271] For example, the decoding device may perform the aforementioned symbol prediction process to predict the sign of the residual coefficient. The decoding device may calculate the cost of the symbol combination hypothesis of the residual coefficient and predict the symbol combination hypothesis with the lowest cost as the sign of the residual coefficient. For example, the decoding device may perform a partial inverse transform and boundary reconstruction on the symbol combination hypothesis to derive a hypothetical reconstructed sample of the current block for the symbol combination hypothesis. The cost of the symbol combination hypothesis may be calculated based on the left neighboring samples, the upper neighboring samples, and the hypothetical reconstructed samples of the current block. The cost may be derived based on the aforementioned formula 1. In addition, the symbol combination hypothesis may also be referred to as a symbol combination candidate.
[0272] The decoding apparatus derives residual samples of the current block based on the symbol (S1420). The decoding apparatus may derive residual samples of the current block based on the symbol.
[0273] For example, the decoding device may determine whether the prediction symbol is correct based on the syntax element for the prediction symbol, and may derive the sign of the residual coefficient based on the result of the determination. For example, the decoding device may obtain the residual information of the current block, and the residual information may include the syntax element for the prediction symbol. For example, the syntax element for the prediction symbol may indicate whether the prediction symbol is correct. That is, for example, among the syntax elements, the target syntax element may indicate whether the prediction symbol for the target syntax element is correct. For example, if the target syntax element indicates that the prediction symbol for the target syntax element is correct, the decoding device may determine that the prediction symbol is correct and derive the prediction symbol as the sign of the specific residual coefficient corresponding to the prediction symbol. Alternatively, for example, if the target syntax element indicates that the prediction symbol for the target syntax element is incorrect, the decoding device may determine that the prediction symbol is incorrect and derive the opposite sign of the prediction symbol as the sign of the specific residual coefficient corresponding to the prediction symbol. That is, for example, if the target syntax element indicates that the prediction sign for the target syntax element is incorrect, if the prediction sign is positive, then the sign of the specific residual coefficient corresponding to the prediction sign can be derived as negative, and if the prediction sign is negative, then the sign of the specific residual coefficient corresponding to the prediction sign can be derived as positive. The syntax element of the prediction sign for the specific residual coefficient of the current block can represent the above-mentioned sign residual.
[0274] Then, for example, the decoding device may derive an absolute value of the residual coefficient based on the residual information of the current block, and based on the derived sign and the absolute value of the residual coefficient, the decoding device may derive a residual sample for the residual coefficient.
[0275] The decoding apparatus generates a reconstructed picture based on the residual samples (S1430). For example, the decoding apparatus may generate a reconstructed picture based on the residual samples.
[0276] In addition, for example, the decoding device may perform an inter-frame prediction mode or an intra-frame prediction mode for the current block based on the prediction mode information obtained through the bitstream to derive a prediction sample, and a reconstructed picture may be generated by adding the prediction sample and the residual sample. For example, the decoding device may derive an intra-frame prediction mode for the current block based on the prediction mode information, may derive a prediction sample of the current block based on the intra-frame prediction mode, and may generate a reconstructed sample and a reconstructed picture by adding the prediction sample and the residual sample. Or, for example, the decoding device may determine to apply inter-frame prediction to the current block based on the prediction mode information, may derive motion information of the current block based on prediction-related information about the current block, may derive a reference block within a reference picture based on the motion information, may derive a prediction sample of the current block based on the reference block, and may generate a reconstructed sample and a reconstructed picture by adding the prediction sample and the residual sample. As described above, an in-loop filtering process such as a deblocking filter, SAO, and / or ALF process may be applied to the reconstructed sample to improve the subjective / objective image quality as required.
[0277] In addition, for example, the decoding device may obtain image information of the current block through a bitstream. The image information may include information about the number of prediction symbols, information about a symbol prediction area, a symbol prediction enable flag, prediction mode information of the current block, and / or residual information. The residual information may include syntax elements of prediction symbols for transform coefficients of the current block.
[0278] Fig.15 A decoding device for performing a video decoding method according to the present disclosure is schematically illustrated. Fig.14 The method disclosed in can be Fig.15 Specifically, for example, Fig.15 The residual processing unit of the decoding device in the embodiment may perform Fig.14 S1400 to S1420, and Fig.15 The adder in the decoding device can perform Fig.14 S1430.
[0279] According to the aforementioned document, the number of prediction symbols based on the last significant coefficient of the current block can be adaptively derived, thereby improving the accuracy of symbol prediction by reflecting the characteristics of each block in the image, and reducing the complexity of symbol prediction to improve coding efficiency.
[0280] In addition, according to this document, syntax elements for the number of prediction symbols can be signaled separately for intra-frame slices and inter-frame slices, thereby improving the accuracy of symbol prediction by reflecting the characteristics of the image and reducing the complexity of symbol prediction to improve encoding efficiency.
[0281] In addition, according to this document, syntax elements for the number of prediction symbols can be signaled separately for luma blocks and chroma blocks, thereby improving the accuracy of symbol prediction by reflecting the characteristics of an image and reducing the complexity of symbol prediction to improve encoding efficiency.
[0282] In addition, according to this document, the syntax elements for the number of prediction symbols can be signaled through a slice header or a picture header, thereby improving the accuracy of symbol prediction by reflecting the characteristics of each slice or picture in the image and reducing the complexity of symbol prediction to improve coding efficiency.
[0283] In the above embodiments, the method is described based on a flow chart with a series of steps or boxes. The present disclosure is not limited to the order of the above steps or boxes. Some steps or boxes can be performed in an order different from other steps or boxes mentioned above or performed simultaneously. In addition, it will be understood by those skilled in the art that the steps shown in the flow chart are not exclusive, and other steps may also be included, or one or more steps in the flow chart may be deleted without affecting the scope of the present disclosure.
[0284] The embodiments described in this specification may be implemented on a processor, a microprocessor, a controller or a chip. For example, the functional units shown in each figure may be implemented on a computer, a processor, a microprocessor, a controller or a chip. In this case, information (e.g., information about instructions) or algorithms for implementation may be stored in a digital storage medium.
[0285] In addition, the decoding device and encoding device of the present disclosure can be included in the following devices: multimedia broadcast sending / receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, portable cameras, VoD service providers, over-the-top (OTT) video devices, Internet streaming service providers, three-dimensional (3D) video devices, teleconferencing video devices, transportation user devices (e.g., vehicle user devices, aircraft user devices, and ship user devices) and medical video equipment, and the decoding device and encoding device of the present disclosure can be used to process video signals or data signals. For example, over-the-top (OTT) video devices can include game consoles, Blu-ray players, Internet access televisions, home theater systems, smart phones, tablet computers, digital video recorders (DVRs), etc.
[0286] In addition, the processing method of the present invention can be generated in the form of a program executed by a computer, and can be stored in a computer-readable recording medium. The multimedia data with a data structure according to the present invention can also be stored in a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices in which data readable by a computer is stored. The computer-readable recording medium may include, for example, BD, universal serial bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage device. In addition, the computer-readable recording medium includes a medium implemented in the form of a carrier wave (e.g., transmission via the Internet). In addition, the bit stream generated by the encoding method can be stored in a computer-readable recording medium or transmitted via a wired / wireless communication network.
[0287] In addition, the embodiments of the present disclosure may be implemented using a computer program product according to a program code, and the program code may be executed in a computer through the embodiments of the present disclosure. The program code may be stored on a computer readable carrier.
[0288] Fig.16 The content streaming system architecture to which the embodiments of this document are applied is exemplarily illustrated.
[0289] The content streaming system to which the embodiments of this document are applied may mainly include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.
[0290] The encoding server compresses the content input from a multimedia input device such as a smartphone, a camera, or a camcorder into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, or a camcorder directly generates a bitstream, the encoding server can be omitted.
[0291] A bitstream may be generated by an encoding method or a bitstream generating method to which an embodiment of the present disclosure is applied, and a streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.
[0292] The streaming server sends multimedia data to the user device through the network server based on the user request, and the network server is used as a medium to notify the user of the service. When the user requests the required service from the network server, the network server delivers the request to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the command / response between the devices within the content streaming system.
[0293] The streaming server may receive content from a media storage device and / or an encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a stable streaming service, the streaming server may store the bitstream for a predetermined period of time.
[0294] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smart watches, smart glasses, and head-mounted displays), digital TVs, desktop computers, and digital signage, etc. Each server within the content streaming system may operate as a distributed server, in which case data received from each server may be distributed.
[0295] The claims described in this disclosure can be combined in various ways. For example, the technical features of the method claims of this disclosure can be combined to implement as a device, and the technical features of the device claims of this disclosure can be combined to implement as a method. In addition, the technical features of the method claims of this disclosure and the technical features of the device claims can be combined to implement as a device, and the technical features of the method claims of this disclosure and the technical features of the device claims can be combined to implement as a method.
Claims
1. A method for image decoding, the method being performed by a decoding device and The following steps are involved: deriving the number of prediction symbols for the current block; Deriving signs of the residual coefficients of the predicted sign quantity by performing sign prediction on the current block; deriving residual samples of the current block based on the symbols; as well as A reconstructed picture is generated based on the residual samples.
2. The method according to claim 1, in, The number of prediction symbols is derived based on whether a last significant coefficient of the current block is located in a specific area of the current block.
3. The method according to claim 2, in, The specific area is a valid symbol prediction area of the current block.
4. The method according to claim 1, in, The number of prediction symbols is derived based on whether a position index of a last significant coefficient of the current block is less than a specific threshold.
5. The method according to claim 1, further comprising: The following steps are involved: Information about the number of symbol predictions is obtained through the bitstream, The predicted number of symbols is derived based on information about the predicted number of symbols.
6. The method according to claim 5, in, The information about the symbol prediction quantity includes information indicating the symbol prediction quantity for the transform block of the intra slice and information indicating the symbol prediction quantity for the transform block of the inter slice, Therein, the information about the number of symbol predictions is signaled as a sequence parameter set SPS.
7. The method according to claim 6, further comprising: The following steps are involved: Acquire, through the bit stream, a symbol prediction enabling flag indicating whether the symbol prediction is available, The information about the number of symbol predictions is signaled based on the symbol prediction enable flag indicating that the symbol prediction is available.
8. The method according to claim 5, in, The information about the symbol prediction quantity includes information indicating the symbol prediction quantity for the luminance block and information indicating the symbol prediction quantity for the chrominance block, Therein, the information about the number of symbol predictions is signaled as a sequence parameter set SPS.
9. The method according to claim 5, in, Information about the number of symbol predictions is signaled via the slice header, The information about the symbol prediction quantity represents the symbol prediction quantity for the transform block in the slice.
10. The method according to claim 5, in, Information about the number of symbol predictions is signaled via the slice header, The information about the symbol prediction quantity includes information indicating the symbol prediction quantity for the luminance block within the slice and information indicating the symbol prediction quantity for the chrominance block within the slice.
11. The method according to claim 5, further comprising: The following steps are involved: Obtaining information about a symbol prediction region through the bit stream, wherein the information about the symbol prediction area and the information about the symbol prediction number are signaled via a slice header, The information about the symbol prediction area represents the symbol prediction area for the transform block in the slice, and The information about the symbol prediction quantity represents the symbol prediction quantity for the transform block.
12. The method according to claim 9, in, The information about the symbol prediction quantity is signaled based on a value of the information about the symbol prediction region.
13. A method for image encoding, the method being performed by an encoding device and The following steps are involved: deriving the number of prediction symbols for the current block; Deriving signs of the residual coefficients of the predicted sign quantity by performing sign prediction on the current block; deriving residual samples of the current block based on the symbols; as well as Image information including residual information of the current block and information on the number of prediction symbols is encoded.
14. A digital storage medium storing a bit stream including information on the number of prediction symbols and residual information encoded by the method for image encoding according to claim 13.
15. A method for transmitting data for an image, the method The following steps are involved: obtaining a bitstream of image information including information about the number of prediction symbols and residual information of a current block; as well as Data of the bit stream including the image information including the information on the number of prediction symbols and the residual information is transmitted.
