Image coding method based on transform, and device therefor

IN605432BActive Publication Date: 2026-10-09LG ELECTRONICS INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
IN202217028926
Authority / Receiving Office
IN · IN
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-11-11
Filing Date
2022-05-19
Publication Date
2026-10-09
Estimated Expiration
2040-11-09

AI Technical Summary

Technical Problem

The increasing demand for high-resolution and high-quality images/videos, such as 4K and 8K UHD, leads to increased transmission and storage costs due to higher data volumes, and there is a need for efficient compression techniques to handle immersive media formats like VR and AR, which require advanced image/video compression methods.

Method used

The method involves using Multiple Transform Selection (MTS) to derive transform coefficients and residual samples, with an MTS index parsed based on significant coefficients in specific regions of the image block, optimizing coding efficiency by zeroing out non-significant regions and signaling the MTS index effectively.

Benefits of technology

This approach enhances overall image/video compression efficiency, reduces data transmission and storage costs, and improves coding performance for high-resolution and immersive media formats by focusing on significant coefficient regions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000093_0000
    Figure 00000093_0000
  • Figure 00000094_0000
    Figure 00000094_0000
  • Figure 00000095_0000
    Figure 00000095_0000
Patent Text Reader

Abstract

An image decoding method according to the present document comprises a step of applying an inverse primary transform to a transform coefficient so as to derive residual samples for a current block, wherein: the inverse primary transform is performed on the basis of an MTS index received from a bitstream; the MTS index is parsed on the basis of an effective coefficient not existing in a second area that excludes the left top first area of the current block; and the first area is a left top 16X16 area of the current block.
Need to check novelty before this filing date? Find Prior Art

Description

SpecificationTitle of Invention: Transformation-based video coding method and device therefortechnical field[One]This document relates to an image coding technique, and more particularly, to a transform-based image coding method in an image coding system and an apparatus thereof.background[2]Recently, demand for high-resolution and high-quality images / videos such as 4K or 8K or higher UHD (Ultra High Definition) images / videos is increasing in various fields. As the image / video data becomes high-resolution and high-quality, the amount of information or bits transmitted relatively increases compared to the existing image / video data. In the case of storing video / video data by using the system, transmission cost and storage cost are increased.[3]In addition, recently, interest and demand for immersive media such as VR (Virtual Reality), AR (Artificial Realtiy) content or holograms are increasing. broadcasting is increasing.[4]Accordingly, high-efficiency image / video compression technology is required to effectively compress, transmit, store, and reproduce information of high-resolution and high-quality images / videos having various characteristics as described above.DETAILED DESCRIPTION OF THE INVENTIONtechnical challenge[5]An object of the present document is to provide a method and an apparatus for increasing image coding efficiency.[6]Another technical problem of the present document is to provide a method and apparatus for increasing the efficiency of transform index coding.[7]Another technical task of this document is to provide an image coding method and apparatus using MTS.[8]Another technical task of this document is to provide a method and apparatus for video coding for MTS index signaling.means of solving the problem[9]According to an embodiment of the present document, there is provided an image decoding method performed by a decoding apparatus. The method includes deriving residual samples for the current block by applying an inverse first-order transform to a transform coefficient, wherein the inverse-first-order transform is performed based on an MTS index received from the bitstream; The MTS index may be parsed based on the fact that a significant coefficient does not exist in a second region other than the upper-left first region of the current block, and the first region may be an upper-left 16X16 region of the current block.

[10] The MTS index is parsed based on flag information indicating whether a significant coefficient exists in the second area, and the flag information is in the second area in units of scan subblocks in which the significant coefficient is scanned. It can be derived by judging whether or not

[11] In the flag information, when a flag value indicating whether the significant coefficient exists in the scan sub-block is 1 and the significant coefficient existing in the scan sub-block is located in the second area, the flag information is displayed in the second area. It may indicate that a significant coefficient is present.

[12] In the flag information, a flag value indicating whether the significant coefficient exists in the scan sub-block is 1, and when the scan sub-block is located in the second area, the significant coefficient exists in the second area can instruct

[13] The flag information may be initially set to 1, and when the significant coefficient is located in the second area, the flag information may be changed to 0.

[14] The scan sub-block is a 4X4 block, and the scan sub-block may be scanned in an inverse diagonal scan direction from the position of the last significant coefficient in the current block.

[15] According to an embodiment of the present document, an image encoding method performed by an encoding apparatus is provided. The method includes applying MTS to residual samples to derive transform coefficients for the current block; zeroing out a second area excluding the first area at the upper left of the current block; deriving residual information based on the transform coefficient of the first region, and image information such that an MTS index indicating a transform kernel of the MTS is parsed based on whether a significant coefficient exists in the second region and configuring, wherein the first area may be an upper left 16X16 area of ​​the current block.

[16] According to another embodiment of the present document, a digital storage medium in which image data including encoded image information and a bitstream generated according to an image encoding method performed by an encoding apparatus are stored may be provided.

[17] According to another embodiment of the present document, there may be provided a digital storage medium in which image data including encoded image information and a bitstream causing the decoding apparatus to perform the image decoding method are stored.Effects of the Invention

[18] According to this document, overall image / video compression efficiency can be improved.

[19] According to this document, it is possible to increase the efficiency of LFNST index coding.

[20] According to this document, it is possible to increase the efficiency of the secondary transformation for the dual tree chroma block through the coding of the LFNST index.

[21] Effects that can be obtained through specific examples of the present specification are not limited to the effects listed above. For example, various technical effects that a person having ordinary skill in the related art can understand or derive from this specification may exist. Accordingly, the specific effects of the present specification are not limited to those explicitly described herein, and may include various effects that can be understood or derived from the technical characteristics of the present specification.Brief description of the drawing

[22] 1 schematically shows an example of a video / image coding system to which this document can be applied.

[23] 2 is a diagram schematically illustrating a configuration of a video / image encoding apparatus to which this document can be applied.

[24] 3 is a diagram schematically illustrating a configuration of a video / image decoding apparatus to which this document can be applied.

[25] 4 schematically illustrates a multiple transform scheme according to an embodiment of the present document.

[26] 5 exemplarily shows intra-directional modes of 65 prediction directions.

[27] 6 is a diagram for explaining RST according to an embodiment of the present document.

[28] 7 is a diagram illustrating a sequence of arranging output data of a forward primary transformation into a one-dimensional vector according to an example.

[29] 8 is a diagram illustrating a sequence of arranging output data of a forward quadratic transform into a two-dimensional block according to an example.

[30] 9 is a diagram illustrating wide-angle intra prediction modes according to an embodiment of the present document.

[31] 10 is a diagram illustrating a block shape to which LFNST is applied.

[32] 11 is a diagram illustrating an arrangement of output data of a forward LFNST according to an example.

[33] 12 is a diagram illustrating that the number of output data for a forward LFNST is limited to a maximum of 16 according to an example.

[34] 13 is a diagram illustrating zero-out in a block to which 4x4 LFNST is applied, according to an example.

[35] 14 is a diagram illustrating zero-out in a block to which 8x8 LFNST is applied, according to an example.

[36] 15 is a diagram for explaining scanning of a 32x32 transform block according to an example of this document.

[37] 16 is a diagram for describing a method of decoding an image according to an example.

[38] 17 is a diagram for explaining a method of encoding an image according to an example.

[39] 18 exemplarily shows a structural diagram of a content streaming system to which this document is applied.Modes for carrying out the invention

[40] Since this document may have various changes and may have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present document to a specific embodiment. Terms commonly used herein are used only to describe specific embodiments, and are not intended to limit the technical spirit of the present document. The singular expression includes the plural expression unless the context clearly dictates otherwise. As used herein, terms such as “comprise” or “have” are intended to designate that a feature, number, step, action, component, part, or combination thereof described in the specification is present, and includes one or more other features or It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof does not preclude in advance the possibility of addition.

[41] On the other hand, each component in the drawings described in this document is shown independently for the convenience of description regarding different characteristic functions, and does not mean that each component is implemented as separate hardware or separate software. For example, two or more components among each component may be combined to form one component, or one component may be divided into a plurality of components. Embodiments in which each component is integrated and / or separated are also included in the scope of the present document as long as they do not depart from the essence of this document.

[42] Hereinafter, with reference to the accompanying drawings, a preferred embodiment of the present document will be described in more detail. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

[43] This article is about video / image coding. For example, the method / embodiment disclosed in this document is a Versatile Video Coding (VVC) standard (ITU-T Rec. H.266), a next-generation video / image coding standard after VVC, or other video coding related standards ( For example, it may be related to the High Efficiency Video Coding (HEVC) standard (ITU-T Rec. H.265), essential video coding (EVC) standard, AVS2 standard, etc.).

[44] This document presents various embodiments related to video / image coding, and unless otherwise stated, the embodiments may be combined with each other.

[45] In this document, a video may mean a set of a series of images according to the passage of time. A picture generally refers to a unit representing one image in a specific time period, and a slice / tile is a unit constituting a part of a picture in coding. A slice / tile may include one or more coding tree units (CTUs). One picture may consist of one or more slices / tiles. One picture may be composed of one or more tile groups. One tile group may include one or more tiles.

[46] A pixel or pel may mean a minimum unit constituting one picture (or image). Also, 'sample' may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a value of a pixel, may represent only a pixel / pixel value of a luma component, or may represent only a pixel / pixel value of a chroma component. Alternatively, the sample may mean a pixel value in the spatial domain, or when the pixel value is transformed into the frequency domain, it may mean a transform coefficient in the frequency domain.

[47] A unit may represent a basic unit of image processing. The unit may include at least one of a specific region of a picture and information related to the region. One unit may include one luma block and two chroma (ex. cb, cr) blocks. A unit may be used interchangeably with terms such as a block or an area in some cases. In a general case, the MxN block may include samples (or sample arrays) or a set (or arrays) of transform coefficients including M columns and N rows.

[48] In this document, “ / ” and “,” are interpreted as “and / or”. For example, “A / B” would be interpreted as “A and / or B”, and “A, B” would be interpreted as “A and / or B”. Additionally, “A / B / C” means “at least one of A, B and / or C”. Also, “A, B, C” means “at least one of A, B and / or C”.

[49] Additionally, “or” in this document is to be construed as “and / or”. For example, “A or B” may mean 1) only “A”, 2) only “B”, or 3) “A and B”. In other words, “or” in this document may mean “additionally or alternatively”.

[50] As used herein, “at least one of A and B” may mean “only A”, “only B” or “both A and B”. In addition, in the present specification, the expression “at least one of A or B” or “at least one of A and / or B” means “at least one It can be interpreted the same as “at least one of A and B”.

[51] Also, as used herein, “at least one of A, B and C” means “only A”, “only B”, “only C”, or “A, B and C” Any combination of A, B and C”. Also, “at least one of A, B or C” or “at least one of A, B and / or C” means may mean “at least one of A, B and C”.

[52] In addition, parentheses used herein may mean “for example”. Specifically, when “prediction (intra prediction)” is indicated, “intra prediction” may be proposed as an example of “prediction”. In other words, “prediction” in the present specification is not limited to “intra prediction”, and “intra prediction” may be proposed as an example of “prediction”. Also, even when “prediction (ie, intra prediction)” is indicated, “intra prediction” may be proposed as an example of “prediction”.

[53] Technical features that are individually described in one drawing in this specification may be implemented individually or may be implemented at the same time.

[54] 1 schematically shows an example of a video / image coding system to which this document can be applied.

[55] Referring to FIG. 1 , a video / image coding system may include a source device and a receive device. The source device may transmit encoded video / image information or data in the form of a file or streaming to the receiving device through a digital storage medium or a network.

[56] The source device may include a video source, an encoding apparatus, and a transmission unit. The receiving device may include a receiving unit, a decoding apparatus, and a renderer. The encoding apparatus may be referred to as a video / image encoding apparatus, and the decoding apparatus may be referred to as a video / image decoding apparatus. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be configured as a separate device or external component.

[57] The video source may acquire a video / image through a process of capturing, synthesizing, or generating a video / image. A video source may include a video / image capture device and / or a video / image generating device. A video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / images, and the like. A video / image generating device may include, for example, a computer, tablet, and smart phone, and may (electronically) generate a video / image. For example, a virtual video / image may be generated through a computer, etc. In this case, the video / image capturing process may be substituted for the process of generating related data.

[58] The encoding device may encode the input video / image. The encoding apparatus may perform a series of procedures such as prediction, transformation, and quantization for compression and coding efficiency. The encoded data (encoded video / image information) may be output in the form of a bitstream.

[59] The transmitting unit may transmit the encoded video / image information or data output in the form of a bitstream to the receiving unit of the receiving device in the form of a file or streaming through a digital storage medium or a network. The digital storage medium may include a variety of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmission unit may include an element for generating a media file through a predetermined file format, and may include an element for transmission through a broadcast / communication network. The receiver may receive / extract the bitstream and transmit it to the decoding device.

[60] The decoding apparatus may decode the video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding apparatus.

[61] The renderer may render the decoded video / image. The rendered video / image may be displayed through the display unit.

[62] 2 is a diagram schematically illustrating a configuration of a video / image encoding apparatus to which this document can be applied. Hereinafter, a video encoding apparatus may include an image encoding apparatus.

[63] 2, the encoding apparatus 200 includes an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, It may be configured to include an adder 250 , a filter 260 , and a memory 270 . The prediction unit 220 may include an inter prediction unit 221 and an intra prediction unit 222 . The residual processing unit 230 may include a transformer 232 , a quantizer 233 , an inverse quantizer 234 , and an inverse transformer 235 . The residual processing unit 230 may further include a subtractor 231 . The adder 250 may be referred to as a reconstructor or a reconstructed block generator. The above-described image segmentation unit 210, prediction unit 220, residual processing unit 230, entropy encoding unit 240, adder 250 and filtering unit 260 may include one or more hardware components ( For example, by an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB), and may be configured by a digital storage medium. The hardware component may further include a memory 270 as an internal / external component.

[64] The image dividing unit 210 may divide an input image (or a picture, a frame) input to the encoding apparatus 200 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit is to be recursively divided according to a quad-tree binary-tree ternary-tree (QTBTTT) structure from a coding tree unit (CTU) or largest coding unit (LCU). can For example, one coding unit may be divided into a plurality of coding units having a lower depth based on a quad tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quad tree structure may be applied first and a binary tree structure and / or a ternary structure may be applied later. Alternatively, the binary tree structure may be applied first. A coding procedure according to this document may be performed based on the final coding unit that is no longer divided. In this case, the largest coding unit may be directly used as the final coding unit based on coding efficiency according to image characteristics, or the coding unit may be recursively divided into coding units having a lower depth than the optimal coding unit if necessary. A coding unit of the size of may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the above-described final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[65] A unit may be used interchangeably with terms such as a block or an area in some cases. In general, an MxN block may represent a set of samples or transform coefficients including M columns and N rows. A sample may generally represent a pixel or a value of a pixel, and may represent only a pixel / pixel value of a luminance component, or may represent only a pixel / pixel value of a chroma component. A sample may be used as a term corresponding to a picture (or image) as a pixel or a pel.

[66] The subtractor 231 subtracts the prediction signal (predicted block, prediction samples, or prediction sample array) output from the prediction unit 220 from the input image signal (original block, original samples, or original sample array) to obtain a residual. A signal (a residual block, residual samples, or a residual sample array) may be generated, and the generated residual signal is transmitted to the converter 232 . The prediction unit 220 may perform prediction on a processing target block (hereinafter, referred to as a current block) and generate a predicted block including prediction samples for the current block. The prediction unit 220 may determine whether intra prediction or inter prediction is applied on a current block or CU basis. The prediction unit may generate various information about prediction, such as prediction mode information, and transmit it to the entropy encoding unit 240 , as will be described later in the description of each prediction mode. The prediction information may be encoded by the entropy encoding unit 240 and output in the form of a bitstream.

[67] The intra prediction unit 222 may predict the current block with reference to samples in the current picture. The referenced samples may be located in the vicinity of the current block or may be located apart from each other according to the prediction mode. In intra prediction, prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional mode may include, for example, a DC mode and a planar mode (Planar mode). The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes according to the granularity of the prediction direction. However, this is an example, and depending on the setting, more or less number of directional prediction modes may be used. The intra prediction unit 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[68] The inter prediction unit 221 may derive the predicted block for the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks existing in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), or the like, and a reference picture including the temporally neighboring block may be called a collocated picture (colPic). may be For example, the inter prediction unit 221 constructs a motion information candidate list based on neighboring blocks, and provides information indicating which candidate is used to derive a motion vector and / or a reference picture index of the current block. can create Inter prediction can be performed based on various prediction modes, for example, in the case of skip mode and merge mode, The inter prediction unit 221 may use motion information of a neighboring block as motion information of a current block. In the skip mode, unlike the merge mode, a residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the current block is determined by using a motion vector of a neighboring block as a motion vector predictor and signaling a motion vector difference. can direct

[69] The prediction unit 220 may generate a prediction signal based on various prediction methods to be described later. For example, the prediction unit may apply intra prediction or inter prediction for prediction of one block, and may simultaneously apply intra prediction and inter prediction. This can be called combined inter and intra prediction (CIIP). Also, the prediction unit may perform intra block copy (IBC) to predict the block. The intra block copy may be used for video / video coding of content such as games, for example, screen content coding (SCC). IBC basically performs prediction within the current picture, but may be performed similarly to inter prediction in that a reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in this document.

[70] The prediction signal generated by the inter prediction unit 221 and / or the intra prediction unit 222 may be used to generate a reconstructed signal or may be used to generate a residual signal. The transform unit 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include a discrete cosine transform (DCT), a discrete sine transform (DST), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, GBT means a transformation obtained from this graph when expressing relationship information between pixels in a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels and based thereon. In addition, the transformation process may be applied to a block of pixels having the same size as a square, or may be applied to a block of a variable size that is not a square.

Claims

Claims

1. An image decoding method performed by a decoding apparatus, the method comprising: obtaining residual information from a bitstream; deriving a position of a last significant coefficient in a current block and transform coefficients for the current block based on the residual information; and deriving residual samples for the current block by applying an inverse primary transform to the transform coefficient, wherein the inverse primary transform is performed based on an MTS index received from the bitstream, and the MTS index is parsed based on the absence of a significant coefficient in a second region other than the upper-left first region of the current block, and the first region is an upper-left 16X16 region of the current block.

2. The method of claim 1 , wherein the MTS index is parsed based on flag information indicating whether a significant coefficient exists in the second area, and the flag information includes: An image decoding method, characterized in that it is derived by determining whether it exists in the second region.

3. The method of claim 2, wherein, in the flag information, when a flag value indicating whether the significant coefficient exists in the scan sub-block is 1, and a significant coefficient existing in the scan sub-block is located in the second area, An image decoding method, characterized in that it indicates that the significant coefficient exists in the second region.

4. The method of claim 2, wherein, in the flag information, a flag value indicating whether the significant coefficient exists in the scan sub-block is 1, and when the scan sub-block is located in the second area, the flag information is stored in the second area. An image decoding method, characterized in that it indicates that the significant coefficient exists.

5. The method of claim 2, wherein the flag information is initially set to 1, and when the significant coefficient is located in the second region, the flag information is changed to 0.

6. The method of claim 2, wherein the scan sub-block is a 4X4 block, and the scan sub-block is scanned in an inverse diagonal scan direction from the position of the last significant coefficient in the current block.

7. An image encoding method performed by an image encoding apparatus, the method comprising: deriving prediction samples for a current block; deriving residual samples for the current block based on the prediction sample; deriving transform coefficients for the current block by applying MTS to the residual samples; zeroing out a second area excluding the first area at the upper left of the current block; deriving residual information based on the transform coefficient of the first region, and image information such that an MTS index indicating a transform kernel of the MTS is parsed based on whether a significant coefficient exists in the second region An image encoding method comprising the step of configuring, wherein the first area is an upper left 16X16 area of ​​the current block.

8. 8. The method of claim 7, wherein the MTS index is encoded based on flag information indicating whether a significant coefficient exists in the second region, and the flag information includes: An image encoding method, characterized in that it is derived by determining whether it exists in the second region.

9. The method of claim 8, wherein, in the flag information, when a flag value indicating whether the significant coefficient exists in the scan sub-block is 1, and the significant coefficient existing in the scan sub-block is located in the second area, and indicating that the significant coefficient exists in the second region.

10. The method of claim 8, wherein, in the flag information, a flag value indicating whether the significant coefficient exists in the scan sub-block is 1, and when the scan sub-block is located in the second area, the flag information is stored in the second area. An image encoding method, characterized in that it indicates that the significant coefficient exists.

11. The method of claim 8, wherein the flag information is initially set to 1, and when the significant coefficient is located in the second region, the flag information is changed to 0.

12. The method of claim 8, wherein the scan sub-block is a 4X4 block, and the scan sub-block is scanned in an inverse diagonal scan direction from the position of the last significant coefficient in the current block.

13. A computer readable digital storage medium having stored thereon instruction information for causing an image decoding method to be performed, the image decoding method comprising: obtaining residual information from a bitstream; deriving a position of a last significant coefficient in a current block and transform coefficients for the current block based on the residual information; and deriving residual samples for the current block by applying an inverse primary transform to the transform coefficient, wherein the inverse primary transform is performed based on an MTS index received from the bitstream, and the MTS index is parsed based on the fact that a significant coefficient does not exist in a second area other than the upper left first area of ​​the current block, and the first area is an upper left 16X16 area of ​​the current block.