Image encoding / decoding method and device, and method for transmitting bitstream
By using reference sample filtering to select the appropriate filter during intra prediction, the problem of low encoding and decoding efficiency of high resolution images is solved, and efficient image transmission and storage is achieved.
Patent Information
- Application Number
- CN202411135694.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2019-03-12
- Filing Date
- 2020-03-12
- Publication Date
- 2024-11-12
AI Technical Summary
The prior art is difficult to efficiently encode and decode high-resolution, high-quality images, resulting in increased transmission and storage costs.
By using reference sample filtering during intra prediction, the appropriate filter is selected to encode and decode the image, generate a bit stream, and transmit and store it through the bit stream.
The efficiency of image encoding and decoding is improved, transmission and storage costs are reduced, and efficient image transmission and storage are achieved.
Smart Images

Figure CN118945338A_ABST
Abstract
Description
[0001] This application is a divisional application of a patent application with application number 202080020342.3 (PCT / KR2020 / 003478), which was submitted to the China Patent Office on September 10, 2021, with an international application date of March 12, 2020, and the invention name is “Image encoding / decoding method and device, and method for sending bit stream”. Technical Field
[0002] The present disclosure relates to an image encoding / decoding method and apparatus, and a method for transmitting a bitstream, and more particularly, to a method and apparatus for encoding / decoding an image by performing intra-frame prediction using reference sample filtering, and a method for transmitting a bitstream generated thereby. Background Art
[0003] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) images and ultra-high-definition (UHD) images, is increasing in various fields. As the resolution and quality of image data are improved, the amount of information or bits transmitted is relatively increased compared to existing image data. The increase in the amount of information or bits transmitted leads to an increase in transmission cost and storage cost.
[0004] Therefore, efficient image compression technology is needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the invention
[0005] Technical issues
[0006] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.
[0007] Furthermore, an object of the present disclosure is to provide a method and apparatus for encoding / decoding an image by selecting a filter applied when deriving a prediction sample according to whether filtering of a sample referenced during intra prediction is used.
[0008] Another object of the present disclosure is to provide a method for transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure.
[0009] Another object of the present disclosure is to provide a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.
[0010] Another object of the present disclosure is to provide a recording medium storing a bit stream received and decoded by the image decoding apparatus according to the present disclosure and used to reconstruct an image.
[0011] The technical problems solved by the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not described here will be clear to those skilled in the art through the following description.
[0012] Technical Solution
[0013] To solve the above problem, an image decoding method performed by an image decoding device according to an embodiment includes: obtaining intra-frame prediction mode information of a current block from a bitstream, determining the intra-frame prediction mode of the current block based on the intra-frame prediction mode information and the intra-frame prediction modes of neighboring blocks located around the current block, deriving a value of a first filter flag for filtering intra-frame prediction reference samples of the current block based on coding parameters for the current block, applying smoothing filtering to the intra-frame prediction reference samples of the current block when the value of the first filter flag indicates filtering using the intra-frame prediction reference samples of the current block, selecting an interpolation filter for intra-frame prediction of the current block based on the first filter flag, and generating an intra-frame prediction block of the current block using the selected interpolation filter.
[0014] When the first filter flag indicates that the intra prediction reference samples of the current block have been smoothed, a smoothing filter may be selected as an interpolation filter for intra prediction of the current block and applied to the reference samples to derive prediction samples of the current block.
[0015] When the first filter flag indicates that the intra prediction reference samples of the current block are not smoothed, a sharpening filter may be selected as an interpolation filter for intra prediction of the current block and applied to the reference samples to derive prediction samples of the current block.
[0016] To this end, in order to solve the above-mentioned problem, the image decoding method performed by the image decoding device of the embodiment includes: obtaining intra-frame prediction mode information of the current block from the bit stream, determining the intra-frame prediction mode of the current block based on the intra-frame prediction mode information and the intra-frame prediction modes of the neighboring blocks located around the current block, deriving the value of the first filter flag for filtering the intra-frame prediction reference samples of the current block based on the coding parameters for the current block, selecting the interpolation filter for intra-frame prediction of the current block based on the first filter flag, and generating the intra-frame prediction block of the current block using the selected interpolation filter.
[0017] The method may further include applying a smoothing filter to the intra-prediction reference samples of the current block when the value of the first filter flag indicates filtering using the intra-prediction reference samples of the current block.
[0018] The coding parameters may include at least one of a multi-reference line index of the current block, a size of the current block, a color component of the current block, whether intra sub-partitioning (ISP) is applied to the current block, or an intra prediction mode of the current block.
[0019] When the value of the first filter flag is a first value, filtering of intra-frame prediction reference samples may not be used, and when the value of the first filter flag is a second value, filtering of intra-frame prediction reference samples may be used, and when the value of the first filter flag is a first value, the interpolation filter used for intra-frame prediction of the current block may be determined as a filter having a sharpening characteristic, and when the value of the first filter flag is a second value, the interpolation filter used for intra-frame prediction of the current block may be determined as a filter having a smoothing characteristic.
[0020] Selecting the interpolation filter for intra prediction of the current block based on the first filter flag may include deriving a value of a second filter flag of the interpolation filter for intra prediction of the current block based on the first filter flag, and selecting the interpolation filter based on the second filter flag.
[0021] The value of the second filter flag may be derived as a first value when a condition for at least one of the coding parameters is met.
[0022] When intra sub-partitioning (ISP) is applied to the current block, the value of the second filter flag may be derived as a first value.
[0023] When intra sub-partitioning (ISP) is applied to the current block, the current block is a luminance block, the intra prediction mode of the current block is equal to or greater than a predetermined mode, and the width of the current block is equal to or greater than a predetermined size, the value of the second filter flag may be derived as a first value. In addition, the predetermined mode may be an upper left diagonal mode and the predetermined size may be 8.
[0024] When intra sub-partitioning (ISP) is applied to the current block, the current block is a luma block, the intra prediction mode of the current block is equal to or less than a predetermined mode, and the width of the current block is equal to or greater than a predetermined size, the value of the second filter flag may be derived as a first value. The predetermined mode may be an upper left diagonal mode and the predetermined size may be 8.
[0025] When the value of the first filter flag is a first value, filtering of intra-frame prediction reference samples may not be used; when the value of the first filter flag is a second value, filtering of intra-frame prediction reference samples may be used; and when the value of the first filter flag is 1, the value of the second filter flag may be derived as 0.
[0026] When the value of the second filter flag is a first value, the interpolation filter may be a filter having a sharpening characteristic, when the value of the second filter flag is a second value, the interpolation filter may be a filter having a smoothing characteristic, and when the value of the first filter flag is a second value, the value of the second filter flag may be derived as a first value.
[0027] To solve the above problem, an image decoding device according to an embodiment includes a memory and at least one processor. The at least one processor can obtain intra-frame prediction mode information of a current block from a bitstream, determine the intra-frame prediction mode of the current block based on the intra-frame prediction mode information and the intra-frame prediction modes of neighboring blocks located around the current block, derive the value of a first filter flag for filtering of intra-frame prediction reference samples of the current block based on the coding parameters for the current block, select an interpolation filter for intra-frame prediction of the current block based on the first filter flag, and generate an intra-frame prediction block of the current block using the selected interpolation filter.
[0028] To solve the above problem, the image encoding method performed by the image encoding device according to the embodiment includes: selecting an intra-frame prediction mode of a current block, deriving a value of a first filter flag for filtering an intra-frame prediction reference sample of the current block based on a coding parameter for the current block, selecting an interpolation filter for intra-frame prediction of the current block based on the first filter flag, generating an intra-frame prediction block of the current block using the selected interpolation filter, and encoding the intra-frame prediction mode of the current block based on the intra-frame prediction modes of neighboring blocks located around the current block.
[0029] To solve the above problems, the image encoding apparatus according to the embodiment may transmit the bit stream generated as described above.
[0030] Beneficial Effects
[0031] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.
[0032] In addition, according to the present disclosure, a method and apparatus for encoding / decoding an image by selecting a filter applied when deriving a prediction sample according to whether filtering of a sample referenced during intra prediction is used may be provided.
[0033] Furthermore, according to the present disclosure, a method of transmitting a bit stream generated by the image encoding method or apparatus according to the present disclosure may be provided.
[0034] Furthermore, according to the present disclosure, a recording medium storing a bit stream generated by the image encoding method or apparatus according to the present disclosure can be provided.
[0035] Furthermore, according to the present disclosure, it is possible to provide a recording medium storing a bit stream received and decoded by the image decoding device according to the present disclosure and used to reconstruct an image.
[0036] Those skilled in the art will understand that the effects that can be achieved through the present disclosure are not limited to the contents that have been specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] Figure 1 is a view schematically illustrating a video coding system according to an embodiment.
[0038] Figure 2 is a view schematically illustrating an image encoding device according to an embodiment.
[0039] Figure 3 is a view schematically illustrating an image decoding device according to an embodiment.
[0040] Figure 4 and Figure 5 is a view illustrating an intra prediction direction according to an embodiment.
[0041] Figure 6 is a flowchart illustrating intra prediction steps according to an embodiment.
[0042] Figure 7 is a flowchart illustrating reference sample construction steps according to an embodiment.
[0043] Figure 8 is a flowchart illustrating a method of determining, by a decoding apparatus, whether to perform reference sample filtering according to an embodiment.
[0044] Fig. 9 is a flowchart illustrating a method of determining a filter for deriving a prediction sample depending on whether reference sample filtering has been performed by a decoding device according to an embodiment.
[0045] Fig.10 is a flowchart illustrating a method of determining, by a decoding apparatus, a filter for deriving a sample value of a current block using filterFlag as a parameter indicating whether to perform reference sample filtering according to an embodiment.
[0046] Figures 11 to 12 It is a graphic Fig.10 Another example of a view.
[0047] Fig.13 is a flowchart illustrating a method of generating intra prediction samples according to a result of reference sample filtering performed by a decoding apparatus according to an embodiment.
[0048] Fig.14 is a flowchart illustrating a method of performing a directional intra prediction mode by a decoding apparatus according to an embodiment.
[0049] Fig.15 is a flowchart illustrating a method of deriving a modified filterFlag value by a decoding device according to an embodiment.
[0050] Fig.16is a mapping table between predModeIntra and intraPredAngle, which are parameters indicating an angle, according to an embodiment.
[0051] Fig.17 is a table illustrating coefficients of a cubic filter (fC) and a Gaussian filter (fG) as interpolation filters for intra prediction according to an embodiment.
[0052] Figures 18 to 20 is a flowchart illustrating a method of deriving a reference sample array ref[x] by a decoding apparatus according to an embodiment.
[0053] Figures 21 to 26 is a view illustrating an example of a reference sample array for generating a prediction sample of a current block.
[0054] Fig. 27 is a flowchart illustrating a process of determining, by a decoding apparatus, a condition for performing reference sample filtering according to an embodiment.
[0055] Fig.28 is a view illustrating an example of a target sample array to which a [1 2 1] / 4 filter is applied and a result sample array of applying the [1 2 1] / 4 filter.
[0056] Fig.29 is a view illustrating another example of performing intra prediction using reference sample filtering by a decoding apparatus according to an embodiment.
[0057] Figures 30 to 31 is a view illustrating another embodiment in which intra prediction is performed using reference sample filtering by a decoding apparatus according to an embodiment.
[0058] Fig.32 is a view illustrating an example of generating intra prediction samples using a linear interpolation filter by a decoding apparatus according to an embodiment.
[0059] Fig.33 is a view illustrating another embodiment in which intra prediction is performed using reference sample filtering by a decoding apparatus according to an embodiment.
[0060] Fig.34 is a view illustrating a content streaming system according to an embodiment. DETAILED DESCRIPTION
[0061] Hereinafter, the embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, the present disclosure can be implemented in various forms and is not limited to the embodiments described herein.
[0062] When describing the present disclosure, if it is determined that the detailed description of related known functions or configurations makes the scope of the present disclosure unnecessarily ambiguous, the detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals are given to like parts.
[0063] In the present disclosure, when a component is "connected", "coupled" or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with intermediate components. In addition, when a component "includes" or "has" other components, unless otherwise specified, it means that other components may also be included, rather than excluding other components.
[0064] In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise specified. Accordingly, within the scope of the present disclosure, the first component in one embodiment may be referred to as the second component in another embodiment, and similarly, the second component in one embodiment may be referred to as the first component in another embodiment.
[0065] In the present disclosure, components that are distinguished from each other are intended to clearly describe each feature and do not mean that the components must be separated. That is, multiple components can be integrated and implemented in one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even if not specifically stated, embodiments in which these components are integrated or distributed are also included in the scope of the present disclosure.
[0066] In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some components may be optional components. Therefore, the embodiments consisting of a subset of the components described in the embodiments are also included in the scope of the present disclosure. In addition, the embodiments further including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.
[0067] The present disclosure relates to encoding and decoding of images. Unless otherwise defined in the present disclosure, terms used in the present disclosure may have general meanings commonly used in the technical field to which the present disclosure belongs.
[0068] Hereinafter, the present disclosure discloses a video / image coding technology. The coding / decoding methods disclosed in the present disclosure and embodiments may be applied to methods disclosed in the Versatile Video Coding (VVC) standard or the next generation video / image coding standard.
[0069] The present disclosure provides various embodiments related to video / image coding, and unless otherwise specified, these embodiments may be performed in combination with each other.
[0070] In the present disclosure, a "picture" generally refers to a unit representing an image within a specific time period, and a slice / tile is a coding unit that constitutes a part of a picture. A picture may be composed of one or more slices / tiles. In addition, a slice / tile may include one or more coding tree units (CTUs).
[0071] In the present disclosure, "pixel" or "picture element" may represent the smallest single unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, or may represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.
[0072] In the present disclosure, a "unit" may refer to a basic unit of image processing. The unit may include at least one of a specific area of a picture and information related to the area. In some cases, the unit may be used interchangeably with terms such as "sample array", "block" or "area". In general, an M×N block may include M columns and N rows of samples (or sample arrays) or a set (or array) of transform coefficients.
[0073] In the present disclosure, the "current block" may mean one of the "current coding block", the "current coding unit", the "coding target block", the "decoding target block", or the "processing target block". When prediction is performed, the "current block" may mean the "current prediction block" or the "prediction target block". When transform (inverse transform) / quantization (dequantization) is performed, the "current block" may mean the "current transform block" or the "transform target block". When filtering is performed, the "current block" may mean the "filtering target block".
[0074] In the present disclosure, a slash ( / ) or a comma may be interpreted as "and / or". For example, "A / B" and "A, B" may be interpreted as "A and / or B". In addition, "A / B / C" and "A, B, and C" may mean "at least one of A, B, and / or C".
[0075] In the present disclosure, "or" may be interpreted as "and / or". For example, "A or B" may mean 1) only "A", 2) only "B", or 3) "A and B". Alternatively, in the present disclosure, "or" may mean "additionally or alternatively".
[0076] Video compilation system overview
[0077] Figure 1 1 is a view schematically illustrating a video coding system according to an embodiment. The video coding system according to an embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may transmit the encoded video and / or image information or data in the form of a file or stream via a digital storage medium or a network to the decoding device 20.
[0078] The encoding device 10 according to the embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding device 20 according to the embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display and the display may be configured as a separate device or an external component.
[0079] The video source generator 11 can obtain the video / image by capturing, synthesizing or generating the video / image process. The video source generator 11 may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc. In this case, the video / image capturing process may be replaced by a generating-related process.
[0080] The encoding unit 12 may encode the input video / image. For compression and encoding efficiency, the encoding unit 12 may perform a series of processes such as prediction, transformation, and quantization. The encoding unit 12 may output encoded data (encoded video / image information) in the form of a bitstream.
[0081] The transmitter 13 may transmit the encoded video / image information or the data output in the form of a bit stream to the receiver 21 of the decoding device 20 in the form of a file or stream through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 may include an element for generating a media file in a predetermined file format, and may include an element for transmitting through a broadcast / communication network. The receiver 21 may extract / receive a bit stream from a storage medium or a network and transmit the bit stream to the decoding unit 22.
[0082] The decoding unit 22 may decode a video / image by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transformation, and prediction.
[0083] The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through a display.
[0084] Overview of image coding device
[0085] Figure 2FIG. 2 is a diagram schematically illustrating an image encoding apparatus according to an embodiment. Figure 2 As shown, the image encoding device 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as a "predictor". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.
[0086] In some embodiments, all or at least some of the components configuring the image encoding apparatus 100 may be configured by one hardware component (eg, an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.
[0087] The image divider 110 may divide the input image (or picture or frame) input to the image encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). The coding unit may be obtained by recursively dividing a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QT / BT / TT) structure. For example, a coding unit may be divided into a plurality of coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the division of the coding unit, the quadtree structure may be applied first, and then the binary tree structure and / or the ternary tree structure may be applied. The coding process according to the present disclosure may be performed based on the final coding unit that is no longer divided. The maximum coding unit may be used as the final coding unit, and the coding unit of a deeper depth obtained by dividing the maximum coding unit may also be used as the final coding unit. Here, the coding process may include the prediction, transformation, and reconstruction processes described later. As another example, the processing unit of the coding process may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transform unit may be partitioned or divided from the final coding unit. The prediction unit may be a sample prediction unit, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.
[0088] The predictor (inter predictor 180 or intra predictor 185) may perform prediction on the block to be processed (current block) and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction based on the current block or CU. The predictor may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. Information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.
[0089] The intra-frame predictor 185 can predict the current block by referring to the samples in the current picture. Depending on the intra-frame prediction mode and / or the intra-frame prediction technology, the reference samples can be located in the neighbors of the current block or can be placed separately. The intra-frame prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional mode may include, for example, a DC mode and a plane mode. Depending on the level of detail of the prediction direction, the directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-frame predictor 185 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.
[0090] The inter-frame predictor 180 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of motion information between the adjacent blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the adjacent blocks may include spatial adjacent blocks existing in the current picture and temporal adjacent blocks existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal adjacent block may be the same or different. The temporal adjacent block may be referred to as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal adjacent block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 180 may configure a motion information candidate list based on the adjacent blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-frame prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter-frame predictor 180 may use the motion information of the neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, the residual signal may not be transmitted. In the case of motion vector prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference may represent the difference between the motion vector of the current block and the motion vector predictor.
[0091] The predictor can generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the predictor can apply not only intra prediction or inter prediction, but also intra prediction and inter prediction at the same time to predict the current block. The prediction method that applies both intra prediction and inter prediction to predict the current block can be called combined inter and intra prediction (CIIP). In addition, the predictor can perform intra-block copying (IBC) to predict the current block. Intra-block copying can be used for content video / image coding of games, etc., for example, screen content coding (SCC). IBC is a method of predicting the current picture using a previously reconstructed reference block in the current picture at a position separated by a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to a predetermined distance. IBC basically performs prediction in the current picture, but can be performed similarly to inter prediction because the reference block is derived within the current picture. That is, IBC can use at least one of the inter prediction techniques described in the present disclosure.
[0092] The prediction signal generated by the predictor can be used to generate a reconstruction signal or to generate a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal can be transmitted to the transformer 120.
[0093] The transformer 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to blocks of variable size instead of square.
[0094] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients of the block type into a one-dimensional vector form based on the coefficient scanning order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.
[0095] The entropy encoder 190 may perform various encoding methods, such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 190 may encode information required for video / image reconstruction (e.g., values of syntax elements, etc.) in addition to quantized transform coefficients together or individually. The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of a network abstraction layer (NAL) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The signaled information, the transmitted information, and / or the syntax elements described in the present disclosure may be encoded and included in the bitstream through the above-mentioned encoding process.
[0096] The bitstream may be transmitted over a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 190 and / or a storage unit (not shown) that stores the signal may be included as an internal / external element of the image encoding device 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.
[0097] The quantized transform coefficients output from the quantizer 130 may be used to generate a residual signal. For example, the residual signal (residual block or residual sample) may be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150.
[0098] The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame predictor 180 or the intra-frame predictor 185 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If there is no residual for the block to be processed, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 155 can be called a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.
[0099] Meanwhile, as described later, luminance mapping with chroma scaling (LMCS) may be applied to the picture encoding process.
[0100] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 160 can generate various information related to filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 190 and output in the form of a bit stream.
[0101] The modified reconstructed picture transferred to the memory 170 may be used as a reference picture in the inter predictor 180. When inter prediction is applied by the image encoding device 100, prediction mismatch between the image encoding device 100 and the image decoding device may be avoided and encoding efficiency may be improved.
[0102] The DPB of the memory 170 may store the modified reconstructed picture for use as a reference picture in the inter-frame predictor 180. The memory 170 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the reconstructed block in the picture. The stored motion information may be transmitted to the inter-frame predictor 180 and used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 may store the reconstructed samples of the reconstructed blocks in the current picture and may transmit the reconstructed samples to the intra-frame predictor 185.
[0103] Overview of image decoding device
[0104] Figure 3 FIG. 2 is a diagram schematically illustrating an image decoding apparatus according to an embodiment. Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as a "predictor". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.
[0105] According to an embodiment, all or at least some of the plurality of components configuring the image decoding apparatus 200 may be configured by hardware components (eg, a decoder or a processor). In addition, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.
[0106] The image decoding apparatus 200 having received a bit stream including video / image information may perform the same operation as that performed by Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding device 100. For example, the image decoding device 200 may perform decoding using a processing unit applied in the image encoding device. Therefore, the processing unit of decoding may be, for example, a coding unit.
[0107] The image decoding apparatus 200 may receive the image in the form of a bit stream from Figure 2The received signal may be decoded by the entropy decoder 210. For example, the entropy decoder 210 may parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may also include general constraint information. The image decoding device may also decode the picture based on the parameter set information and / or the general constraint information. The information and / or syntax elements signaled / received described in the present disclosure may be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values of the syntax elements required for image reconstruction and the quantized values of the transform coefficients of the residual. More specifically, the CABAC entropy decoding method can receive a bin corresponding to each syntax element in the bitstream, use the decoding target syntax element information, the decoding information of the adjacent block and the decoding target block, or the information of the symbol / bin decoded in the previous stage to determine the context model, and arithmetically decode the bin by predicting the probability of occurrence of the bin according to the determined context model, and generate a symbol corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model by using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. The information related to the prediction in the information decoded by the entropy decoder 210 can be provided to the predictor (inter-frame predictor 260 and intra-frame predictor 265), and the residual value for which entropy decoding is performed in the entropy decoder 210, that is, the quantized transform coefficient and related parameter information can be input to the dequantizer 220. In addition, information about filtering among the information decoded by the entropy decoder 210 can be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the image encoding device may be further configured as an internal / external element of the image decoding device 200 , or the receiver may be a component of the entropy decoder 210 .
[0108] Meanwhile, the image decoding device according to the present disclosure may be referred to as a video / image / picture decoding device. The image decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 160, or at least one of an intra-frame predictor.
[0109] The dequantizer 220 may dequantize the quantized transform coefficient and output the transform coefficient. The dequantizer 220 may rearrange the quantized transform coefficient in the form of a two-dimensional block. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the image encoding device. The dequantizer 220 may perform dequantization on the quantized transform coefficient by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.
[0110] The inverse transformer 230 may inversely transform the transform coefficients to obtain a residual signal (residual block, residual sample array).
[0111] The predictor may perform prediction on the current block and generate a prediction block including prediction samples of the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on information about the prediction output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction technique).
[0112] As described in the predictor of the image encoding device 100 , the predictor can generate a prediction signal based on various prediction methods (techniques) described later.
[0113] The intra predictor 265 may predict the current block by referring to samples in the current picture. The description of the intra predictor 185 is also applicable to the intra predictor 265.
[0114] The inter-frame predictor 260 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the adjacent blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, the adjacent blocks may include spatial adjacent blocks present in the current picture and temporal adjacent blocks present in the reference picture. For example, the inter-frame predictor 260 may configure a motion information candidate list based on the adjacent blocks and derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode of the current block.
[0115] The adder 235 can generate a reconstructed block by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 260 and / or the intra-predictor 265). If the block to be processed has no residual, for example, when the skip mode is applied, the prediction block can be used as the reconstructed block. The description of the adder 155 is also applicable to the adder 235. The adder 235 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can be used for inter-frame prediction of the next picture by filtering as described below.
[0116] Meanwhile, as described later, Luma Mapping and Chroma Scaling (LMCS) may be applied to the picture decoding process.
[0117] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.
[0118] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame predictor 260. The memory 250 can store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the reconstructed block in the picture. The stored motion information can be transmitted to the inter-frame predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 250 can store the reconstructed samples of the reconstructed block in the current picture and transmit the reconstructed samples to the intra-frame predictor 265.
[0119] In the present disclosure, the embodiments described in the filter 160, the inter-frame predictor 180, and the intra-frame predictor 185 of the image encoding device 100 may be equally or correspondingly applied to the filter 240, the inter-frame predictor 260, and the intra-frame predictor 265 of the image decoding device 200. The embodiments described in the filter 240, the inter-frame prediction unit 260, and the intra-frame prediction unit 265 of the image decoding device 200 may be applied to the same or corresponding to the filter unit 160, the inter-frame prediction unit 180, and the intra-frame prediction unit 185 of the image encoding device 100, respectively.
[0120] Intra Prediction Overview
[0121] Hereinafter, an intra prediction method according to an embodiment will be described. Intra prediction may indicate a prediction of generating a prediction sample for a current block based on a reference sample in a picture to which the current block belongs (hereinafter referred to as the current picture). When intra prediction is applied to the current block, adjacent reference samples to be used for intra prediction of the current block may be derived. The adjacent reference samples of the current block may include samples adjacent to the left boundary of the current block having a size of nWxnH and a total of 2xnH samples adjacent to the lower left corner, samples adjacent to the upper boundary of the current block and a total of 2xnW samples adjacent to the upper right corner, and one sample adjacent to the upper left corner of the current block. Alternatively, the adjacent reference samples of the current block may include multiple columns of top adjacent samples and multiple rows of left adjacent samples. In addition, the adjacent reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block having a size of nWxnH, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right corner of the current block. At the same time, when the ISP to be described later is applied, the adjacent reference samples may be derived in units of sub-partitions.
[0122] On the other hand, some neighboring reference samples of the current block have not been decoded or may not be available. In this case, the decoding device can construct neighboring reference samples for prediction by replacing unavailable samples with available samples. Alternatively, interpolation of available samples can be used to construct neighboring reference samples to be used for prediction.
[0123] When deriving adjacent reference samples, (i) prediction samples may be derived based on an average or interpolation of adjacent reference samples of the current block, and (ii) prediction samples may be derived based on reference samples present in a specific (prediction) direction relative to the prediction samples in the adjacent reference samples of the current block. The case of (i) may be referred to as a non-directional mode or a non-angle mode and the case of (ii) may be referred to as a directional mode or an angle mode. In addition, prediction samples may be generated based on the prediction samples of the current block in the adjacent reference samples by interpolating the second adjacent samples and the first adjacent samples located in the opposite direction of the prediction direction of the intra prediction mode of the current block. The above case may be referred to as linear interpolation intra prediction (LIP). In addition, chroma prediction samples may be generated based on luminance samples using a linear model. This case may be referred to as LM mode. In addition, temporary prediction samples of the current block may be derived based on filtered adjacent reference samples, and the prediction samples of the current block may be derived by weighted summing the temporary prediction samples with at least one reference sample derived from the intra prediction mode in the existing adjacent reference samples (i.e., unfiltered adjacent reference samples). The above case may be referred to as position-dependent intra prediction (PDPC). In addition, the reference sample line with the highest prediction accuracy can be selected from multiple adjacent reference sample lines of the current block to derive the prediction sample using the reference sample located in the prediction direction of the corresponding line. At this time, the intra-frame prediction coding can be performed by indicating (notifying with a signal) the reference sample line used to the decoding device. The above situation can be called multi-reference line (MRL) intra-frame prediction or MRL-based intra-frame prediction. In addition, the current block can be divided into vertical or horizontal sub-partitions for intra-frame prediction based on the same intra-frame prediction mode, and adjacent reference samples can be derived and used in units of sub-partitions. That is, in this case, the intra-frame prediction mode of the current block is equally applied to the sub-partition, and the adjacent reference samples are derived and used in units of sub-partitions, thereby improving the intra-frame prediction performance. This prediction method can be called intra-frame sub-partitioning (ISP) or intra-frame prediction based on ISP. In addition, when the prediction direction based on the prediction sample indicates the space between adjacent reference samples, that is, when the prediction direction indicates a fractional sample position, the value of the prediction sample can be derived by interpolation of multiple reference samples located around the prediction direction (around the fractional sample position). The above intra prediction method may be referred to as an intra prediction type to distinguish it from an intra prediction mode. In addition, after generating a prediction signal of a sub-sampled pixel set of a current block using reconstructed adjacent pixels located on the left and top of the current block, the generated prediction signal and adjacent sample values may be interpolated in the vertical direction and the horizontal direction to generate a prediction signal having an original size, thereby applying a matrix weighted intra prediction (MIP) to perform intra prediction of the current block.
[0124] The intra prediction type may be referred to as various terms, such as an intra prediction scheme or an additional intra prediction mode. For example, the intra prediction type (or additional intra prediction mode) may include at least one of LIP, PDPC, MRL, ISP, or MIP. Information about the intra prediction type may be encoded by an encoding device, included in a bitstream, and signaled to a decoding device. Information about the intra prediction type may be implemented in various forms, such as flag information indicating whether each intra prediction type is applied or index information indicating one of several intra prediction types.
[0125] At the same time, if necessary, the derived prediction samples can be post-filtered. Specifically, the intra prediction process may include an intra prediction mode / type determination step, an adjacent reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. In addition, if necessary, the derived prediction samples can be post-filtered.
[0126] Hereinafter, a video / image encoding method based on intra-frame prediction will be described. First, the encoding device performs intra-frame prediction for the current block. The encoding device may derive the intra-frame prediction mode / type of the current block, derive the adjacent reference samples of the current block, and generate the prediction samples in the current block based on the intra-frame prediction mode / type and the adjacent reference samples. Here, the intra-frame prediction mode / type determination, adjacent reference sample derivation and prediction sample generation processes may be performed simultaneously, or any one process may be performed before the other processes. At the same time, when the prediction sample filtering process described below is performed, the intra-frame predictor 185 may also include a prediction sample filter. The encoding device may determine the mode / type applied to the current block among multiple intra-frame prediction modes / types. The encoding device may compare the rate-distortion (RD) costs of the intra-frame prediction modes / types and determine the optimal intra-frame prediction mode / type for the current block.
[0127] At the same time, the encoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post-filtering. Through the prediction sample filtering process, some or all prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.
[0128] Next, the encoding device may generate residual samples of the current block based on the prediction samples. The encoding device may compare the original samples of the current block with the prediction samples in terms of phase and derive the residual samples.
[0129] Next, the encoding device may encode image information including information about intra prediction (prediction information) and residual information about residual samples. The prediction information may include intra prediction mode information and intra prediction type information. The encoding device may output the encoded image information in the form of a bit stream. The output bit stream may be transmitted to a decoding device via a storage medium or a network.
[0130] The residual information may include a residual encoding syntax, which will be described later. The encoding device may transform / quantize the residual samples and derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.
[0131] At the same time, as described above, the encoding device can generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the encoding device can perform inverse quantization / inverse transformation on the quantized transform coefficients and derive (modified) residual samples. The reason for transforming / quantizing the residual samples and then inverse quantizing / inverse transforming is to derive the same residual samples as the residual samples derived by the above-mentioned decoding device. The encoding device can generate a reconstructed block including reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. Based on the reconstructed block, a reconstructed picture of the current picture can be generated. As described above, the loop filtering process is applicable to the reconstructed picture.
[0132] Hereinafter, a video / image encoding method based on intra prediction will be described. The decoding device may perform operations corresponding to those performed by the encoding device.
[0133] First, the decoding device may derive the intra prediction mode / type of the current block based on the received prediction information (intra prediction mode / type information). The decoding device may derive the adjacent reference samples of the current block. The decoding device may generate prediction samples in the current block based on the intra prediction mode / type and the adjacent reference samples. In this case, the decoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post filtering. Through the prediction sample filtering process, some or all prediction samples may be filtered. In some cases, the prediction sample filtering process may be omitted.
[0134] The decoding device may generate residual samples of the current block based on the received residual information. The decoding device may generate reconstructed samples of the current block based on the predicted samples and the residual samples and derive a reconstructed block including the reconstructed samples. Based on the reconstructed block, a reconstructed picture of the current picture may be generated. The loop filtering process is further applied to the reconstructed picture.
[0135] The intra-frame prediction mode information may include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether the most probable mode (MPM) or the residual mode is applied to the current block, and when the MPM is applied to the current block, the prediction mode information may also include index information (e.g., intra_luma_mpm_idx) indicating one of the intra-frame prediction mode candidates (MPM candidates). The intra-frame prediction mode candidates (MPM candidates) may configure an MPM candidate list or an MPM list. For example, the MPM candidate list may include the intra-frame prediction modes of neighboring blocks or a preset basic intra-frame prediction mode. In addition, when the MPM is not applied to the current block, the intra-frame prediction mode information may also include residual mode information (e.g., intra_luma_mpm_remainder), which indicates one of the remaining intra-frame prediction modes other than the intra-frame prediction mode candidates (MPM candidates). The decoding device may determine the intra-frame prediction mode of the current block based on the intra-frame prediction mode information.
[0136] In addition, the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block, and which reference sample line is used if applied, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of the sub-partition when ISP is applied, flag information indicating whether PDPC is applied, or at least one of flag information indicating whether LIP is applied.
[0137] The intra-frame prediction mode information and / or the intra-frame prediction type information may be encoded / decoded by the coding method described in the present disclosure. For example, the intra-frame prediction mode information and / or the intra-frame prediction type information may be encoded / decoded by entropy coding (e.g., CABAC, CAVLC) based on truncated (rice) binary code.
[0138] Hereinafter, the intra prediction mode will be described in more detail. Figure 4 In order to capture any edge direction present in natural video, such as Figure 4 As shown, the intra prediction modes may include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes may include a plane intra prediction mode and a DC intra prediction mode, and the directional intra prediction modes may include the second to 66th intra prediction modes.
[0139] Meanwhile, in addition to the above intra prediction modes, the intra prediction mode may also include a cross component linear model (CCLM) mode for chroma samples. The CCLM mode may be divided into L_CCLM, T_CCLM, LT_CCLM according to whether the left sample, the upper sample, or both are considered for LM parameter derivation, and may be applied only to the chroma component. For example, the intra prediction mode may be indexed as shown in the following table.
[0140] [Table 1]
[0141] Intra prediction mode Association Name 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM,INTRA_L_CCLM,INTRA_T_CCLM
[0142] Figure 5 FIG. 2 shows the intra prediction direction according to another embodiment. Here, the dotted line direction shows the wide angle mode applied only to non-square blocks. Figure 5 As shown, in order to capture any edge direction presented in natural video, the intra prediction mode according to the embodiment may include two non-directional intra prediction modes and 93 directional intra prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode and a DC intra prediction mode, and the directional intra prediction mode may include the 2nd to 80th and -1st to -14th intra prediction modes, such as Figure 5 As shown by the arrow in the middle. The planar prediction mode can be represented by INTRA_PLANAR, and the DC prediction mode can be represented by INTRA_DC. In addition, the directional intra prediction mode can be represented by INTRA_ANGULAR-14 to INTRA_ANGULAR-1 and INTRA_ANGULAR2 to INTRA_ANGULAR80.
[0143] Meanwhile, the intra prediction type (or additional intra prediction mode) may include at least one of LIP, PDPC, MRL, ISP, or MIP. The intra prediction type may be indicated based on the intra prediction type information, and the intra prediction type information may be implemented in various forms. For example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. As another example, the intra prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block, and which reference sample line is used if applied, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the current block applies ISP, ISP type information (e.g., intra_subpartitions_split_flag) indicating the split type of the sub-partition when ISP is applied, flag information indicating whether PDPC is applied, flag information indicating whether LIP is applied, or MIP flag information indicating whether MIP is applied.
[0144] In the following, reference sample filtering will be described in more detail. The filtering is applied to the neighboring reference samples of the current block. This can be called pre-filtering because it is applied to the neighboring reference samples before intra prediction, unlike post-filtering where the filtering is applied to the predicted samples after intra prediction. For example, filtering of the neighboring reference samples can be performed as shown in the following mathematical formula.
[0145] The filtered sample value p[x][y] may be derived as follows: Here, x=-1, y=-1 to refH-1 and x=0 to refW-1, y=-1. refH may be determined as twice the height of the transform block or coding block, and refW may be determined as twice the width of the transform block or coding block.
[0146] [Mathematical formula 1]
[0147] p[-1][-1]=(refUnfilt[-1][0]+2*refUnfilt[-1][-1]+refUnfilt[0][-1]+2)>>2
[0148] [Mathematical formula 2]
[0149] p[-1][y]=(refUnfilt[-1][y+1]+2*refUnfilt[-1][y]+refUnfilt[-1][y-1]+2)>>2, where y=0 to refH-2.
[0150] [Mathematical formula 3]
[0151] p[-1][refH-1]=refUnfilt[-1][refH-1]
[0152] [Formula 4]
[0153] p[x][-1]=(refUnfilt[x-1][-1]+2*refUnfilt[x][-1]+refUnfilt[x+1][-1]+2)>>2, where x=0 to refW-2.
[0154] [Mathematical formula 5]
[0155] p[refW-1][-1]=refUnfilt[refW-1][-1]
[0156] Where refUnfilt represents the neighboring reference sample that has not been filtered, and [x][y] represents the x and y coordinates of the corresponding sample. For example, this can indicate the coordinates when the upper left sample position coordinates of the current block are (0,0).
[0157] When filtering is applied to the neighboring reference samples, the filtered neighboring reference samples may be used as reference samples in the prediction sample derivation step, and when filtering is not applied to the neighboring reference samples, the unfiltered neighboring reference samples may be used as reference samples in the prediction sample derivation step.
[0158] For example, when some or all of the following specific conditions are met, adjacent reference sample filtering may be used.
[0159] (Condition 1) nTbW*nTbH is greater than 32, where nTbW represents the width of the TB, that is, the width of the transform block (current block), and nTbH represents the height of the TB, that is, the height of the transform block (current block).
[0160] (Condition 2) The value of cIdx is 0. cIdx represents the color component of the current block, and a value of 0 represents the brightness component.
[0161] (Condition 3) IntraSubPartitionsSplitType indicates non-split (ISP_NO_SPLIT) Here, IntraSubPartitionsSplitType is a parameter indicating the split type of the current luma coding block.
[0162] (Condition 4) At least one of the following conditions 4-1 to 4-4 is true.
[0163] (Condition 4-1) The value of predModeIntra indicating the intra prediction mode indicates the planar prediction mode (INTRA_PLANAR).
[0164] (Condition 4-2) The value of predModeIntra indicates the 34th directional intra prediction mode (INTRA_ANGULAR34).
[0165] (Condition 4-3) The value of predModeIntra indicates the second directional intra prediction mode (INTRA_ANGULAR2), and the value of nTbH is greater than or equal to the value of nTbW.
[0166] (Condition 4-4) The value of predModeIntra is the 66th directional intra prediction mode (INTRA_ANGULAR66), and the value of nTbW is greater than or equal to nTbH.
[0167] For example, processes such as intra prediction mode / type determination, neighboring reference sample derivation, and prediction sample derivation can all be performed in units of CUs. As another example, intra prediction mode / type determination can be performed in units of CUs, while neighboring reference sample derivation and prediction sample derivation processes can be performed in units of TUs in the CU. In this case, the TUs in the CU can share the same intra prediction mode / type. Therefore, considering this situation, nTbW and nTbH, which are the width and height of the TU (or TB), can be considered to determine whether to perform neighboring reference sample filtering.
[0168] Overview of the intra prediction process with reference sample filtering
[0169] Hereinafter, an intra-frame prediction method to which sample filtering is applied according to an embodiment will be described. When a reference sample is filtered using a smoothing filter, the value of the reference sample has a characteristic of smooth variation. However, when intra-frame prediction is performed for a current sample by applying a filter having a sharpening characteristic to the reference sample, the sharpening characteristic is added to the smoothed reference sample, thereby reducing the efficiency of intra-frame prediction. Depending on whether a smoothing filter is used to filter the reference sample, it is necessary to select a filter to be applied to the reference sample from a smoothing filter and a sharpening filter to generate an intra-frame prediction sample for the current sample. Therefore, the efficiency of intra-frame prediction can be improved by applying a smoothing filter to a reference sample to which the smoothing filter is applied to generate a prediction sample and applying a sharpening filter to a reference sample to which the smoothing filter is not applied.
[0170] The intra prediction method performed by the decoding device according to the embodiment can determine the type of filter used when performing intra prediction according to whether the reference sample filtering is applied. Therefore, the decoding device can effectively generate a prediction sample by applying a filter suitable for the sample characteristics of the reference sample to the reference sample according to the reference sample filtering result.
[0171] Figure 6is a flowchart illustrating the intra prediction steps according to an embodiment. As described above, intra prediction may include three main steps. First, a decoding device according to an embodiment may construct a reference sample (S110). Then, the decoding device may use the constructed reference sample to predict the sample value of the current block (S120). The reference sample may be a sample located around the current block. The decoding device may apply a post filter to the intra prediction sample generated by performing step S120 after intra prediction of the sample (S130).
[0172] Figure 7 is a flowchart illustrating the reference sample construction step S110 according to an embodiment. Figure 7 The reference sample configuration step S110 is described in more detail. First, the decoding according to the embodiment can check the availability of the reference sample (S111). Next, the decoding device can determine whether to replace the reference sample according to the availability of the reference sample and perform the replacement of the reference sample (S112). For example, when the reference sample is not available, the reference sample can be replaced with a value generated based on the adjacent available samples. Therefore, all reference samples can be constructed.
[0173] Thereafter, the decoding device may filter the constructed reference sample according to the filtering condition of the reference sample (S113). When the size of the intra coding unit (CU) or the transform unit (TU) is large, the prediction results of the intra coding unit and the transform unit may be smoother. In this case, it may be necessary to perform reference sample filtering so that the generated intra prediction sample is smooth and planar.
[0174] In the following, reference will be made to Figure 8 Reference sample filtering conditions for intra prediction according to an embodiment are described. Figure 8 1 is a flowchart illustrating a method for determining whether to perform reference sample filtering by a decoding device according to an embodiment in the reference sample filtering step S113 of the reference sample construction step S110. First, the decoding device according to the embodiment may determine whether a condition for performing reference sample filtering is satisfied (S210). When the condition for performing reference sample filtering is not satisfied, the decoding device may not perform reference sample filtering (S220). At the same time, when the condition for performing reference sample filtering is satisfied, the decoding device may perform reference sample filtering (S230).
[0175] More specifically, the decoding device may determine whether to perform reference sample filtering according to the following conditions. In an embodiment, when all or at least one of the following conditions is true, the decoding device may determine to filter the reference samples referenced by the intra prediction of the current block, otherwise it may determine not to filter.
[0176] (Condition 1) The reference sample line closest to the current block is used for intra prediction, that is, the value of intra_luma_ref_idx is 0.
[0177] (Condition 2) The width-height product of the CU / TU is greater than 32.
[0178] (Condition 3) The current block is related to the luminance component.
[0179] (Condition 4) ISP (Intra Sub-Partitioning) coding mode is not used.
[0180] (Condition 5) At least one of the following conditions 5-1 or 5-2 is true. Hereinafter, the second directional intra prediction mode may be referred to as the lower left diagonal mode, the 34th directional intra prediction mode may be referred to as the upper left diagonal mode, and the 66th directional intra prediction mode may be referred to as the upper right diagonal mode. The 18th directional intra prediction mode may be referred to as the horizontal mode, and the 50th directional intra prediction mode may be referred to as the vertical mode.
[0181] (Condition 5-1) The width-height product of the CU / TU is equal to or less than 128, and the intra prediction mode corresponds to the intra plane mode (INTRA_PLANAR), the intra DC mode (INTRA_DC), the second directional intra prediction mode (INTRA_ANGULAR2) to the fourth directional intra prediction mode (INTRA_ANGULAR4), the 32nd directional intra prediction mode (INTRA_ANGULAR32) to the 36th directional intra prediction mode (INTRA_ANGULAR36), or the 64th directional intra prediction mode (INTRA_ANGULAR64) to the 66th directional intra prediction mode (INTRA_ANGULAR66).
[0182] (Condition 5-2) The width-height product of the CU / TU is greater than 128, and the intra prediction mode is neither the 18th directional intra prediction mode (INTRA_ANGULAR18) nor the 50th directional intra prediction mode (INTRA_ANGULAR50).
[0183] Meanwhile, the filter applied to the reference sample according to the above conditions may be a filter for smoothing the reference sample, and may be, for example, a [1 2 1] / 4 filter to be described later. As described above, it may be advantageous that the filter applied to the reference sample filtering and the filter used for intra-frame prediction may have the same characteristics. Therefore, when performing intra-frame prediction using a reference sample smoothed by applying a filter, applying a filter with a sharp sample trend may be inefficient. This is because, since the reference sample has a sharp trend (e.g., an edge), its sample value contrasts with the adjacent sample value, and since the reference sample has a smooth characteristic, its sample value is similar to the adjacent sample value. Therefore, when applying a reference sample filter, it is necessary to use a filter having characteristics that do not contradict the characteristics of the reference sample filter in the intra-frame prediction sample generation step. For example, the smoothing filter that can be used in the intra-frame prediction sample generation step may be any of a 4-tap Gaussian filter or a linear interpolation filter to be described later.
[0184] Meanwhile, when the reference sample filter is not applied, the reference sample may not be smoothed. In this case, the unsmoothed reference sample may have a sharp trend (e.g., edge). Similar to the above description, when intra prediction is performed using a reference sample with a sharp trend, applying a smoothing filter may be inefficient. Therefore, when the reference sample filter is not applied, the filter used in the intra prediction sample generation step may be a sharpening filter. For example, the sharpening filter may be a cubic filter, which will be described later.
[0185] In this regard, the decoding device according to the embodiment may determine the filter used to predict the sample value of the current block based on whether reference sample filtering is performed. Figure 6 An embodiment in which reference sample filtering is performed in the prediction sample generation step S120 to determine a filter for deriving the prediction sample. Fig. 9 1 is a flowchart illustrating a method for determining a filter for deriving a prediction sample according to whether reference sample filtering has been performed by a decoding device according to an embodiment. The decoding device according to an embodiment first determines whether reference sample filtering has been performed (310). Next, when reference sample filtering has not been performed, the decoding device derives a prediction sample using a first filter (S320). Here, the first filter may be a sharpening filter and may be a cubic filter.
[0186] Next, when the reference sample filtering has been performed, the decoding apparatus derives the prediction sample using the second filter (S330). Here, the second filter is a smoothing filter, which may be any one of a 4-tap Gaussian filter or a linear interpolation filter.
[0187] Overview of Reference Sample Filtering Execution Information Signaling
[0188] As reference Figures 6 to 9 As described, the filter applied in the prediction sample generation step S120 may vary depending on whether reference sample filtering is performed in the reference sample filtering step S113. Therefore, it is necessary to notify the prediction sample generation step S120 whether reference sample filtering is performed in the reference sample filtering step S113. Therefore, the decoding device selects the filter applied to the prediction sample according to whether reference sample filtering is performed in the reference sample filtering step S113, thereby improving the intra-frame prediction efficiency.
[0189] In an embodiment, the decoding apparatus may signal whether to perform reference sample filtering to the prediction sample generating step of the current block using filterFlag, which is a parameter indicating whether to perform reference sample filtering.
[0190] To this end, after the reference sample filtering step, the filterFlag indicating whether the reference sample has been filtered may be input to the intra prediction mode step for directional intra prediction modes (intra angle modes) 2 to 66, which will be described later. For example, in directional intra prediction modes 2 to 66, the decoding apparatus may use the received filterFlag to determine a filter for predicting a sample value of the current block.
[0191] In the following, reference will be made to Fig.10 Describe this in more detail. Fig.10 is a flowchart illustrating a method of determining, by a decoding apparatus, a filter for deriving a sample value of a current block using filterFlag as a parameter indicating whether to perform reference sample filtering according to an embodiment. Fig.10 The above reference Figure 8 and Fig. 9 For example, the reference sample filtering step S401 may be performed in reference Figure 8 The reference sample filtering step S113 described above is performed, and the prediction sample generation step S402 can be performed in reference Fig. 9 The described prediction sample generation step S120 is performed.
[0192] First, the decoding device according to the embodiment may determine whether the condition for performing reference sample filtering is met (S410). The condition for performing reference sample filtering may include the above conditions. When the condition for performing reference sample filtering is not met, the decoding device according to the embodiment may set the value of filterFlag to a first value, in which case the first value may be 0 (S420). At the same time, when the condition for performing reference sample filtering is met and reference sample filtering is performed, the decoding device according to the embodiment may set the value of filterFlag to a second value, in which case the second value may be 1 (S430).
[0193] For example, when filtering is not applied to the reference sample, the decoding device according to the embodiment may set the value of filterFlag to a first value. In addition, when filtering is applied to the reference sample, the decoding device according to the embodiment may set the value of filterFlag to a second value. For example, the first value may be 0, and the second value may be 1. In this embodiment, a filterFlag value of 0 may indicate that reference sample filtering is not performed, and a filterFlag value of 1 may indicate that reference sample filtering is performed.
[0194] In one embodiment, the filterFlag value generated in the reference sample filtering step S410 may be signaled to the prediction sample generation step S402. The decoding apparatus according to the embodiment may receive the filterFlag value generated in the reference sample filtering step S410 as a filterFlag parameter with the same parameter name in the prediction sample generation step S402.
[0195] In the prediction sample generation step of the current block, the decoding device according to the embodiment may determine the filter used to generate the prediction sample based on the filterFlag value received from the reference sample filtering step. For example, the decoding device according to the embodiment may determine whether the filterFlag value received from the reference sample filtering step corresponds to the second value (S440). When the filterFlag value does not correspond to the second value, the decoding device according to the embodiment may derive the prediction sample using the first filter (S450). The first filter may be a sharpening filter and may be a cubic filter. When the filterFlag value corresponds to the second value, the decoding device according to the embodiment may derive the prediction sample using the second filter (S460). Here, the second filter is a smoothing filter, which may be any one of a 4-tap Gaussian filter or a linear interpolation filter.
[0196] More specifically, when the filterFlag value received from the reference sample filtering step is a first value indicating that filtering is not applied to the reference sample, the decoding device according to the embodiment may determine that filtering is not applied to the reference sample and apply a sharpening filter to predict the sample value of the current block. Similarly, when the filterFlag value received from the reference sample filtering step is a second value indicating that filtering is applied to the intra-frame reference sample, the decoding device according to the embodiment may determine that filtering is applied to the intra-frame reference sample and apply a smoothing filter to predict the sample value of the current block.
[0197] In an embodiment, specific values of the first value and the second value of filterFlag may be set as required. Fig.11 It is shown in Fig.10 Flowchart of the operation of the decoding device when the first value is set to 0 and the second value is set to 1 in the embodiment of FIG. In addition to setting the first value and the second value, Fig.11 The steps corresponding to the reference numerals S510 to S560 may correspond to Fig.10 Meanwhile, in another embodiment, the first value indicating that reference sample filtering is not performed may be set to 1, and the second value indicating that reference sample filtering is performed may be set to 0.
[0198] In addition, when the intra prediction mode of the current block is a predetermined mode, filterFlag can be set to 1, otherwise it can be set to 0. The predetermined mode can be a plane mode or a directional mode, which indicates a reference sample at an integer position. In this case, even if the vertical mode and the horizontal mode indicate a reference sample at an integer position, the filterFlag value can be set to 0. That is, when the intra prediction mode of the current block is one of the predetermined modes described above, reference sample filtering can be performed. Otherwise, reference sample filtering may not be performed.
[0199] At the same time, it indicates whether filtering Fig.10 The parameter of the intra-frame reference sample may be called a name other than filterFlag. Fig.12 As shown in Fig.12 As shown in the example, filterFlag can be called refFilterFlag. In addition to the change in terminology, the corresponding Fig.12 The steps of reference numerals S610 to S660 may correspond to the steps corresponding to Fig.10 Steps marked S410 to S460 in the accompanying drawings.
[0200] First embodiment
[0201] In the following, reference will be made to Fig.13A reference sample filtering and intra prediction method according to an embodiment is described. It is necessary to provide a configuration for signaling whether filtering is applied in a reference sample filtering step, so that a smoothing filter is applied to a reference sample to which the smoothing filter is applied to generate a prediction sample, and a sharpening filter is applied to a reference sample to which the smoothing filter is not applied. Therefore, it can be determined in the prediction sample generation step whether filtering is applied in the reference sample filtering step, and a filter can be selected to generate a prediction sample, thereby improving the efficiency of intra prediction.
[0202] Fig.13 1 is a flowchart illustrating a method for generating intra-frame prediction samples according to a result of reference sample filtering performed by a decoding device according to an embodiment. First, a decoding device according to an embodiment may obtain the following parameters by input or derivation (S710). For example, the decoding device may receive the following parameters. Here, the integer value of x ranges from 0 to nTbW-1, and the integer value of y ranges from 0 to nTbH-1.
[0203] -(xTbCmp, yTbCmp) coordinate pair indicates the relative position of the top left sample of the current transform block and the top left sample of the current picture
[0204] -predModeIntra is a parameter indicating the intra prediction mode
[0205] -nTbW is a parameter indicating the width of the transform block
[0206] -nTbH is a parameter indicating the height of the transform block
[0207] -nCbW is a parameter indicating the width of the compilation block
[0208] -nCbH is a parameter indicating the height of the compilation block
[0209] -cIdx is a parameter indicating the color component of the current block
[0210] Meanwhile, the decoding apparatus according to an embodiment may derive parameters refW and refH as follows to generate intra prediction samples.
[0211] For example, when the parameter IntraSubPartitionsSplitType indicating the partition type of the current luma coding block indicates non-partitioning (ISP_NO_SPLIT) or cIdx as the color component of the current block is not 0 (ie, when the current block is not a luma block but a chroma block), refW and refH can be derived according to the following mathematical formula.
[0212] [Mathematical formula 6]
[0213] refW=nTbW*2
[0214] [Mathematical formula 7]
[0215] refH=nTbH*2
[0216] At the same time, unlike the above case, when IntraSubPartitionsSplitType indicates splitting and the value of cIdx indicates a luma block, for example, when IntraSubPartitionsSplitType is ISP_HOR_SPLIT indicating horizontal splitting, or ISP_VER_SPLIT indicating vertical splitting, the value of cIdx is 0 indicating a luma block, refW and refH can be derived as follows.
[0217] [Mathematical formula 8]
[0218] refW=nCbW*2
[0219] [Mathematical formula 9]
[0220] refH=nCbH*2
[0221] Furthermore, a parameter refIdx indicating an intra prediction reference line may be derived as follows based on IntraLumaRefLineIdx, which is an array for storing indices of reference lines referred to during intra prediction of a luma component relative to a position of a current transform block.
[0222] [Formula 10]
[0223] refIdx=(cIdx==0)? IntraLumaRefLineIdx[xTbCmp][yTbCmp]:0
[0224] Next, the decoding apparatus according to the embodiment generates a reference sample p[x][y] using the above parameters (S720). Figure 7 This is described in more detail. Here, the x, y coordinate pairs may consist of x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx.
[0225] First, the decoding device according to the embodiment may check the availability of the reference sample with respect to the sample position (xTbCmp, yTbCmp) (S111). To this end, the decoding device may obtain refIdx as a parameter indicating the intra-frame prediction reference line index, refW as a parameter indicating the reference sample width, refH as a parameter indicating the reference sample height, and cIdx as a parameter indicating the color component index as input. In addition, the decoding device may provide refUnfilt[x][y] as an output as a parameter indicating the reference sample. Here, the x, y coordinate pair may consist of x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx.
[0226] Next, when it is determined that at least one sample refUnfilt[x][y] of the x,y coordinate pair consisting of x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx is unavailable, the decoding device may perform a reference sample replacement step (S112).
[0227] The decoding device may obtain refIdx as a parameter indicating an intra-prediction reference line index, refW as a parameter indicating a width of a reference sample, refH as a parameter indicating a height of a reference sample, cIdx as a parameter indicating a color component index, and refUnfilt[x][y] as a parameter indicating a reference sample (wherein the x, y coordinates consist of x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx) as input to perform the reference sample replacement step. In addition, the decoding device may generate refUnfilt[x][y], which is a parameter indicating a modified reference sample (wherein the x, y coordinate pair consists of x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx) as a result of the reference sample replacement step.
[0228] Next, the decoding device may perform a reference sample filtering step (S113). The decoding device may obtain refIdx as an index indicating an intra-prediction reference line, nTbW as a parameter indicating a transform block width, nTbH as a parameter indicating a transform block height, refW as a parameter indicating a width of a reference sample, refH as a parameter indicating a height of a reference sample, refUnfilt[x][y] as a parameter indicating samples that are not filtered with respect to x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx, and cIdx as a parameter indicating a color component index as input. In addition, the decoding device may generate filterFlag, which is a parameter indicating whether reference sample filtering is performed as a result of the reference sample filtering step, and p[x][y] as a parameter indicating a reference sample. Here, x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx.
[0229] Will refer to Fig.13 Described again. In the next step, the decoding device according to the embodiment generates an intra-frame prediction sample (S730). The decoding device according to the embodiment can determine the intra-frame prediction mode to be applied to the current block according to the value of predModeIntra (S731). In addition, the decoding device can generate an intra-frame prediction sample by performing intra-frame prediction in the determined intra-frame prediction mode (S732).
[0230] For example, when the value of predModeIntra indicates an intra plane prediction mode (INTRA_PLANAR), the decoding device according to the embodiment may generate a prediction sample of the corresponding block in the intra plane prediction mode. In this case, the decoding device may obtain nTbW as a parameter indicating the width of the transform block, nTbH as a parameter indicating the height of the transform block, and p as a parameter indicating the reference sample array as input. As a result, the decoding device may generate predSamples as a parameter indicating the prediction sample array as a result.
[0231] Meanwhile, when predModeIntra is different from the above and predModeIntra indicates an intra DC prediction mode (INTRA_DC), the decoding device may generate a prediction sample of the corresponding block in the intra DC prediction mode. In this case, the decoding device may obtain nTbW as a parameter indicating the width of the transform block, nTbH as a parameter indicating the height of the transform block, and p as a parameter indicating the reference sample array as input. As a result, the decoding device may generate predSamples as a parameter indicating the prediction sample array as a result.
[0232] Meanwhile, when predModeIntra is different from the above and predModeIntra is a value indicating a cross-component linear model (CCLM) mode of a chroma sample, such as INTRA_LT_CCLM, INTRA_L_CCLM, or INTRA_T_CCLM described above, the decoding device may perform an intra-prediction mode processing process corresponding thereto. In this case, the decoding device may obtain predModeIntra as a parameter indicating an intra-prediction mode, a sample position (xTbC, yTbC) set equal to (xTbCmp, yTbCmp), nTbW as a parameter indicating the width of a transform block, nTbH as a parameter indicating the height of a transform block, and p as a parameter indicating a reference sample array as input. The decoding device may generate predSamples as a parameter indicating a prediction sample array as output.
[0233] Meanwhile, when predModeIntra is different from the above, the decoding device may perform a directional intra prediction mode. The decoding device may obtain predModeIntra as a parameter indicating the intra prediction mode, refIdx as a parameter indicating the intra prediction reference line index, nTbW as a parameter indicating the width of the transform block, nTbH as a parameter indicating the height of the transform block, refW as a parameter indicating the width of the reference sample, refH as a parameter indicating the height of the reference sample, nCbW indicating the width of the coding block, nCbH as a parameter indicating the height of the coding block, cIdx as a parameter indicating the color component index, filterFlag as a parameter indicating whether to perform reference sample filtering, and p as a parameter indicating the reference sample array as input. The decoding device may generate predModeIntra and predSamples, predModeIntra being a parameter indicating the modified intra prediction mode as a result of performing this, and predSamples being a parameter indicating the prediction sample array.
[0234] In the following, reference will be made to Fig.14A method in which a decoding apparatus performs a directional intra prediction mode according to an embodiment is described. Fig.14 2 is a flowchart illustrating a method for performing a directional intra prediction mode by a decoding device according to an embodiment. The decoding device according to the embodiment may generate a modified intra prediction mode predModeIntra as a result of performing the 2nd to 66th directional intra prediction modes (INTRA_ANGULAR2 to INTRA_ANGULAR66), and prediction samples predSamples[x][y]. Here, x=0 to nTbW-1, y=0 to nTbH-1.
[0235] First, the decoding apparatus according to the embodiment obtains parameters required to perform the directional intra prediction mode (S810). For example, the decoding apparatus according to the present embodiment may obtain the following parameters as input to perform the directional intra prediction mode.
[0236] -predModeIntra is a parameter indicating the intra prediction mode
[0237] -refIdx is a parameter indicating the intra prediction reference line index
[0238] -nTbW is a parameter indicating the transform block width
[0239] -nTbH is a parameter indicating the transform block height
[0240] -refW is a parameter indicating the reference sample width
[0241] -refH is a parameter indicating the reference sample height
[0242] -nCbW is a parameter indicating the width of the compilation block
[0243] -nCbH is a parameter indicating the compilation block height
[0244] -cIdx is a parameter indicating the color component of the current block
[0245] -filterFlag is a parameter indicating reference sample filtering
[0246] - Neighboring samples p[x][y] (here, the x,y pairs consist of x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx)
[0247] Here, the adjacent samples p[x][y] can be obtained by performing the above reference Fig.13 The reference sample array p[x][y] generated by the reference sample generation step S720 is described.
[0248] Meanwhile, nTbS, which is a parameter indicating a horizontal sample size of a transform block, may be determined as (Log2(nTbW)+Log2(nTbH))>>1.
[0249] In addition, the decoding device may derive a parameter nW indicating the width of the current block and a parameter nH indicating the height of the current block. In an embodiment, the decoding device may derive nW and nH based on the values of IntraSubPartitionsSplitType and cIdx. When IntraSubPartitionsSplitType is ISP_NO_SPLIT or cIdx is not 0, the decoding device may derive nW and nH as follows.
[0250] [Mathematical formula 11]
[0251] nW=nTbW
[0252] [Mathematical formula 12]
[0253] nH=nTbH
[0254] Meanwhile, when the values of IntraSubPartitionsSplitType and cIdx are different from those described above (eg, IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT and cIdx is equal to 0), the decoding apparatus may derive nW and nH as follows.
[0255] [Mathematical formula 13]
[0256] nW=nCbW
[0257] [Mathematical formula 14]
[0258] nH=nCbH
[0259] In addition, the decoding device may set whRatio, which is a parameter indicating the aspect ratio (ratio of width to height) of the current block, using nW and nH according to the calculation result of Abs(Log2(nW / nH)). In addition, the decoding device may determine wideAngle, which is a parameter indicating whether it is a wide angle, to 0.
[0260] Meanwhile, in the case of a non-square block (e.g., nW is not equal to nH), the decoding device may modify predModeIntra, which is a parameter indicating the intra prediction mode. For example, when all of the following conditions are true, the decoding device may set wideAngle to 1 and modify predModeIntra to a value of (predModeIntra+65).
[0261] (Condition 1) nW is greater than nH.
[0262] (Condition 2) predModeIntra is greater than or equal to 2.
[0263] (Condition 3) predModeIntra is less than the value calculated by (whRatio>1)?(8+2*whRatio):8.
[0264] However, if all of the above conditions are true, the decoding device may set wideAngle to 1 and modify predModeIntra to a value of (predModeIntra-67) when all of the following conditions are true.
[0265] (Condition 1) nH is greater than nW.
[0266] (Condition 2) The value of predModeIntra is 66 or less.
[0267] (Condition 3) Is the value of predModeIntra greater than (whRatio>1)? (60-2*whRatio): 60 The calculated value.
[0268] Furthermore, the decoding device may derive a modified filterFlag. Fig.15 is a flow chart illustrating a method for deriving a modified filterFlag value by a decoding device. In an embodiment, the decoding device may determine whether at least one of the following conditions is true (S910). When at least one of the following conditions is true, the decoding device may set the value of the modified filterFlag to a first value (S930). In an embodiment, the first value may be 0.
[0269] (Condition 1) IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, cIdx is 0, predModeIntra is greater than or equal to INTRA_ANGULAR34, and nW is greater than 8.
[0270] (Condition 2) IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, cIdx is 0, predModeIntra is less than INTRA_ANGULAR34, and nH is greater than 8.
[0271] At the same time, when the above conditions are not met, the decoding device may set the value of the modified filterFlag to a second value (S920). In an embodiment, the second value may be calculated as 1-filterFlag. For example, when the value of filterFlag is 0, the value of the modified filterFlag may be determined to be 1, and when the value of filterFlag is 1, the value of the modified filterFlag may be determined to be 0.
[0272] Fig.16 is a mapping table between parameters predModeIntra and intraPredAngle indicating angles. The decoding device may be based on Fig.16 The invAngle as a parameter indicating the inverse angle is derived from the intraPredAngle of the mapping table as follows.
[0273] [Mathematical formula 15]
[0274] invAngle=Round((256*32) / "intraPredAngle")
[0275] Here, the Round function refers to a rounding operation for integers.
[0276] Fig.17 is a table illustrating coefficients of a cubic filter (fC) and a Gaussian filter (fG) as interpolation filters for intra prediction. When the phase is from 0 to 31 and j has a value of 0 to 3, Fig.17 , an embodiment of fC[phase][j] and fG[phase][j] as coefficients of a cubic filter (fC) and a Gaussian filter (fG) is shown.
[0277] Next, the decoding apparatus may derive a reference sample array ref[x] based on the adjacent sample array p[x] (S820), and derive a prediction sample array predSamples[x][y] (S830). Figure 5 and Figures 16 to 20 Give a description. Figures 18 to 20 is a flowchart illustrating a method of deriving a reference sample array ref[x] by a decoding apparatus according to an embodiment.
[0278] First, if Fig.18As shown, according to the embodiment, the decoding device can determine the reference sample referenced by the prediction sample array predSample[x][y] for deriving the current block according to the value of predModeIntra (S1010). When the value of predModeIntra is greater than or equal to 34, the decoding device can mainly use the adjacent samples above the current block in the adjacent samples p[x][y] to derive predSample[x][y] (S1020). When the value of predModeIntra is less than 34, the decoding device can mainly use the samples located on the left side of the current block in the adjacent samples p[x][y] to derive predSample[x][y] (S1030).
[0279] Will refer to Fig.19 Step S1020 of constructing predSample[x][y] by the decoding device mainly using adjacent samples located above the current block is described in more detail. The decoding device can derive a reference sample array ref[x]. ref[x] can be constructed by adding additional samples after generating a main reference sample array. First, ref[x] can be composed of a main reference sample array, which is a set of samples that are referenced as main values to derive sample values of the current block. For example, when the Gaussian or cubic filter described below is applied to the reference sample array to derive the value of each sample of the current block, the reference sample to which the highest filter coefficient is applied can be the reference sample referenced as the main value. In this example, the main reference sample array can be composed of a reference sample set, which is composed of samples to which the highest filter coefficient is applied. More specifically, in Fig.17 In the example of the filter coefficient table of , when the phase p is 0, the fG interpolation filter coefficient has the highest value in fG[0][1], and the reference sample corresponding thereto can be determined as the primary reference sample. In order to construct such a primary reference sample array, the primary reference sample array can be constructed by the values of the neighboring samples specified in the intra prediction direction of the current block.
[0280] The decoding apparatus may derive a primary reference sample array ref[x] using neighboring samples located above the current block, so as to derive a reference sample array ref[x] ( S1110 ). The following mathematical formula may be used to perform this step.
[0281] [Formula 16]
[0282] ref[x]=p[-1-refIdx+x][-1-refIdx], where the value of x can be from 0 to nTbW+refIdx.
[0283] When predModeIntra is 34, nTbW is 8, and refIdx is 0, the correspondence between ref[0] to ref[8] and the adjacent blocks generated according to formula 16 is as follows: Fig.21 shown.
[0284] Next, the decoding device is based on Fig.16 The derived value of intraPredAngle determines the neighboring block referenced by the extended primary reference sample array (S1120). When the value of intrapredAngle is less than 0, the decoding device may use the neighboring samples located on the left side of the current block to extend the primary reference sample array (S1130). For example, when the calculation result of (nTbH*intraPredAngle)>>5 is less than -1, the decoding device may use the following mathematical formula 17 to extend the primary reference sample array from the value of the left neighboring block.
[0285] [Mathematical formula 17]
[0286] ref[x]=p[-1-refIdx][-1-refIdx+((x*invAngle+128)>>8)], where the value of x may be from -1 to (nTbH*intraPredAngle)>>5.
[0287] When predModeIntra is 34, nTbW is 8, nTbH is 8, and refIdx is 0, the correspondence between ref[-1] to ref[-8] generated according to formula 17 and the adjacent blocks is as follows: Fig. 22 shown.
[0288] Next, the decoding device may add additional samples to both ends of the main reference sample array (S1140). As described below, in the case of applying a 4-tap filter to derive sample values of the current block, when reference samples at both ends of the main reference sample can be selected as the main reference sample and the filter is applied to the reference sample, the value of the reference sample input to the filter may not exist.
[0289] For example, when having 0 as Fig.17 The Gaussian filter (fG) of the phase (p) value in is used Fig.21 When ref[8] is used as the main reference sample, the reference sample values corresponding to fG[0][2] and fG[0][3] do not exist in ref[]. In this example, since the coefficient value of fG[0][3] is 0, the value of the reference value corresponding to the corresponding position can be left unspecified. However, since the coefficient of fG[0][2] exists, the sample value of fG[0][2] needs to be stored in ref[].
[0290] To eliminate this problem, additional samples can be added to the end of the main reference sample array. Based on the index of the main reference sample array, the additional sample can be set to the value of the reference sample with an index closest to the index of the additional sample. To this end, the decoding device can copy the value of the reference sample with an index closest to the additional sample from ref[], or obtain it from the value of the neighboring block corresponding to it and store it in ref[].
[0291] Maths 18 and 19 below represent calculations of a decoding device for adding additional samples to both ends of a primary reference sample array using a pre-calculated primary reference sample array. The decoding device may add additional samples to a reference sample array using the following maths.
[0292] [Mathematical formula 18]
[0293] ref[((nTbH*intraPredAngle)>>5)-1]=ref[(nTbH*intraPredAngle)>>5]
[0294] [Mathematical formula 19]
[0295] ref[nTbW+1+refIdx]=ref[nTbW+refIdx]
[0296] When predModeIntra is 34, nTbW is 8, and refIdx is 0, the correspondence between ref[-9] and ref[9] generated according to formula 18 and the adjacent blocks is as follows: Fig.23 shown.
[0297] Meanwhile, when intraPredAngle is not less than 0, the decoding apparatus may extend the primary reference sample array using the neighboring sample located at the upper right corner of the current block (S1150). The decoding apparatus may extend the primary reference sample array from the value of the upper right neighboring block using the following Math 20.
[0298] [Mathematical formula 20]
[0299] ref[x]=p[-1-refIdx+x][-1-refIdx], where x=nTbW+1+refIdx to refW+refIdx.
[0300] When predModeIntra is 66, nTbW is 8, nTbH is 8, refW is 16, and refIdx is 0, the correspondence between ref[9] to ref
[16] and the adjacent blocks generated according to equation 20 is as follows: Fig.24 shown.
[0301] Next, the decoding apparatus may add the additional sample to the left end of the main reference sample array (S1160). In an embodiment, the decoding apparatus may determine the sample value of ref[-1] using the sample value of ref[0] using the following Math 21.
[0302] [Mathematical formula 21]
[0303] ref[-1] = ref[0]
[0304] Next, the decoding device adds the additional sample to the right end of the main reference sample array (S1170). The decoding device may add the additional sample to the right end of the main reference sample array according to the aspect ratio whRatio of the current block. In an embodiment, the decoding device may derive ref[refW+refIdx+x] as a parameter indicating the additional sample using the following mathematical formula 22. Here, x is 1 to (Max(1,nTbW / nTbH)*refIdx+1).
[0305] [Mathematical formula 22]
[0306] ref[refW+refIdx+x]=p[-1+refW][-1-refIdx]
[0307] In the embodiment where predModeIntra is 66, nTbW is 8, nTbH is 8, refW is 16, and refIdx is 0, the correspondence between the additional samples ref[-1] and ref
[17] generated according to equations 21 and 22 and the adjacent blocks is as follows: Fig.24 shown.
[0308] Next, the decoding device can use the generated reference sample array to derive a prediction sample array (S1180). Fig. 20 Describe this step. For example, the decoding device may derive prediction samples predSamples[x][y] having values of x=0 to nTbW-1 and y=0 to nTbH-1 as follows. First, the decoding device may derive iIdx, which is an index parameter and a multiplication factor iFact for deriving the prediction sample as follows (S1210).
[0309] [Mathematical formula 23]
[0310] iIdx=((y+1+refIdx)*intraPredAngle)>>5+refIdx
[0311] [Mathematical formula 24]
[0312] iFact=((y+1+refIdx)*intraPredAngle)&31
[0313] Next, the decoding device determines whether the current block is a luma block according to the value of cIdx (S1220). In this case, when cIdx is 0, the decoding device selects an interpolation filter for deriving luma prediction samples according to the value of filterFlag (S1230). The decoding device can derive the interpolation filter coefficient fT[j] as follows, where j=0 to 3.
[0314] [Mathematical formula 25]
[0315] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j]
[0316] More specifically, when the value of filterFlag is equal to 1, the decoding device may select a Gaussian filter (fG) coefficient for the phase (p) value selected as the iFact value. In this case, the decoding device may derive a luminance prediction sample of the current block by applying a Gaussian filter to the reference sample using the selected filter coefficient value (S1240).
[0317] Meanwhile, when the value of filterFlag is not equal to 1, the decoding apparatus may select a cubic filter (fC) coefficient for the phase (p) value selected as the iFact value. In this case, the decoding apparatus may derive a luminance prediction sample of the current block by applying a cubic filter to the reference sample using the selected filter coefficient value (S1250).
[0318] In an embodiment, the value of the prediction samples predSamples[x][y] may be derived using the selected filter coefficients fT[i] as follows.
[0319] [Mathematical formula 26]
[0320]
[0321] In an embodiment, predModeIntra may be 66, nTbW may be 8, nTbH may be 8, refW may be 16, refIdx may be 0, and cIdx may be set to 0. In this embodiment, according to the above mathematical formulas 23 and 24, the value of iIdx of predSamples[0][0] may be set to 1, the value of iFact may be set to 0, the value of iIdx of predSamples[7][7] may be set to 8, and the value of iFact may be set to 0. In addition, when the value of filterFlag is 1, fT[j] may be set to a filter coefficient according to a Gaussian filter, and may be set to fT[] = (16, 32, 16, 0).
[0322] Fig.25 The results of deriving predSamples[0][0] and predSamples[7][7] in this embodiment are shown. predSamples[0][0] can be derived based on the calculation result of ((ref[1]*16+ref[2]*32+ref[3]*16+ref[4]*0)+32)>>6. In this example, the value of predSamples[0][0] can be calculated as 2.
[0323] predSamples[7][7] can be derived from the calculation result of ((ref
[15] *16+ref
[16] *32+ref
[17] *16+ref
[18] *0)+32)>>6. In this example, the value of predSamples[7][7] can be calculated as 1. In the example of calculating predSamples[7][7], the value of ref
[18] is not defined, but the value of the Gaussian filter coefficient fT[3] multiplied by ref
[18] is 0. Therefore, the above calculation is possible.
[0324] Meanwhile, in the present embodiment, when the value of filterFlag is 0, fT[j] may be set to a filter coefficient according to a cubic filter and may be set to fT[]=(0, 64, 0, 0).
[0325] Fig.26 The results of deriving predSamples[0][0] and predSamples[7][7] in this embodiment are shown. predSamples[0][0] can be derived based on the result of calculating ((ref[1]*0+ref[2]*64+ref[3]*0+ref[4]*0)+32)>>6. In this example, the value of predSamples[0][0] can be calculated as 2.
[0326] predSamples[7][7] can be derived from the result of calculating ((ref
[15] *0+ref
[16] *64+ref
[17] *0+ref
[18] *0)+32)>>6. In this example, the value of predSamples[7][7] can be calculated as 0. In the example of calculating predSamples[7][7], the value of ref
[18] is not defined, but the value of the cubic filter coefficient fT[3] multiplied by ref
[18] is 0. Therefore, the above calculation is possible.
[0327] Meanwhile, when cIdx is not 0, the value of the chroma prediction sample may be determined according to the value of iFact (S1260). In an embodiment, when iFact is not 0, the value of the chroma prediction sample predSamples[x][y] may be derived as follows.
[0328] [Mathematical formula 27]
[0329] predSamples[x][y]=
[0330] ((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5
[0331] At the same time, when iFact is 0, the value of the chrominance prediction sample predSamples[x][y] can be derived as follows.
[0332] [Mathematical formula 28]
[0333] predSamples[x][y]=ref[x+iIdx+1]
[0334] Reference again Fig.18 , when predModeIntra is less than 34, the decoding device according to the embodiment may mainly use the adjacent samples located on the left side of the current block to derive the prediction sample array predSample[x][y] (S1030). This step corresponds to the above-mentioned step S1030, so the difference from step S1020 will be focused on. First, the decoding device may set the main reference sample array ref[x] as follows.
[0335] [Mathematical formula 29]
[0336] ref[x]=p[-1-refIdx][-1-refIdx+x], where the value of x is 0 to nTbH+refIdx.
[0337] In this case, when intraPredAngle is less than 0, the main reference sample array may be extended. For example, when the calculation result of (nTbW*intraPredAngle)>>5 is less than -1, ref[x] may be extended as follows.
[0338] [Math 30]
[0339] ref[x]=p[-1-refIdx+((x*invAngle+128)>>8)][-1-refIdx],
[0340] Here, the value of x is -1 to (nTbW*intraPredAngle)>>5.
[0341] [Mathematical formula 31]
[0342] ref[((nTbW*intraPredAngle)>>5)-1]=ref[(nTbW*intraPredAngle)>>5]
[0343] [Math 32]
[0344] ref[nTbG+1+refIdx]=ref[nTbH+refIdx]
[0345] Meanwhile, when intraPredAngle is not less than 0, ref[x] may be determined as follows.
[0346] [Mathematical formula 33]
[0347] ref[x]=p[-1-refIdx][-1-refIdx+x], where the value of x is nTbH+1+refIdx to refH+refIdx.
[0348] [Math 34]
[0349] ref[-1] = ref[0]
[0350] ref[refH+refIdx+x], which is a parameter indicating an additional sample, can be derived as follows: Here, the value of x is 1 to (Max(1,nTbW / nTbH)*refIdx+1).
[0351] [Mathematical formula 35]
[0352] ref[refH+refIdx+x]=p[-1+refH][-1-refIdx]
[0353] Next, when x has a value of 0 to nTbW-1 and y has a value of 0 to nTbH-1, the decoding device may derive the prediction sample predSamples[x][y] as follows. First, iIdx and the multiplication factor iFact as index parameters for deriving the prediction sample may be derived as follows.
[0354] [Mathematical formula 36]
[0355] iIdx=((x+1+refIdx)*intraPredAngle)>>5
[0356] [Mathematical formula 37]
[0357] iFact=((x+1+refIdx)*intraPredAngle)&31
[0358] In addition, when cIdx is 0, the interpolation filter coefficient fT[j], where j=0 to 3, can be derived as follows.
[0359] [Math 38]
[0360] fT[j]=filterFlag? fG[iFact][j]:fC[iFact][j]
[0361] In addition, the value of the predicted sample predSamples[x][y] can be derived as follows.
[0362] [Math 39]
[0363]
[0364] Meanwhile, when cIdx is not 0, the value of the predicted sample may be determined according to the value of iFact. In an embodiment, when iFact is not 0, the value of the predicted sample predSamples[x][y] may be derived as follows.
[0365] [Math 40]
[0366] predSamples[x][y]=
[0367] ((32-iFact)ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5
[0368] At the same time, when iFact is 0, the value of the predicted sample predSamples[x][y] can be derived as follows.
[0369] [Mathematical formula 41]
[0370] predSamples[x][y]=ref[y+iIdx+1]
[0371] Meanwhile, when all of the following conditions are true, the decoding device may perform a position-dependent prediction sample filtering step.
[0372] (Condition 1) IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT or cIdx is not 0.
[0373] (Condition 2) The value of refIdx is 0 or the value of cIdx is not 0.
[0374] (Condition 3) predModeIntra indicates a value of any one of INTRA_PLANAR, INTRA_DC, INTRA_ANGULAR18, or INTRA_ANGULAR50, or predModeIntra indicates a value equal to or less than INTRA_ANGULAR10 or a value equal to or greater than INTRA_ANGULAR58.
[0375] To this end, the decoding device can obtain predModeIntra as a parameter indicating the intra prediction mode, nTbW as a parameter indicating the width of the transform block, nTbH as a parameter indicating the height of the transform block, predSamples[x][y] as a parameter indicating the sample predicted for x=0 to nTbW-1 and y=0 to nTbH-1, refW as a parameter indicating the width of the reference sample, refH as a parameter indicating the height of the reference sample, reference sample p[x][y] (where x=-1 and y=-1 to refH-1 and x=0 to refW-1 and y=-1), and cIdx as a parameter indicating the color component index, as input. In addition, the decoding device can generate a prediction sample array predSamples modified as a result of performing the processing as an output.
[0376] Another embodiment of reference sample filtering condition
[0377] At the same time, for the above reference Figure 8 According to the reference sample filtering step described above, the decoding device of the embodiment may use the following modified conditions instead of the above conditions to perform step S210 to determine the filterFlag value. By setting the filtering conditions of the reference samples as follows, reference sample filtering may be performed for an intra-frame prediction mode different from the above reference sample filtering conditions. For example, when the intra-frame prediction mode of the current block is INTRA_DC mode, reference sample filtering may be omitted.
[0378] This will refer to Fig. 27 Described in more detail. For example, the decoding device according to the embodiment may determine whether the following modification conditions for performing reference sample filtering are met (S1310). When any of the conditions are not met, the decoding device according to the embodiment may set the filterFlag value to 0 (S1320). When all of the following conditions are met, the decoding device according to the embodiment may set the filterFlag value to 1 (S1330). At the same time, in order to increase the situation where reference sample filtering occurs, when any of the following conditions are met, the decoding device according to the present embodiment may set the filterFlag value to 1. In this case, when all of the following conditions are not met, the decoding device may set the filterFlag value to 0.
[0379] (Condition 1) The value of refIdx is 0.
[0380] (Condition 2) cIdx is equal to 0.
[0381] (Condition 3) IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT.
[0382] (Condition 4) predModeIntra is not equal to INTRA_DC.
[0383] (Condition 5) In addition, one or more of the following conditions 5-1 to 5-4 should also be satisfied.
[0384] (Condition 5-1) predModeIntra is equal to INTRA_PLANAR and nTbW*nTbH is greater than 32.
[0385] (Condition 5-2) nTbW is greater than nTbH, predModeIntra is equal to or greater than 2, and predModeIntra is less than (whRatio>1)? (8+2*whRatio): The calculation result of 8.
[0386] (Condition 5-3) nTbH is greater than nTbW, predModeIntra is less than or equal to 66, and predModeIntra is greater than (whRatio>1)? The calculation result of (60-2*whRatio): 60.
[0387] (Condition 5-4) minDistVerHor is greater than intraHorVerDistThres[nTbS]. In this case, the value of the parameter nTbS is (Log2(nTbW)+Log2(nTbH))>>1, and the parameter minDistVerHor is set to the calculation result of Min(Abs(predModeIntra-50) and Abs(predModeIntra-18)). In this case, the parameter intraHorVerDistThres[nTbS] can be determined with respect to the transform block size nTbS, as shown in the following table.
[0388] [Table 2]
[0389]
[0390]
[121] / 4 filter for reference sample filtering
[0391] The smoothing filter applied to the above reference sample filtering step will be described in more detail. In an embodiment, the decoding device may use a [1 2 1] / 4 filter to perform reference sample filtering. Fig.28 The smoothing filter used for reference sample filtering is described in more detail.
[0392] The decoding apparatus may perform reference sample filtering using the reference sample and values of samples adjacent to the reference sample. Fig.28 A target sample array 301 to which a [1 2 1] / 4 filter is to be applied and a result sample array 302 to which a [1 2 1] / 4 filter is to be applied are shown. In an embodiment, the target sample array 301 may be updated to the values of the result sample array 302.
[0393] like Fig.28 As shown, when the reference sample is located above the current block, the decoding device can apply a [1 21] / 4 filter to the reference sample by updating the value of the reference sample to (left sample value of the reference sample*1+sample value of the reference sample*2+right sample value of the reference sample*1) / 4.
[0394] Meanwhile, when the reference sample is located on the left side of the current block, the decoding device may apply a [1 2 1] / 4 filter to the reference sample in a similar manner. For example, the decoding device may update the value of the reference sample to a value of (left sample value of the reference sample * 1 + sample value of the reference sample * 2 + right sample value of the reference sample * 1) / 4.
[0395] like Fig.28 As shown, there is a difference between the sample array 301 before applying the [1 2 1] / 4 filter and the sample array 302 after applying the [1 2 1] / 4 filter in that the sample values change according to the increase of the array index. The sample array 302 to which the [1 2 1] / 4 filter is applied has a tendency to smoothly change the sample values compared to the sample array to which the [1 2 1] / 4 filter is not applied.
[0396] Second embodiment
[0397] Hereinafter, the reference sample filtering and intra prediction method according to another embodiment will be described accordingly. Only when the prediction mode is a directional intra prediction mode with an integer multiple slope, the decoding device according to the embodiment can perform filtering by applying a smoothing filter to the reference sample. In this case, in a non-integer multiple directional intra prediction mode, the reference sample is not filtered. In this case, the decoding device can generate a prediction sample by selectively applying any one of a sharpening filter and a smoothing filter to the reference sample. By performing in this way, the decoding device can improve the prediction performance while reducing the filtering complexity in the directional intra prediction process.
[0398] In the following, reference will be made to Fig.29 Description will be made. First, an embodiment of filtering reference samples will be described. When the following conditions are met, the decoding device may filter the reference samples. For example, when all of the following conditions are met, the decoding device according to the embodiment may set the filterFlag value to 1. When both conditions are not met, the decoding device according to the embodiment may set the filterFlag value to 0. At the same time, in order to increase the situation where reference sample filtering occurs, when any of the following conditions is met, the decoding device according to the embodiment may set the filterFlag value to 1. In this case, when all of the following conditions are not met, the decoding device may set the filterFlag value to 0.
[0399] (Condition 1) The current coding unit is the luma component.
[0400] (Condition 2) The value of the MRL index is 0.
[0401] (Condition 3) In ISP, no compilation unit splitting occurs.
[0402] (Condition 4) Not in MIP mode.
[0403] (Condition 5) H*W>32.
[0404] (Condition 6) The prediction mode is planar mode or integer slope mode. (ie, abs(predIntraAngle) is equal to 32 or a multiple thereof).
[0405] According to the above conditions, the decoding device can determine whether the value of filterFlag is 1 (S1410), and filter the reference sample only when the prediction mode is a directional intra-frame prediction mode with an integer multiple slope (S1420). In addition, the decoding device can generate a prediction sample by copying the filtered reference sample (S1430).
[0406] Meanwhile, when the value of filterFlag is not 1, in the case of a directional intra prediction mode with a non-integer multiple slope, the decoding device may not filter the reference sample. For a directional intra prediction mode with a non-integer multiple slope, filtering of the reference sample is not performed, but an interpolation filter may be applied in the process of generating the prediction sample. To this end, the above-mentioned cubic filter with sharpening characteristics or a Gaussian filter with a smoothing filter may be applied. The selection of the filter for interpolating the unfiltered reference sample may be determined according to the size of the current block and the intra prediction mode.
[0407] To this end, when the following conditions are satisfied, the decoding apparatus may set the value of a parameter interpolationFlag for selecting a filter for interpolating reference samples to 1.
[0408] (Condition 1) The current coding unit is the luma component.
[0409] (Condition 2) The value of the MRL index is 0.
[0410] (Condition 3) In ISP, unit coding splitting does not occur.
[0411] (Condition 4) Not in MIP mode.
[0412] (Condition 5) H*W>32.
[0413] (Condition 6) The value of predModeIntra is 0 or a value greater than 1, and minDistVerHor is greater than intraHorVerDistThres[nTbS]. Here, the parameter minDistVerHor has a value of Min(Abs(predModeIntra-50) and Abs(predModeIntra-18)). The parameter intraHorVerDistThres[nTbS] can be determined according to the following table showing the correspondence between the intraHorVerDistThres[nTbS] value and the transform block size nTbS.
[0414] [Table 3]
[0415]
[0416] The decoding device may select a filter according to the determined value of interpolationFlag (S1440). When the value of interpolationFlag is 1, the decoding device may use a smoothing filter (eg, reference Fig.25 The Gaussian filter described above) is used to generate prediction samples (S1450). When the value of interpolationFlag is 0, the decoding device may use a sharpening filter (e.g., reference Fig.26 The cubic filter described above) is used to generate prediction samples (S1460).
[0417] Third embodiment
[0418] Hereinafter, the reference sample filtering and intra prediction method according to another embodiment will be described accordingly. Unlike the second embodiment, by applying reference sample filtering even in the case of a directional intra prediction mode with a non-integer slope, the intra prediction efficiency can be improved. To this end, the reference sample filtering condition can be changed so that the reference sample is filtered even in the case of a directional intra prediction mode with a non-integer slope. In addition, when a smoothing filter is used for reference sample filtering, the reference sample is interpolated using the smoothing filter to generate a prediction sample, and when the reference sample filtering is not performed, the reference sample is interpolated using a sharpening filter to generate a prediction sample. Therefore, the intra prediction efficiency can be improved.
[0419] In the following, reference will be made to Fig.30 Describe. First, another case of filtering reference samples will be described. In an embodiment, when the current coding unit is a luminance component, the value of the MRL index is 0, no segmentation of the coding unit occurs in the ISP, it is not a MIP mode, H*W>32, and the prediction mode is a planar mode or has a preset slope according to the size of the CU. The decoding device can set the filterFlag value to 1, and reference sample filtering can be performed. Therefore, regardless of whether the slope of the directional prediction mode is an integer multiple, the decoding device can perform reference sample filtering.
[0420] For the above process, the decoding device according to the present embodiment may determine the following modification conditions in the reference sample filtering step S1510 to determine the filterFlag value. For example, when all of the following conditions are met, the decoding device according to the embodiment may set the filterFlag value to 1. When both conditions are not met, the decoding device according to the embodiment sets the filterFlag value to 0. At the same time, in order to increase the occurrence of reference sample filtering, when any of the following conditions is met, the decoding device according to the embodiment may set the filterFlag value to 1. In this case, when all of the following conditions are not met, the decoding device may set the filterFlag value to 0.
[0421] (Condition 1) The value of refIdx is 0.
[0422] (Condition 2) nTbW*nTbH is greater than 32.
[0423] (Condition 3) cIdx is equal to 0.
[0424] (Condition 4) IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT.
[0425] (Condition 5) The value of predModeIntra is 0 or a value greater than 1, and minDistVerHor is greater than intraHorVerDistThres[nTbS]. Here, the parameter minDistVerHor has a value of Min(Abs(predModeIntra-50) and Abs(predModeIntra-18)). The parameter intraHorVerDistThres[nTbS] can be determined according to the following table showing the correspondence between the intraHorVerDistThres[nTbS] value and the transform block size nTbS.
[0426] [Table 4]
[0427]
[0428] The decoding device may determine whether to perform reference sample filtering according to the value of filterFlag (S1511). When the value of filterFlag is 1, the reference sample may be filtered using a [1 2 1] / 4 filter (S1512). Meanwhile, when the value of filterFlag is not 1, the decoding device may omit the reference sample filtering. In addition, the decoding device may output the value of filterFlag to the prediction sample generation step S1520 (S1513).
[0429] Next, in order to perform the prediction sample generation step S1520, the decoding device may receive the filterFlag value generated in the reference sample filtering step S1510 and set the filterFlag value in the prediction sample generation step S1520 (S1521). The decoding device may determine whether the filterFlag value is equal to 1 (S1522). When the prediction sample is generated using the reference sample for reference sample filtering, the filterFlag value may be equal to 1. When the filterFlag value is 1, the decoding device determines whether the slope of the intra-frame prediction mode of the current block is an integer slope (S1523). The decoding device may use Fig.16The table obtains the value of predModeIntra indicating the intra-frame prediction mode of the current block and the value of intraPredAngle indicating the current slope, as described above. When the obtained intraPredAngle value is a multiple of 32, the decoding device can determine that the slope of the intra-frame prediction mode of the current block is an integer multiple. When the slope of the intra-frame prediction mode of the current block is an integer slope, the decoding device can generate a prediction sample by simply copying the value of the reference sample to the prediction sample (S1524). When the slope of the intra-frame prediction mode of the current block is not an integer slope (non-integer slope), for example, when the obtained intraPredAngle value is not a multiple of 32, the decoding device can generate a prediction sample by applying linear interpolation samples to the value of the reference sample, such as Fig.32 As shown in the example (S1524).
[0430] Meanwhile, when the value of filterFlag is not 1, it indicates that the reference sample is not filtered. In this case, when the value is not 1, the decoding device may determine whether to apply an interpolation filter (S1526) according to whether the intra prediction mode has an integer slope. When the intra prediction mode of the current block is a directional prediction mode with a non-integer slope, the decoding device may apply a sharpening filter such as a 4-tap cubic filter to the reference sample to generate a prediction sample (S1528), and when the intra prediction mode of the current block is a directional prediction mode with an integer slope, copy the reference sample without applying filtering of the prediction sample to generate a prediction sample (S1527).
[0431] Fourth embodiment
[0432] Hereinafter, the reference sample filtering and intra prediction method according to another embodiment will be described accordingly. In the third embodiment, a method for generating a prediction sample by determining whether to perform reference sample filtering and the slope of the intra prediction mode of the current block is determined. At the same time, the decoding device according to the embodiment can apply filtering of the prediction sample based on whether the current block is a luminance block, whether to perform reference sample filtering, and the intra prediction mode of the current block. Therefore, compared with the third embodiment, the decoding device can generate intra prediction samples more simply.
[0433] In the following, reference will be made to Fig.31 Describe it. Fig.30 The step of filtering the reference sample S1510 is performed in the same manner as in the described embodiment. The decoding apparatus may receive the filterFlag value in the reference sample filtering step S1510 and set it as the filterFlag value in the intra prediction sample generating step S1530 (S1531).
[0434] When the intra prediction mode is equal to or greater than the 34th directional prediction mode (INTRA_ANGULAR34), the decoding apparatus according to this embodiment may derive the reference sample array ref[x] as described above and derive the iIdx and iFact values according to the following mathematical formulas.
[0435] [Mathematical formula 42]
[0436] iIdx=(((y+1+refIdx)*intraPredAngle)>>5)+refIdx
[0437] [Mathematical formula 43]
[0438] iFact=((y+1+refIdx)*intraPredAngle)&31
[0439] Next, the decoding device may determine the filter applied to the reference sample to derive the prediction sample according to whether cIdx is 0 and filterFlag is 0 (S1532). When cIdx is 0 and filterFlag is 0, the decoding device may generate the prediction sample by applying a sharpening filter as follows (S1533). For example, the decoding device may use a cubic filter as one of the sharpening filters. The decoding device may derive the interpolation filter coefficient fT[j] as shown in the following mathematical formula. Here, j=0 to 3.
[0440] [Mathematical formula 44]
[0441] fT[j]=fC[iFact][j]
[0442] In addition, the decoding device can derive the value of the prediction sample predSamples[x][y] as shown in the following mathematical formula.
[0443] [Mathematical formula 45]
[0444]
[0445] Meanwhile, when the intra prediction mode is equal to or greater than the 34th directional prediction mode (INTRA_ANGULAR34), but cIdx is not 0 or filterFlag is 1, the decoding apparatus may generate a prediction sample by applying a linear interpolation filter as follows (S1534).
[0446] When the value of iFact is not 0, the decoding device can derive the value of the prediction sample predSamples[x][y] according to the following mathematical formula.
[0447] [Mathematical formula 46]
[0448] predSamples[x][y]=
[0449] ((32-iFact)*ref[x+iIdx+1]+iFact*ref[x+iIdx+2]+16)>>5
[0450] According to the above mathematical formula, for a current block with refIdx of 0, nTbw of 8, nTbH of 8, cIdx of 0, and predModeIntra of 65, the value of iFact is calculated to be 29, and the value of iIdx is calculated to be 0. In this case, an example of the predicted sample value of predSamples[0],[0] calculated according to the above mathematical formula is as follows Fig.32 shown.
[0451] Meanwhile, when the value of iFact is 0, the decoding apparatus may derive the value of the prediction sample predSamples[x][y] as follows.
[0452] [Mathematical formula 47]
[0453] predSamples[x][y]=ref[x+iIdx+1]
[0454] Meanwhile, when the intra prediction mode is less than the 34th directional prediction mode (INTRA_ANGULAR34), the decoding apparatus according to an embodiment may derive the reference sample array ref[x] as described above, and may derive the iIdx and iFact values according to the following mathematical formulas.
[0455] [Mathematical formula 48]
[0456] iIdx=(((x+1+refIdx)*intraPredAngle)>>5)+refIdx
[0457] [Math 49]
[0458] iFact=((x+1+refIdx)*intraPredAngle)&31
[0459] Next, when cIdx is 0 and refFilterFlag is 0, the decoding apparatus may generate a prediction sample by applying a cubic filter as follows. First, the interpolation filter coefficient fT[j] may be derived as shown in the following mathematical formula. Here, j=0 to 3.
[0460] [Mathematical formula 50]
[0461] fT[j]=fC[iFact][j]
[0462] Next, the decoding device can derive the value of the predicted sample predSamples[x][y] as shown in the following mathematical formula.
[0463] [Mathematical formula 51]
[0464]
[0465] Meanwhile, when the intra prediction mode is less than the 34th directional prediction mode (INTRA_ANGULAR34) but cIdx is not 0 or refFilterFlag is 1, the decoding apparatus according to an embodiment may generate a prediction sample by applying a linear interpolation filter as follows.
[0466] When the value of iFact is not 0, the decoding device can derive the value of the prediction sample predSamples[x][y] according to the following mathematical formula.
[0467] [Mathematical formula 52]
[0468] predSamples[x][y]=
[0469] ((32-iFact)*ref[y+iIdx+1]+iFact*ref[y+iIdx+2]+16)>>5
[0470] Meanwhile, when the value of iFact is 0, the decoding apparatus may derive the value of the prediction sample predSamples[x][y] as follows.
[0471] [Mathematical formula 53]
[0472] predSamples[x][y]=ref[y+iIdx+1]
[0473] Fifth embodiment
[0474] In an embodiment, the decoding device may determine whether at least one condition for performing reference sample filtering is satisfied. The decoding device may store the result of the determination in refFilterFlag, which is a parameter indicating the executability of reference sample filtering. Thereafter, the decoding device may signal the parameter refFilterFlag to the reference sample filtering step and the prediction sample generating step. Therefore, the decoding device may simply use the value of refFilterFlag without recalculating the conditions determined in the reference sample filtering step and the prediction sample generating step for determining the value of refFilterFlag. Therefore, the intra-frame prediction speed of the decoding device may be increased.
[0475] In the following, reference will be made to Fig.33 Give a description. Fig.331 is a view illustrating another embodiment of performing intra prediction using reference sample filtering. First, the decoding device according to the embodiment may perform step S1610 of determining whether certain conditions for reference sample filtering are satisfied before the reference sample filtering step S1620 and the prediction sample generating step S1630. The decoding device according to the embodiment may determine whether certain conditions for reference sample filtering are satisfied based on the value of predModeIntra (S1611).
[0476] For example, when predModeIntra indicates a planar mode or indicates a directional mode of a reference sample at an integer position, the decoding device may determine that certain conditions are met. In this case, the decoding device may determine the value of refFilterFlag to be a first value (S1612). In this case, although the vertical mode and the horizontal mode are modes indicating reference samples at integer positions, the decoding device may determine that the first condition is not met. For example, when the value of predModeIntra is any one of 0, -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, and 80, since the first condition is met, the decoding device may determine the value of refFilterFlag to be a first value. For example, the first value may be 1. In addition, when the value of predModeIntra is not the above value, the decoding device may determine that the first condition is not met and determine the value of refFilterFlag to be a second value (S1613). For example, the second value may be 0. Next, the decoding device may output the determined refFilterFlag value (S1614). The decoding device may output the refFilterFlag value to the reference sample filtering step S1620 and the prediction sample generating step S1630 .
[0477] Next, the decoding device according to the embodiment may perform the reference sample filtering step S1620. The decoding device may determine whether to perform reference sample filtering according to whether the condition for performing reference sample filtering is met. In an embodiment, the decoding device may determine whether to perform reference sample filtering according to the value of refFilterFlag, and generate a reference sample p[x][y] by performing reference sample filtering. Here, x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx. The decoding device according to the embodiment may receive the following encoding information (S1621) including the refFilterFlag for reference sample filtering generated in step S1610.
[0478] -refIdx is a parameter indicating the intra prediction reference line index
[0479] -nTbW is a parameter indicating the transform block width
[0480] -nTbH is a parameter indicating the transform block height
[0481] -refW is a parameter indicating the reference sample width
[0482] -refH is a parameter indicating the reference sample height
[0483] -refFilterFlag is a parameter indicating whether to apply reference sample filtering
[0484] - Unfiltered neighboring samples refUnfilt[x][y], where x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx.
[0485] -cIdx is a parameter indicating the color component of the current block
[0486] The decoding device according to the embodiment may use the obtained encoding information to determine whether to perform reference sample filtering (S1622). When all of the following conditions are true, the decoding device according to the embodiment may set the parameter filterFlag indicating whether to perform reference sample filtering to a first value (S1623). For example, the first value may be 1. In addition, when all of the following conditions are true, the decoding device may perform reference sample filtering. At the same time, when any of the following conditions is not true, the decoding device may set the parameter filterFlag to a second value and may not perform reference sample filtering (S1624). For example, the second value may be 0.
[0487] (Condition 1) refIdx is equal to 0.
[0488] (Condition 2) nTbW*nTbH is greater than 32.
[0489] (Condition 3) cIdx is equal to 0.
[0490] (Condition 4) IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT.
[0491] (Condition 5) refFilterFlag is equal to 1.
[0492] When the value of filterFlag generated in the reference sample filtering step is equal to 1, the decoding device according to this embodiment can derive the filtered sample value p[x][y] as follows: Here, x=-1, y=-1 to refH-1 and x=0 to refW-1, y=-1.
[0493] [Mathematical formula 54]
[0494] p[-1][-1]=(refUnfilt[-1][0]+2*refUnfilt[-1][-1]+refUnfilt[0][-1]+2>>2
[0495] [Mathematical formula 55]
[0496] p[-1][y]=(refUnfilt[-1][y+1]+2*refUnfilt[-1][y]+refUnfilt[-1][y-1]+2)>>2
[0497] Among them, y=0 to refH-2.
[0498] [Mathematical formula 56]
[0499] p[-1][refH-1]=refUnfilt[-1][refH-1]
[0500] [Mathematical formula 57]
[0501] p[x][-1]=(refUnfilt[x-1][-1]+2*refUnfilt[x][-1]+refUnfilt[x+1][-1]+2)>>2, where x=0 to refW-2.
[0502] [Mathematical formula 58]
[0503] p[refW-1][-1]=refUnfilt[refW-1][-1]
[0504] When the value of filterFlag generated in the reference sample filtering step is not 1, the decoding device of this embodiment may set the reference sample value p[x][y] to be equal to the unfiltered sample value refUnfilt[x][y]. Here, x=-1-refIdx, y=-1-refIdx to refH-1 and x=-refIdx to refW-1, y=-1-refIdx.
[0505] Next, the decoding device according to the embodiment may perform the prediction sample generation step S1630. The decoding device may determine the filter of the intra prediction mode according to the value of refFilterFlag in the prediction sample generation step S1630. For example, the decoding device according to the embodiment may determine the filter of the intra prediction mode according to the value of refFilterFlag of the directional intra prediction mode of INTRA_ANGULAR2 to INTRA_ANGULAR66, and generate the prediction sample predSamples[x][y] by applying the filter to the reference sample. Here, x=0 to nTbW-1, y=0 to nTbH-1.
[0506] For the above steps, the decoding device according to the embodiment can obtain the following encoding information, including the refFilterFlag (S1631) generated in step S1610.
[0507] –predModeIntra is a parameter indicating the intra prediction mode
[0508] –refIdx is a parameter indicating the intra prediction reference line index
[0509] –nTbW is a parameter indicating the transform block width
[0510] –nTbH is a parameter indicating the transform block height
[0511] –refW is a parameter indicating the reference sample width
[0512] –refH is the height of the reference sample
[0513] –nCbW is a parameter indicating the coding block width
[0514] –nCbH is a parameter indicating the compilation block height
[0515] –refFilterFlag is a parameter indicating the reference filter flag value
[0516] –cIdx is a parameter indicating the color component of the current block
[0517] – Neighboring samples p[x][y], where x=-1-refIdx, y=-1-refIdx to refH-1, x=-refIdx to refW-1, y=-1-refIdx.
[0518] Furthermore, the decoding apparatus according to the embodiment may generate a variable nTbS having a value of (Log2(nTbW)+Log2(nTbH))>>1.
[0519] The decoding device according to the embodiment determines whether the condition for using the sharpening filter is met (S1632). For example, when one or more of the following conditions are true, the decoding device may determine that the condition for using the sharpening filter is met and set the value of filterFlag to a first value (S1633). For example, the first value may be 0.
[0520] (Condition 1) The value of refFilterFlag is 1.
[0521] (Condition 2) The value of refIdx is not 0.
[0522] (Condition 3) IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT.
[0523] At the same time, when all the above conditions are not met, the decoding device can set filterFlag to a second value (S1634). Here, the second value can be 1, that is, when the value of refFilterFlag is 0, the decoding device can set the value of filterFlag to 1, and when the value of refFilterFlag is 1, the decoding device can set the value of filterFlag to 0.
[0524] Meanwhile, in an embodiment, the decoding apparatus may perform steps S1632 to S1634 by storing the calculation result of 1-refFilterFlag as the filterFlag value.
[0525] At the same time, when all conditions 1 to 3 are not met, the decoding device may determine the filterFlag value by further considering additional conditions. For example, when minDistVerHor is greater than intraHorVerDistThres[nTbS], the decoding device may set filterFlag to a second value. Here, the second value may be 1. At the same time, when minDistVerHor is less than or equal to intraHorVerDistThres[nTbS], the decoding device may set filterFlag to a first value. Here, the first value may be 0. For this calculation, the decoding device may set the parameter minDistVerHor to Min(Abs(predModeIntra-50) and Abs(predModeIntra-18)) and set the parameter intraHorVerDistThres[nTbS] according to the transform block size nTbSa, as shown in the following table.
[0526] [Table 5]
[0527]
[0528] Although the exemplary method of the present disclosure described above is represented as a series of operations for the sake of clarity, it is not intended to limit the order of performing the steps, and these steps can be performed simultaneously or in different orders as needed. In order to implement the method according to the present disclosure, the steps described may also include other steps, may include the remaining steps except some steps, or may include other additional steps except some steps.
[0529] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming an execution condition or situation of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is met, the image encoding device or the image decoding device may perform the predetermined operation after determining whether the predetermined condition is met.
[0530] The various embodiments of the present disclosure are not a list of all possible combinations, but are intended to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.
[0531] Various embodiments of the present disclosure may be implemented in hardware, firmware, software or a combination thereof. In the case of implementing the present disclosure in hardware, the present disclosure may be implemented in an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.
[0532] In addition, the image decoding device and the image encoding device to which the embodiments of the present disclosure are applied may be included in multimedia broadcast sending and receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video on demand (VoD) service providing devices, OTT video (over-the-top video) devices, Internet streaming service providing devices, three-dimensional (3D) video devices, video phone video devices, medical video devices, etc., and may be used to process video signals or data signals. For example, OTT video devices may include game consoles, Blu-ray players, Internet access televisions, home theater systems, smart phones, tablet PCs, digital video recorders (DVRs), etc.
[0533] Fig.34 is a view showing a content streaming system to which an embodiment of the present disclosure can be applied.
[0534] like Fig.34As shown, the content streaming transmission system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming transmission server, a network server, a media storage, a user device, and a multimedia input device.
[0535] The encoding server compresses the content input from multimedia input devices such as smart phones, cameras, and camcorders into digital data to generate a bit stream and transmits the bit stream to the streaming server. For example, when multimedia input devices such as smart phones, cameras, and camcorders directly generate bit streams, the encoding server can be omitted.
[0536] The bit stream may be generated by applying the image encoding method or the image encoding device according to the embodiment of the present invention, and the streaming server may temporarily store the bit stream during the process of transmitting or receiving the bit stream.
[0537] The streaming server transmits multimedia data to the user device based on the user's request through the network server, and the network server acts as a medium to notify the user of the service. When the user requests the required service from the network server, the network server can transmit it to the streaming server, and the streaming server can transmit the multimedia data to the user. In this case, the content streaming system can include a separate control server. In this case, the control server is used to control the command / response between the devices in the content streaming system.
[0538] The streaming server may receive content from a media storage and / or encoding server. For example, when content is received from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined time.
[0539] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet computers, tablet computers, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.
[0540] Each server in the content streaming system may operate as a distributed server, in which case data received from each server may be distributed.
[0541] The operation of the method according to the various embodiments described above can be run on a device or computer through software executed on the device or computer or instructions (e.g., operating system, application, firmware, program, etc.) that can be executed on the device. And such software or instructions executable in the device can be recorded in a non-transitory computer-readable medium, which can be executed by a component of the device or computer. Computer-readable recording media may include program instructions, data files, data structures, etc., alone or in combination. The program instructions recorded in the computer-readable recording medium may be specially designed and configured to implement the present disclosure, or may be known and used by technicians in the field of computer software. Examples of computer-readable recording media may include magnetic recording media such as hard disks, floppy disks, and tapes; optical data storage media such as CD-ROMs or DVD-ROMs; magnetic optimized media such as floppy disks; hardware devices such as read-only memories (ROMs), random access memories (RAMs), flash memories, etc., which are specially constructed to store and implement program instructions. Examples of program instructions include not only machine language codes formatted by compilers, but also high-level language codes that can be implemented by computers using interpreters. Hardware devices may be configured to be operated by one or more software modules, or vice versa, to carry out processes according to the present invention.
[0542] In the above, the specific technical ideas of video or image encoding and decoding are described in combination with limited embodiments and drawings, but this is only to help overall understanding, and the technical ideas described in the present invention are not limited to the embodiments. Ordinary technicians in the field of the technical ideas described in the present disclosure can design various modifications and changes from these descriptions.
[0543] Therefore, the technical ideas described in the present disclosure should not be limited to the above-mentioned embodiments, and the claims to be described below and all modifications equivalent or equivalent to these claims fall within the scope of the present disclosure.
[0544] Industrial Applicability
[0545] The embodiments of the present disclosure may be used to encode or decode a video.
Claims
1. An image decoding method performed by an image decoding device, the image decoding method include: Obtaining intra prediction mode information of a current block from a bitstream; Determine the intra prediction mode of the current block based on the intra prediction mode information of the current block and the intra prediction modes of neighboring blocks located around the current block; deriving a value of a reference sample filter flag based on coding parameters for the current block; Determine whether to filter the reference sample for intra prediction of the current block based on the reference sample filter flag and the size of the current block; selecting an interpolation filter for intra-prediction samples of the current block based on the reference sample filter flag; as well as generating intra prediction samples of the current block using the selected interpolation filter, Among them, the coding parameters include a multi-reference line index of the current block, a size of the current block, a color component of the current block, whether to apply an intra-sub-partition ISP to the current block, or at least one of an intra-prediction mode of the current block.
2. The image decoding method according to claim 1, in, Based on the value of the reference sample filter flag being a first value, filtering of the reference sample for intra prediction is not performed, wherein, based on the value of the reference sample filter flag being a second value, filtering of the reference sample for intra-frame prediction is used, wherein, based on the value of the reference sample filter flag being a first value, the interpolation filter for the intra-frame prediction samples of the current block is determined to be a cubic filter, and Wherein, based on the value of the reference sample filter flag being a second value, the interpolation filter for the intra-frame prediction samples of the current block is determined to be a Gaussian filter.
3. The image decoding method according to claim 1, in, Selecting the interpolation filter for intra-prediction samples of the current block based on the reference sample filter flag comprises: deriving a value of an interpolation filter flag for intra-prediction samples of the current block based on the reference sample filter flag; and An interpolation filter is selected based on the interpolation filter flag.
4. The image decoding method according to claim 3, in, The value of the interpolation filter flag is derived as a first value based on the intra prediction mode corresponding to a predetermined intra prediction mode.
5. The image decoding method according to claim 4, in, Based on the intra sub-partition ISP being applied to the current block, the value of the interpolation filter flag is derived as a first value.
6. The image decoding method according to claim 4, in, Based on the intra-frame sub-partition ISP being applied to the current block, the current block being a luminance block, the intra-frame prediction mode of the current block being equal to or greater than a predetermined mode and the width of the current block being equal to or greater than a predetermined size, the value of the interpolation filter flag is derived as a first value.
7. The image decoding method according to claim 6, in, The predetermined mode is the intra prediction mode numbered 34, and the predetermined size is 8.
8. The image decoding method according to claim 4, in, Based on the intra-frame sub-partition ISP being applied to the current block, the current block being a luminance block, the intra-frame prediction mode of the current block being equal to or less than a predetermined mode and the width of the current block being equal to or greater than a predetermined size, the value of the interpolation filter flag is derived as a first value.
9. The image decoding method according to claim 8, in, The predetermined mode is the intra prediction mode numbered 34, and the predetermined size is 8.
10. The image decoding method according to claim 3, in, Based on the value of the reference sample filter flag being a first value, filtering of the reference sample for intra prediction is not performed, wherein filtering of the reference sample of intra-frame prediction is performed based on the value of the reference sample filter flag being a second value, and Wherein, based on the value of the reference sample filter flag being 1, the value of the interpolation filter flag is derived to be 0.
11. The image decoding method according to claim 10, in, Based on the value of the interpolation filter flag being the first value, the interpolation filter is a cubic filter, wherein the value based on the interpolation filter flag is a second value, the interpolation filter is a Gaussian filter, and Wherein, based on the value of the reference sample filter flag being the second value, the value of the interpolation filter flag is derived as the first value.
12. An image encoding method performed by an image encoding device, the image encoding method include: Selecting an intra prediction mode for the current block; deriving a value of a reference sample filter flag based on coding parameters for the current block; Determine whether to filter the reference sample for intra prediction of the current block based on the reference sample filter flag and the size of the current block; selecting an interpolation filter for intra-prediction samples of the current block based on the reference sample filter flag; generating intra-frame prediction samples of the current block using the selected interpolation filter; as well as encoding the intra prediction mode of the current block based on the intra prediction modes of neighboring blocks located around the current block, Among them, the coding parameters include a multi-reference line index of the current block, a size of the current block, a color component of the current block, whether to apply an intra-sub-partition ISP to the current block, or at least one of an intra-prediction mode of the current block.
13. A method for transmitting a bit stream generated by an image encoding method, the image encoding method include: Selecting an intra prediction mode for the current block; deriving a value of a reference sample filter flag based on coding parameters for the current block; Determine whether to filter the reference sample for intra prediction of the current block based on the reference sample filter flag and the size of the current block; selecting an interpolation filter for intra-prediction samples of the current block based on the reference sample filter flag; generating intra-frame prediction samples of the current block using the selected interpolation filter; as well as encoding the intra prediction mode of the current block based on the intra prediction modes of neighboring blocks located around the current block, Among them, the coding parameters include a multi-reference line index of the current block, a size of the current block, a color component of the current block, whether to apply an intra-frame sub-partition ISP to the current block, or at least one of the intra-frame prediction modes of the current block.
