Method and device for processing image based on internal prediction mode
By configuring the MPM list according to predefined conditions, and using the intraprediction mode of neighboring blocks to generate the prediction block of the video block, the problems of imbalance in the complexity of the intraprediction mode and the compilation efficiency in the prior art are solved, and the compression efficiency of video content is improved and Throughput.
Patent Information
- Application Number
- CN201980058756.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2018-09-14
- Filing Date
- 2019-09-11
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2039-09-11
AI Technical Summary
Prior art When using 67 intraprediction modes, it is difficult to balance the complexity of the MPM candidate list and the compilation efficiency of the conventional image compression technology, resulting in a reduced throughput.
By configuring the MPM list according to predefined conditions, the prediction blocks of the current block are generated using the intraprediction mode of the left neighbor block and the upper neighbor block, reducing signaling overhead and improving compilation efficiency.
It effectively reduces complexity, improves compression efficiency, enhances compilation efficiency, reduces hardware operation cycles, and improves the throughput of processing video content.
Smart Images

Figure CN112689999B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a method for processing a still image or a video, and more particularly, to a method for encoding / decoding a still image or a video based on an intra prediction mode and a device supporting the method. Background Art
[0002] Compression coding means a series of signal processing techniques for sending digitized information through communication lines, or a technique for storing information in a form suitable for storage media. Media including pictures, images, audio, etc. can be targets for compression coding, and in particular, a technique for performing compression coding on pictures is called video image compression.
[0003] The next generation of video content is assumed to have the characteristics of high spatial resolution, high frame rate and high dimensionality of scene representation. In order to process such content, it will lead to a dramatic increase in memory storage, memory access rate and processing power.
[0004] Therefore, there is a need for compilation tools designed to efficiently process next generation video content. Summary of the invention
[0005] Technical issues
[0006] When 67 intra prediction modes are used for intra prediction, the MPM list including three MPM candidates used in conventional image compression techniques is insufficient to indicate the increased diversity of intra prediction modes. In addition, the complexity of the MPM list including six MPM candidates including a pruning check function and a function of searching all defined neighboring positions until a predetermined number of MPM candidates (or the length of the MPM list) is too high, which may greatly affect throughput.
[0007] Therefore, the present disclosure proposes a method for effectively configuring an MPM list to effectively balance complexity and compilation efficiency so as to alleviate such problems.
[0008] The objects of the present disclosure are not limited to the aforementioned objects, and other unmentioned objects will be apparent to those of ordinary skill in the art from the following description.
[0009] Technical Solution
[0010] According to an embodiment of the present disclosure, a method for decoding an image based on an intra-prediction mode may include: obtaining a most probable mode (MPM) flag indicating whether a most probable mode (MPM) is applied to a current block, the MPM representing a mode in which the intra-prediction mode of the current block is derived from intra-prediction blocks around the current block; when the MPM is applied to the current block, configuring an MPM list based on whether the intra-prediction modes of the left neighboring block and the upper neighboring block of the current block meet a predefined condition, obtaining an MPM index indicating a specific intra-prediction mode in the MPM list, and generating a prediction block of the current block using the intra-prediction mode specified by the MPM index.
[0011] Preferably, the left neighboring block may be set to a block including pixels horizontally adjacent to a lower left sample in the current block, and the upper neighboring block may be set to a block including pixels vertically adjacent to an upper right sample in the current block.
[0012] Preferably, configuring the MPM list may include configuring one of a specific number of MPM lists based on four predefined conditions.
[0013] Preferably, configuring the MPM list may include: configuring the first MPM list when the first condition is met; configuring the second MPM list when the second and third conditions are met but the first condition is not met; configuring the third MPM list when the second and fourth conditions are met but the first and third conditions are not met, configuring the fourth MPM list when the second condition is met but the first, third and fourth conditions are not met, and configuring the fifth MPM list when the first and second conditions are not met.
[0014] Preferably, the first condition may be set to whether the intra-prediction modes of the left neighboring block and the right neighboring block are less than or equal to the DC mode, the second condition may be set to whether the intra-prediction modes of the left neighboring block and the right neighboring block are unequal to each other, the third condition may be set to whether the intra-prediction mode of the left neighboring block or the intra-prediction mode of the right neighboring block is less than or equal to the DC mode, and the fourth condition may be set to whether the difference between the maximum mode indicating a mode with a larger mode index value among the intra-prediction modes of the left neighboring block and the right neighboring block and the minimum mode indicating a mode with a smaller mode index value among the intra-prediction modes of the left neighboring block and the right neighboring block is less than a predefined specific value.
[0015] According to another embodiment of the present disclosure, a device for decoding an image based on an intra prediction mode may include: a memory storing a video signal; and a processor coupled to the memory. The processor may obtain a most probable mode (MPM) flag indicating whether a most probable mode (MPM) is applied to a current block, the MPM indicating a mode in which the intra prediction mode of the current block can be derived from intra prediction blocks around the current block, when the MPM is applied to the current block, configure an MPM list based on whether the intra prediction modes of the left neighboring block and the upper neighboring block of the current block meet a predefined condition, obtain an MPM index indicating a specific intra prediction mode in the MPM list, and use the intra prediction mode specified by the MPM index to generate a prediction block of the current block.
[0016] Beneficial Effects
[0017] According to an embodiment of the present disclosure, complexity can be alleviated and compression efficiency can be increased by effectively deriving an MPM list according to predefined conditions.
[0018] The effects of the present disclosure are not limited to the foregoing, and other unmentioned effects will be apparent to those of ordinary skill in the art from the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are included as a part of the detailed description to help understand the present disclosure, provide embodiments of the present disclosure, and describe technical features of the present disclosure together with the detailed description.
[0020] Figure 1 An example of functional elements of an encoder as an example of an apparatus for processing a video signal according to an embodiment of the present disclosure is illustrated.
[0021] Figure 2 is a schematic block diagram of a decoding device to which an embodiment of the present disclosure is applied, and in which decoding of a video / image signal is performed.
[0022] Figure 3 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating an example of a multi-type tree structure.
[0023] Figure 4 is a diagram to which an embodiment of the present disclosure can be applied and which illustrates a signaling mechanism of partition division information of a quadtree structure having nested multi-type trees.
[0024] Figure 5 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating a method of splitting a CTU into a plurality of CUs based on a quadtree and a nested multi-type tree structure.
[0025] Figure 6 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating a method of constraining binary tree partitioning.
[0026] Figure 7 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating a redundant partitioning pattern that may occur in binary tree partitioning and ternary tree partitioning.
[0027] Figure 8 and Fig. 9 2 is a diagram illustrating a video / image encoding method based on intra prediction according to an embodiment of the present disclosure and an intra predictor within an encoding device according to an embodiment of the present disclosure.
[0028] Fig.10 and Fig.11 2 is a diagram illustrating a video / image decoding method based on intra prediction according to an embodiment of the present disclosure and an intra predictor in a decoding device according to an embodiment of the present disclosure.
[0029] Fig.12 and Fig.13 is a view illustrating a prediction direction of an intra prediction mode according to an embodiment of the present disclosure.
[0030] Fig.14 An example of neighboring blocks used as MPM candidates according to an embodiment of the present disclosure is illustrated.
[0031] Fig.15 is a flowchart illustrating a method of configuring an MPM list based on predefined conditions according to an embodiment of the present disclosure.
[0032] Fig.16 is a flowchart illustrating a method of configuring an MPM list according to a predefined condition by using a shared list according to an embodiment of the present disclosure.
[0033] Fig.17 is a flowchart illustrating a method of configuring an MPM list based on predefined conditions according to an embodiment of the present disclosure.
[0034] Fig.18 is a flowchart illustrating a method of configuring an MPM list based on predefined conditions according to an embodiment of the present disclosure.
[0035] Fig.19 is a flowchart illustrating a method of generating an intra prediction block according to an embodiment of the present disclosure.
[0036] Fig.21 is a block diagram illustrating an example apparatus for processing a video signal according to an embodiment of the present invention. Fig.18 FIG. 1 illustrates a video coding system according to the present disclosure.
[0037] Fig. 22 is a view illustrating a structure of a content streaming system according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0038] Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. The description to be described below in conjunction with the accompanying drawings is for the purpose of describing exemplary embodiments of the present disclosure and is not intended to describe the only embodiment in which the present disclosure can be implemented. The following description includes specific details in order to provide a complete understanding of the present disclosure. However, it should be understood that for those skilled in the art, the present disclosure can be embodied without the specific details.
[0039] In some cases, in order to prevent the technical concepts of the present disclosure from being unclear, well-known structures or devices may be omitted, or may be described as a block diagram centered on the core functions of the structure or device.
[0040] In addition, although the currently widely used general terms are selected as terms as much as possible in the present disclosure, terms arbitrarily selected by the applicant are used in specific cases. Because the meaning of the terms will be clearly described in the corresponding part of the specification in this case, it is understood that the present disclosure will not be simply interpreted only by the terms used in the specification of the present disclosure, but the meaning of the terms should be clarified.
[0041] Specific terms used in the following description may be provided to help understand the present disclosure. In addition, within the scope of the technical concept of the present disclosure, specific terms may be modified to other forms. For example, signals, data, samples, pictures, frames, blocks, etc. may be appropriately replaced and interpreted in each compilation process.
[0042] Hereinafter, in this specification, a "processing unit" means a unit in which a coding / decoding process such as prediction, transformation and / or quantization is performed. Hereinafter, for ease of description, a processing unit may also be referred to as a "processing block" or a "block".
[0043] A processing unit may be interpreted as having a meaning including a unit for a luma component and a unit for a chroma component. For example, a processing unit may correspond to a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU).
[0044] In addition, the processing unit may be interpreted as a unit for a luma component or a unit for a chroma component. For example, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB) for a luma component. Alternatively, the processing unit may correspond to a coding tree block (CTB), a coding block (CB), a prediction block (PB), or a transform block (TB) for a chroma component. In addition, the present disclosure is not limited thereto, and the processing unit may be interpreted as including the meaning of a unit for a luma component and a unit for a chroma component.
[0045] Furthermore, the processing unit is not limited to a square block per se and may be configured in the form of a polygon having three or more vertices.
[0046] In addition, hereinafter, in this specification, pixels, picture elements, etc. are generally referred to as samples. In addition, using samples may mean using pixel values, picture element values, etc.
[0047] Figure 1 An example of functional elements of an encoder as an example of an apparatus for processing a video signal according to an embodiment of the present disclosure is illustrated.
[0048] refer to Figure 1 , the encoding device 100 may be configured to include an image divider 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter predictor 180, an intra predictor 185, and an entropy encoder 190. The inter predictor 180 and the intra predictor 185 may be generally referred to as a predictor. In other words, the predictor may include the inter predictor 180 and the intra predictor 185. The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may further include a subtractor 115. In one embodiment, the image divider 110, the subtractor 115, the transformer 120, the quantizer 130, the dequantizer 140, the inverse transformer 150, the adder 155, the filter 160, the inter-predictor 180, the intra-predictor 185 and the entropy encoder 190 may be configured as one hardware component (e.g., an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured with a digital storage medium.
[0049] The image divider 110 may divide the input image (or picture or frame) input to the encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively partitioned from a coding tree unit (CTU) or a maximum coding unit (LCU) based on a quadtree binary tree (QTBT) structure. For example, based on a quadtree structure and / or a binary tree structure, a coding unit may be partitioned into a plurality of coding units with a deeper depth. In this case, for example, a quadtree structure may be applied first, and then a binary tree structure may be applied. Alternatively, a binary tree structure may be applied first. The coding process according to the present disclosure may be performed based on a final coding unit that is no longer partitioned. In this case, the maximum coding unit may be directly used as a final coding unit based on coding efficiency according to image characteristics, or, if necessary, the coding unit may be recursively partitioned into coding units of a deeper depth. Therefore, a coding unit with an optimal size may be used as a final coding unit. In this case, the coding process may include a process such as prediction, transformation, or reconstruction, which will be described later. For another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, each of the prediction unit and the transform unit may be divided or partitioned from each final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit from which a transform coefficient is derived and / or a unit in which a residual signal is derived from the transform coefficient.
[0050] Depending on the situation, a unit may be used interchangeably with a block or region. In general, an M×N block may indicate a sample set or a transform coefficient set configured with M columns and N rows. In general, a sample may indicate a pixel or a pixel value, and may indicate only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component. In a sample, a picture (or image) may be used as a term corresponding to a pixel or a pel.
[0051] The encoding device 100 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output by the inter predictor 180 or the intra predictor 185 from the input image signal (original block or original sample array). The generated residual signal is sent to the transformer 120. In this case, as illustrated, the unit in the encoder 100 in which the prediction signal (prediction block or prediction sample array) is subtracted from the input image signal (original block or original sample array) can be referred to as a subtractor 115. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block including prediction samples for the current block. The predictor can determine whether intra prediction or inter prediction is applied in the current block or CU unit. The predictor can generate various information about prediction, such as prediction mode information to be described later in the description of each prediction mode, and can send the information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and can be output in the form of a bitstream.
[0052] The intra predictor 185 may predict the current block with reference to samples within the current picture. The referenced samples may be positioned adjacent to the current block or may be separated from the current block depending on the prediction mode. In the intra prediction, the prediction mode may include multiple non-angular modes and multiple angular modes. For example, the non-angular mode may include a DC mode and a planar mode, for example. For example, depending on the fineness of the prediction direction, the angular mode may include 33 angular prediction modes or 65 angular prediction modes. In this case, for example, depending on the configuration, an angular prediction mode greater than or less than 33 angular prediction modes or 65 angular prediction modes may be used. The intra predictor 185 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.
[0053] The inter-predictor 180 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-prediction mode, the motion information may be predicted as a block, a sub-block, or a sample unit based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-prediction, the neighboring blocks may include spatial neighboring blocks within the current picture and temporal neighboring blocks within the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a co-located reference block or a co-located CU (colCU). The reference picture including the temporal neighboring block may be referred to as a co-located picture (colPic). For example, the inter-predictor 180 may construct a motion information candidate list based on the neighboring blocks, and may generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter-prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter predictor 180 may use the motion information of the neighboring block as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be sent. In the case of motion information prediction (MVP) mode, the motion vector of the neighboring block may be used as a motion vector predictor. The motion vector of the current block may be indicated by signaling a motion vector difference.
[0054] The prediction signal generated by the inter predictor 180 or the intra predictor 185 may be used to generate a reconstructed signal or a residual signal.
[0055] The transformer 120 may generate a transform coefficient by applying a transform scheme to the residual signal. For example, the transform scheme may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loève transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). In this case, if the relationship information between pixels is represented as a graph, the GBT means a transform obtained from the graph. The CNT means a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform process may be applied to pixel blocks having the same size of a square, or may be applied to blocks having a variable size instead of a square.
[0056] The quantizer 130 may quantize the transform coefficients and send them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output it in the form of a bitstream. Information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the quantized transform coefficients in the form of a block in the form of a one-dimensional vector based on the coefficient scanning sequence, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the form of a one-dimensional vector. The entropy encoder 190 may perform various encoding methods such as exponential Golomb, context adaptive variable length coding (CAVLC), and context adaptive binary arithmetic coding (CABAC). In addition to the quantized transform coefficients, the entropy encoder 190 may also encode information necessary for video / image reconstruction (e.g., the value of a syntax element) together or separately. The encoded information (e.g., the encoded video / image information) may be transmitted or stored in a network abstraction layer (NAL) unit in the form of a bitstream. The bitstream may be transmitted over a network or may be stored in a digital storage medium. In this case, the network may include a broadcast network and / or a communication network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) that transmits a signal output by the entropy encoder 190 and / or a memory (not shown) for storing the signal may be configured as an internal / external element of the encoding device 100, or the transmitter may be an element of the entropy encoder 190.
[0057] The quantized transform coefficients output by the quantizer 130 can be used to generate a prediction signal. For example, the residual signal can be reconstructed by applying dequantization and inverse transformation to the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150 in the loop. The adder 155 can add the reconstructed residual signal to the prediction signal output by the inter-predictor 180 or the intra-predictor 185, so that a reconstructed signal (reconstructed picture, reconstructed block or reconstructed sample array) can be generated. If there is no residual for processing the target block as in the case where the skip mode has been applied, the prediction block can be used as a reconstructed block. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for the intra-prediction of the next processing target block in the current picture, and can be used for the inter-prediction of the next picture performed by filtering as will be described later.
[0058] The filter 160 can improve the subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture. The modified reconstructed picture can be stored in the memory 170, specifically, in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filter 160 can generate various information for filtering, as will be described later in the description of each filtering method, and they can be sent to the entropy encoder 190. The filter information can be encoded by the entropy encoder 190 and output in the form of a bit stream.
[0059] The modified reconstructed picture transmitted to the memory 170 may be used as a reference picture in the inter predictor 180. The encoding apparatus may avoid prediction mismatch in the encoding apparatus 100 and the decoding apparatus, and improve encoding efficiency if inter prediction is applied.
[0060] The DPB of the memory 170 may store the modified reconstructed picture so that the modified reconstructed picture is used as a reference picture in the inter predictor 180. The memory 170 may store the motion information of the block from which the motion information in the current picture has been derived (or encoded), and / or the motion information of the block in the reconstructed picture. The stored motion information may be sent to the inter predictor 180 so as to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 may store the reconstructed samples of the reconstructed block in the current picture and may send them to the intra predictor 185.
[0061] Figure 2 is a schematic block diagram of a decoding device to which an embodiment of the present disclosure is applied, and in which decoding of a video / image signal is performed.
[0062] refer to Figure 2 , the decoding device 200 may be configured to include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-predictor 260, and an intra-predictor 265. The inter-predictor 260 and the intra-predictor 265 may be collectively referred to as a predictor. That is, the predictor may include an inter-predictor 180 and an intra-predictor 185. The dequantizer 220 and the inverse transformer 230 may be collectively referred to as a residual processor. That is, the residual processor may include a dequantizer 220 and an inverse transformer 230. The entropy decoder 210, the dequantizer 220, the inverse transformer 230, the adder 235, the filter 240, the inter-predictor 260, and the intra-predictor 265 may be configured as a hardware component (e.g., a decoder or a processor) according to an embodiment. In addition, the memory 170 may include a decoded picture buffer (DPB) and may be configured with a digital storage medium.
[0063] When a bit stream including video / image information is input, the decoding apparatus 200 can Figure 1 The image is reconstructed by the process of processing video / image information in the encoding device of the decoding device. For example, the decoding device 200 can use the processing unit applied in the encoding device to perform decoding. Therefore, for example, the processing unit for decoding can be a coding unit. Depending on the quadtree structure and / or the binary tree structure, the coding unit can be divided from the coding tree unit or the maximum coding unit. In addition, the reconstructed image signal decoded and output by the decoding device 200 can be played back by a playback device.
[0064] The decoding device 200 may receive the Figure 1 The signal output by the encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can derive information (e.g., video / image information) for image reconstruction (or picture reconstruction) by parsing the bitstream. For example, the entropy decoder 210 can decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and can output the value of the syntax element for image reconstruction or the quantized value of the transform coefficient about the residual. More specifically, in the CABAC entropy decoding method, a box (bin) corresponding to each syntax element can be received from the bitstream, and the context model can be determined using the decoding target syntax element information and the decoding information of the neighboring and decoding target blocks or the information of the symbol / bin decoded in the previous step, and the probability of bin occurrence can be predicted based on the determined context model, and the symbol corresponding to the value of each syntax element can be generated by performing arithmetic decoding on the bin. In this case, in the CABAC entropy decoding method, after determining the context model, the context model can be updated using the information of the symbol / bin decoded by the context model for the next symbol / bin. Information about prediction among the information decoded in the entropy decoder 2110 may be provided to the predictor (inter-predictor 260 and intra-predictor 265). Parameter information about the residual value on which entropy decoding has been performed in the entropy decoder 210, that is, the quantized transform coefficient, may be input to the dequantizer 220. In addition, information about filtering among the information decoded in the entropy decoder 210 may be provided to the filter 240. Meanwhile, a receiver (not shown) receiving a signal output by the encoding device may be further configured as an internal / external element of the decoding device 200, or the receiver may be an element of the entropy decoder 210.
[0065] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in a two-dimensional block form. In this case, the rearrangement may be performed based on a coefficient scanning sequence performed in the encoding device. The dequantizer 220 may perform dequantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information), and may obtain the transform coefficients.
[0066] The inverse transformer 230 obtains a residual signal (a residual block or a residual sample array) by inversely transforming the transform coefficients.
[0067] The predictor may perform prediction on the current block and may generate a prediction block including prediction samples for the current block. The predictor may determine whether to apply intra prediction to the current block or inter prediction to the current block based on the information about prediction output by the entropy decoder 210, and may determine a detailed intra / inter prediction mode.
[0068] The intra predictor 265 may predict the current block with reference to samples within the current picture. The referenced samples may be located adjacent to the current block or may be separated from the current block depending on the prediction mode. In intra prediction, the prediction mode may include multiple non-angle modes and multiple angle modes. The intra predictor 265 may use the prediction mode applied to the neighboring blocks to determine the prediction mode applied to the current block.
[0069] The inter-predictor 260 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-prediction mode, the motion information may be predicted as a block, a sub-block, or a sample unit based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter-prediction direction (L0 prediction, L1 prediction, Bi prediction) information. In the case of inter-prediction, the neighboring blocks may include spatial neighboring blocks within the current picture and temporal neighboring blocks within the reference picture. For example, the inter-predictor 260 may configure a motion information candidate list based on neighboring blocks, and may derive a motion vector and / or a reference picture index of the current block based on the received candidate selection information. Inter-prediction may be performed based on various prediction modes. Information about prediction may include information indicating a mode of inter-prediction for the current block.
[0070] The adder 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, or reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block or prediction sample array) output by the inter predictor 260 or the intra predictor 265. If there is no residual for processing the target block in the case where the skip mode has been applied, the prediction block can be used as the reconstructed block.
[0071] The adder 235 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next processing target block within the current picture, and may be used for inter prediction of the next picture by filtering as will be described later.
[0072] The filter 240 can improve the subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be sent to the memory 250, in particular the DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.
[0073] The reconstructed picture stored (modified) in the DPB of the memory 250 can be used as a reference picture in the inter predictor 260. The memory 250 can store the motion information of the block from which the motion information in the current picture has been derived (or decoded) and / or the motion information of the block in the reconstructed picture. The stored motion information can be sent to the inter predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 170 can store the reconstructed samples of the reconstructed block in the current picture and can send them to the intra predictor 265.
[0074] In the present disclosure, the embodiments described in the filter 160, the inter predictor 180, and the intra predictor 185 of the encoding device 100 may be applied to the filter 240, the inter predictor 260, and the intra predictor 265 of the decoding device 200, respectively, identically or in a corresponding manner.
[0075] Block partitioning
[0076] The video / image coding method according to the present document can be performed based on various detailed techniques. Each detailed technique is schematically described as follows. It is obvious to those skilled in the art that the following techniques can be associated with related processes, such as prediction, residual processing ((inverse) transform, (de)quantization), syntax element coding, filtering, and partitioning / partitioning in the video / image encoding / decoding process that has been described and / or will be described later.
[0077] The block partitioning process according to the present document may be performed in the image divider 110 of the encoding device. The information related to the partition may be processed (encoded) in the entropy encoder 190 and sent to the decoding device in the form of a bit stream. The entropy decoder 210 of the decoding device may derive the block partition structure of the current picture based on the information related to the partition obtained from the bit stream, and may perform a series of processes (e.g., prediction, residual processing, block reconstruction, and loop filtering) based on the block partition structure.
[0078] Partition the image into CTUs
[0079] A picture may be partitioned into a series of coding tree units (CTUs). A CTU may correspond to a coding tree block (CTB). Alternatively, a CTU may include a coding tree block of luma samples and two corresponding coding tree blocks of chroma samples. In other words, for a picture including three sample arrays, a CTU may include an N×N block of luma samples and two corresponding blocks of chroma samples.
[0080] The maximum allowed size of a CTU for coding and prediction may be different from the maximum allowed size of a CTU for transform. For example, the maximum allowed size of a luma block within a CTU may be 128×128.
[0081] Use tree structure to divide CTU district
[0082] The CTU may be partitioned based on a quadtree (QT) structure. The quadtree structure may be referred to as a quadtree structure. This is to incorporate various local features. Meanwhile, in this article, the CTU may be partitioned based on a multi-type tree structure partition including a binary tree (BT) and a ternary tree (TT) in addition to the quadtree. Hereinafter, the QTBT structure may include a partition structure based on a quadtree and a binary tree. The QTBTTT may include a partition structure based on a quadtree, a binary tree, and a ternary tree. Alternatively, the QTBT structure may include a partition structure based on a quadtree, a binary tree, and a ternary tree. In the coding tree structure, the CU may have a square or rectangular shape. The CTU may be partitioned into a quadtree structure. Thereafter, the leaf nodes of the quadtree structure may be additionally partitioned by a multi-type tree structure.
[0083] Figure 3 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating an example of a multi-type tree structure.
[0084] In an embodiment of the present disclosure, the multi-type tree structure may include four types of partitions, such as Figure 3As illustrated in . The four partition types may include vertical binary partition SPLIT_BT_VER, horizontal binary partition SPLIT_BT_HOR, vertical ternary partition SPLIT_TT_VER and horizontal ternary partition SPLIT_TT_HOR. The leaf node of the multi-type tree structure may be referred to as a CU. Such a CU may be used for prediction and transformation processes. In this document, generally, a CU, PU or TU may have the same block size. In this case, if the maximum supported transform length is less than the width or height of the color component of the CU, the CU and TU may have different block sizes.
[0085] Figure 4 is a diagram to which an embodiment of the present disclosure can be applied and which illustrates a signaling mechanism of partition division information having a quadtree structure of a nested multi-type tree.
[0086] In this case, the CTU is regarded as the root of the quadtree and is first partitioned into a quadtree structure. Each quadtree leaf node can be further partitioned into a multi-type tree structure. In the multi-type tree structure, a first flag (e.g., mtt_split_cu_flag) is signaled to indicate whether the corresponding node is additionally partitioned. If the corresponding node is additionally partitioned, a second flag (e.g., mtt_split_cu_verticla_flag) can be signaled to indicate the partition direction. Thereafter, a third flag (e.g., mtt_split_cu_binary_flag) can be signaled to indicate whether the partition type is binary or ternary. For example, the multi-type tree partition mode (MttSplitMode) of the CU can be derived as in Table 1 based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag.
[0087] [Table 1]
[0088] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1
[0089] Figure 5 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating a method of splitting a CTU into a plurality of CUs based on a quadtree and a nested multi-type tree structure.
[0090] In this case, the bold block edge indicates a quadtree partition, and the remaining edges indicate a multi-type tree partition. Quadtree partitioning with a multi-type tree can provide a content-adaptive coding tree structure. A CU can correspond to a coding block (CB). Alternatively, a CU can include a coding block of luma samples and two coding blocks of corresponding chroma samples. In a luma sample unit, the size of a CU can be as large as a CTU, or as small as 4x4. For example, in the case of a 4:2:0 color format (or chroma format), the maximum chroma CB size can be 64x64, and the minimum chroma CB size can be 2x2.
[0091] For example, in this document, the maximum allowed luma TB size may be 64x64, and the maximum allowed chroma TB size may be 32x32. If the width or height of the tree-structured CB split is larger than the maximum transform width or height, the corresponding CB may be split until the TB size constraints in the horizontal and vertical directions are automatically (or implicitly) satisfied.
[0092] Meanwhile, for a quadtree compilation tree scheme with multiple types of trees, the following parameters may be defined and recognized as SPS syntax elements.
[0093] CTU size: the root node size of the quadtree
[0094] MinQTSize: The minimum allowed size of a quadtree leaf node
[0095] MaxBtSize: Maximum allowed binary tree root node size
[0096] MaxTtSize: Maximum allowed ternary tree root node size
[0097] MaxMttDepth: The maximum allowed hierarchical depth of the multi-type tree split from the leaves of the quadtree
[0098] MinBtSize: The minimum allowed binary tree leaf node size
[0099] MinTtSize: The minimum allowed ternary tree leaf node size
[0100] For example, in the case of a quadtree coding tree structure with multiple types of trees, the CTU size can be set to a 64x64 block of 128x128 luma samples and two corresponding chroma samples (in 4:2:0 chroma format). In this case, MinOTSize can be set to 16x16, MaxBtSize can be set to 128x128, MaxTtSzie can be set to 64x64, and MinBtSize and MinTtSize (for width and height) can be set to 4x4, and MaxMttDepth can be set to 4. Quadtree partitioning can be applied to the CTU to generate a quadtree leaf node. The quadtree leaf node can be referred to as a leaf QT node. The quadtree leaf node can have a size of 16x16 (i.e., MinOTSize) to a size of 128x128 (i.e., CTU size). If the leaf QT node is 128x128, it may not be additionally split into a binary tree / ternary tree. The reason for doing this is that although the leaf QT node is split, it exceeds MaxBtsize and MaxTtszie (i.e., 64x64). In other cases, the leaf QT node can be additionally split into a multi-type tree. Therefore, the leaf QT node is the root node for the multi-type tree, and the leaf QT node can have a multi-type tree depth (mttDepth) value of 0. If the multi-type tree depth reaches MaxMttdepth (for example, 4), additional splitting methods may no longer be considered. If the width of the multi-type tree node is equal to MinBtSize and is less than or equal to 2xMinTtSize, additional horizontal splitting may no longer be considered. If the height of the multi-type tree node is equal to MinBtSize and is less than or equal to 2xMinTtSize, additional vertical splitting may no longer be considered.
[0101] Figure 6 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating a method of constraining binary tree partitioning.
[0102] refer to Figure 6 In order to allow 64x64 luma blocks and 32x32 chroma pipeline design in hardware decoders, TT splitting can be restricted in certain cases. For example, if the width or height of the luma coding block is greater than a certain preset value (e.g., 32, 64), then Figure 6 As shown in FIG, TT splitting can be restricted.
[0103] In this document, the coding tree scheme can support luma and chroma blocks to have separate block tree structures. With respect to P and B slices, the luma and chroma CTBs within a CTU can be restricted to have the same coding tree structure. However, for I slices, luma and chroma blocks may have separate block tree structures. If a separate block tree mode is applied, the luma CTB can be partitioned into CUs based on a specific coding tree structure. The chroma CTB can be partitioned into chroma CUs based on different coding tree structures. This may mean that a CU within an I slice can be configured with a coding block for a luma component or a coding block for two chroma components, while a CU in a P or B slice can be configured with blocks for three color components.
[0104] In "Partitioning CTU Using Tree Structure", a quadtree coding tree structure having a multi-type tree has been described, but the structure in which the CU is partitioned is not limited thereto. For example, the BT structure and the TT structure may be interpreted as concepts included in a multi-partition tree (MPT) structure, and the CU may be interpreted as being partitioned by the QT structure and the MPT structure. In an example in which the CU is partitioned by the QT structure and the MPT structure, the partition structure may be determined by signaling a syntax element (e.g., MPT_split_type) including information indicating how many blocks a leaf node of the QT structure is partitioned into, and a syntax element (e.g., MPT_split_mode) including information indicating that a leaf node of the QT structure is partitioned in one of a vertical direction and a horizontal direction.
[0105] In another example, a method different from the QT structure, the BT structure, or the TT structure may be used to partition a CU. That is, unlike the case where a CU of a deeper depth is partitioned into a 1 / 4 size of a CU of a shallower depth based on the QT structure, or a CU of a deeper depth is partitioned into a 1 / 2 size of a CU of a shallower depth based on the BT structure, or a CU of a deeper depth is partitioned into a 1 / 4 or 1 / 2 size of a CU of a shallower depth based on the TT structure, a CU of a deeper depth may be partitioned into a size of 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 of a CU of a shallower depth. The method of partitioning a CU is not limited thereto.
[0106] If a portion of the tree node block exceeds the bottom or right picture boundary, the corresponding tree node block can be restricted so that all samples of all coded CUs are located within the picture boundary. In this case, for example, the following partitioning rules can be applied.
[0107] If part of the tree node block exceeds the bottom and right image boundaries,
[0108] If the block is a QT node and the size of the block is greater than the minimum QT size, the block is forced to be split in QT split mode.
[0109] Otherwise, force the block to be split in SPLIT_BT_HOR mode
[0110] Otherwise, if part of the tree node block exceeds the bottom image boundary,
[0111] If the block is a QT node, and the block size is greater than the minimum QT size, and the block size is greater than the maximum BT size, then force the block to be split in QT split mode.
[0112] Otherwise, if the block is a QT node, and the block size is greater than the minimum QT size, and the block size is less than or equal to the maximum BT size, then force the block to be split in QT split mode or SPLIT_BT_HOR mode.
[0113] Otherwise (the block is a BTT node, or the block size is less than or equal to the minimum QT size), force the block to be split in SPLIT_BT_HOR mode.
[0114] Otherwise, if part of the tree node block exceeds the right image boundary,
[0115] If the block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the block is greater than the maximum BT size, then force the block to be split in QT split mode.
[0116] Otherwise, if the block is a QT node, and the size of the block is greater than the minimum QT size, and the size of the block is less than or equal to the maximum BT size, then force the block to be split in QT split mode or SPLIT_BT_VER mode.
[0117] Otherwise (the block is a BTT node, or the block size is less than or equal to the minimum QT size), force the block to be split in SPLIT_BT_VER mode.
[0118] At the same time, the quadtree coding block structure with multiple types of trees can provide a very flexible block partition structure. Due to the partition types supported by the multiple types of trees, depending on different partition modes, the same coding block structure result may potentially be obtained. By limiting the occurrence of such redundant partition modes, the amount of data of the partition information can be reduced. This is described with reference to the following figures.
[0119] Figure 7 is an embodiment to which the present disclosure can be applied, and is a diagram illustrating a redundant partitioning pattern that may occur in binary tree partitioning and ternary tree partitioning.
[0120] like Figure 7As illustrated in , two levels of consecutive binary partitions in one direction have the same coding block structure as the binary partition for the center partition after the ternary partition. In this case, the binary tree partition (in a given direction) for the center partition of the ternary tree partition may be restricted. Such a constraint may be applied to CUs of all pictures. If such a specific partition is constrained, the signaling of the corresponding syntax elements may be modified by incorporating such constraint cases. Therefore, the number of bits signaled for the partition may be reduced. For example, as Figure 7 As in the example illustrated in , if a binary-tree partition for the center partition of a CU is constrained, the mtt_split_cu_binary_flag syntax element indicating whether the partition is a binary or ternary partition is not signaled, and the decoder may infer the corresponding value to be 0.
[0121] predict
[0122] To reconstruct the current processing unit in which decoding is performed, a decoded portion of a current picture or other pictures including the current processing unit may be used.
[0123] A picture (slice) that uses only the current picture for reconstruction, that is, a picture (slice) on which only intra-frame prediction is performed, may be referred to as an intra picture or an I picture (slice). A picture (slice) that uses at most one motion vector and reference index for predicting each unit may be referred to as a predicted picture or a P picture (slice). A picture (slice) that uses at most two motion vectors and reference indexes may be referred to as a bidirectionally predicted picture or a B picture (slice).
[0124] Inter prediction means a prediction method for deriving a current processing block based on data elements (e.g., sample values or motion vectors) of pictures other than the current picture. That is, this method means a method for predicting the pixel value of the current processing block with reference to a reconstructed area within other reconstructed pictures other than the current picture.
[0125] Hereinafter, intra prediction is described in more detail.
[0126] In-screen prediction
[0127] Intra prediction refers to a prediction method that derives the current processing block from data elements (e.g., sample values) of the same decoded picture (or slice). That is, intra prediction refers to a method of predicting the pixel value of the current processing block by referring to a reconstructed area in the current picture.
[0128] The intra prediction may mean a prediction of generating a prediction sample for a current block based on a reference sample outside the current block in a picture to which the current block belongs (hereinafter referred to as the current picture).
[0129] The present disclosure describes the above Figure 1 and Figure 2 The detailed technique of the intra prediction method is described, and in the case of the decoder, it can be expressed as Fig.10 Video / image decoding method based on intra-prediction and Fig.11 In addition, the encoder can be expressed as Figure 8 Video / image coding method based on intra-prediction and Fig. 9 The intra prediction unit in the encoding device is as described below. Figure 8 and Fig. 9 The encoded data may be stored in the form of a bit stream.
[0130] When intra prediction is applied to the current block, neighboring reference samples to be used for intra prediction of the current block may be derived. The neighboring reference samples of the current block may include samples of size nW×nH adjacent to the left boundary of the current block, a total of 2×nH samples adjacent to the lower left side of the current block, samples adjacent to the top boundary of the current block, a total of 2xnW samples adjacent to the upper right side of the current block, and one sample adjacent to the upper left side of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples of size nW×nH adjacent to the right boundary of the current block, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the lower right side of the current block.
[0131] However, some neighboring reference samples of the current block have not been decoded or may not be available. In this case, the decoder can configure the neighboring reference samples to be used for prediction by replacing the unavailable samples with available samples. Alternatively, the decoder can configure the neighboring reference samples for prediction by interpolating the available samples.
[0132] When deriving neighboring reference samples, the prediction samples may be derived i) based on the average or interpolation of the neighboring reference samples of the current block, and (ii) based on the reference samples existing in a specific (prediction) direction of the prediction samples among the neighboring reference samples of the current block. Case (i) may be referred to as a non-directional mode or a non-angle mode, and case (ii) may be referred to as a directional mode or an angle mode. In addition, the prediction samples may be generated by interpolating between the first neighboring sample and the second neighboring sample positioned in the direction opposite to the prediction direction of the intra-prediction mode of the current block relative to the prediction samples of the current block among the neighboring reference samples. The above case may be referred to as linear interpolation intra prediction (LIP). In addition, the temporary prediction samples of the current block may be derived based on the filtered neighboring reference samples, and the prediction samples of the current block may be derived by weighted summing the temporary prediction samples and at least one reference sample derived according to the intra-prediction mode among the existing neighboring reference samples, that is, the unfiltered neighboring reference samples. The above case may be referred to as position-dependent intra prediction (PDPC). At the same time, post-filtering may be performed on the derived prediction samples if necessary.
[0133] Specifically, the intra prediction process may include an intra prediction mode determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode. In addition, a post-filtering step may be performed on the derived prediction samples if necessary.
[0134] Essentially, the following may be included in a video / image encoding process based on intra prediction and intra prediction units in an encoding device.
[0135] Figure 8 and Fig. 9 is a view illustrating an intra prediction unit in a video / image encoding method and an encoding device based on intra prediction according to an embodiment of the present disclosure.
[0136] refer to Figure 8 and Fig. 9 , S801 may be performed by the intra-prediction unit 185 of the encoding device, and S802 may be performed by the residual processing unit of the encoding device. Specifically, S802 may be performed by the subtraction unit 115 of the encoding device. In S803, the prediction information may be derived by the intra-prediction unit 185, and may be encoded by the entropy encoding unit 190. In S803, the residual information may be derived by the residual processing unit, and may be encoded by the entropy encoding unit 190. The residual information is information of the residual sample. The residual information may include information of the quantized transform coefficient for the residual sample.
[0137] As described above, the residual samples may be derived as transform coefficients by the transform unit 120 of the encoding apparatus, and the transform coefficients may be derived as quantized transform coefficients by the quantization unit 130. Information about the quantized transform coefficients may be encoded by the entropy encoding unit 190 through a residual coding process.
[0138] The encoding device performs intra prediction on the current block (S801). The encoding device may derive an intra prediction mode of the current block, derive neighboring reference samples of the current block, and generate prediction samples in the current block based on the intra prediction mode and the neighboring reference samples. Here, the intra prediction mode determination and the derivation of the neighboring reference samples may be performed simultaneously, or one may be performed before the other. For example, the intra prediction unit 185 of the encoding device may include a prediction mode determination unit 186, a reference sample derivation unit 187, and a prediction sample derivation unit 188. The prediction mode determination unit 186 may determine an intra prediction mode for the current block. The reference sample derivation unit 187 may derive neighboring reference samples of the current block, and the prediction sample derivation unit 188 may derive motion samples of the current block. Meanwhile, although not shown, when performing the prediction sample filtering process described below, the intra prediction unit 185 may further include a prediction sample filtering unit (not shown). The encoding device may determine a mode applied to the current block from a plurality of intra prediction modes. The encoding device may compare the RD costs of the intra prediction modes and determine the best intra prediction mode for the current block.
[0139] At the same time, the encoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post-filtering. Some or all prediction samples may be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process may be omitted.
[0140] The encoding device generates residual samples for the current block based on the (filtered) prediction samples (S802). The encoding device can encode image information, which includes prediction mode information indicating an intra-prediction mode and residual information for the residual samples (S803). The encoded image information can be output in the form of a bit stream. The output bit stream can be delivered to a decoding device via a storage medium or a network.
[0141] At the same time, as described above, the encoding device can generate a reconstructed picture (including a reconstructed sample and a reconstructed block) based on the reference sample and the residual sample. This is to enable the encoding device to derive the same prediction result as the prediction result obtained by the decoding device, and thereby improve the coding efficiency. As described above, loop filtering can also be performed on the reconstructed picture as described above.
[0142] Fig.10 and Fig.11is a view illustrating an intra prediction unit in a video / image decoding method and a decoding device based on intra prediction according to an embodiment of the present disclosure.
[0143] refer to Fig.10 and Fig.11 The decoding device may perform operations corresponding to the operations performed by the encoding device. The decoding device may perform prediction on the current block based on the received prediction information and derive a prediction sample.
[0144] Specifically, the decoding device may derive the intra prediction mode of the current block based on the received prediction mode information (S1001). The decoding device may derive the neighboring reference samples of the current block (S1002). The decoding device generates prediction samples in the current block based on the intra prediction mode and the neighboring reference samples (S1003). In this case, the decoding device may perform a prediction sample filtering process. Prediction sample filtering may be referred to as post filtering. Some or all prediction samples may be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process may be omitted.
[0145] The decoding device generates residual samples of the current block based on the received residual information (S1004). The decoding device may generate reconstructed samples of the current block based on the (filtered) prediction samples and the residual samples, and generate a reconstructed picture based thereon (S1005).
[0146] Here, the intra prediction unit 265 of the decoding device may include a prediction mode determination unit 266, a reference sample derivation unit 267, and a prediction sample derivation unit 268. The prediction mode determination unit 266 may determine the intra prediction mode of the current block based on the prediction mode information received from the prediction mode determination unit 186 of the encoding device. The reference sample derivation unit 266 may derive the neighboring reference samples of the current block, and the sample derivation unit 267 may derive the prediction sample of the current block. Meanwhile, although not shown, when the prediction sample filtering process as described above is performed, the intra prediction unit 265 may further include a prediction sample filtering unit (not shown).
[0147] The prediction mode information may include flag information (e.g., prev_intra_luma_pred_flag) indicating whether the most probable mode (MPM) is applied to the current block or the remaining modes are applied thereto. When MPM is applied to the current block, the prediction mode information may further include index information (e.g., mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be configured as an MPM candidate list or an MPM list. In addition, when MPM is not applied to the current block, the prediction mode information may further include remaining mode information (e.g., rem_inra_luma_pred_mode) indicating one of the remaining intra prediction modes other than the intra prediction mode candidates (MPM candidates). The decoding device may determine the intra prediction mode of the current block based on the prediction mode information. The prediction mode information may be encoded / decoded by the coding method described below. For example, the prediction mode information may be encoded / decoded by entropy coding (e.g., CABAC or CAVLC) based on a truncated (rice) binary code.
[0148] Determine the internal prediction mode
[0149] When intra-prediction is applied, the intra-prediction mode of the neighboring block can be used to determine the intra-prediction mode applied to the current block. For example, the decoding device can select one of the most probable mode (mpm) candidates derived based on the intra-prediction mode of the left block and the intra-prediction mode of the upper block of the current block based on the received mpm index, or can select one of the remaining intra-prediction modes not included in the mpm candidates based on the remaining intra-prediction mode information. The mpm index can be signaled in the form of an mpm_idx syntax element, and the remaining intra-prediction mode information can be signaled in the form of a rem_intra_luma_pred_mode syntax element. For example, the remaining intra-prediction mode information can indicate one of the remaining intra-prediction modes, which is not included in the mpm candidates among all intra-prediction modes and has been indexed in the order of the prediction mode numbers.
[0150] Fig.12 and Fig.13 is a view illustrating a prediction direction of an intra prediction mode according to an embodiment of the present disclosure.
[0151] refer to Fig.12 , the intra prediction mode may include two non-directional intra prediction modes and 33 directional intra prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode and a DC intra prediction mode, and the directional intra prediction mode may include intra prediction modes numbered 2 to 34. The plane intra prediction mode may be referred to as a plane mode, and the DC intra prediction mode may be referred to as a DC mode.
[0152] At the same time, in order to capture any edge directions present in natural videos, such as Fig.13 As illustrated in , the directional intra prediction mode can be extended from 33 to 65. In this case, the intra prediction mode may include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode and a DC intra prediction mode, and the directional intra prediction mode may include intra prediction modes numbered 2 to 66. Extended directional intra prediction can be applied to blocks of all sizes and can be applied to both luma components and chroma components.
[0153] refer to Fig.13 In an embodiment, the intra prediction mode may include 67 modes. The prediction direction according to each prediction mode index (or mode number) is as follows: Fig.13 As shown in FIG. Prediction mode indexes 0 and 1 represent planar mode and DC mode, respectively. Prediction mode indexes 2 to 66 represent Fig.13 The prediction directions of the angles divided from the lower left arrow to the upper right arrow are illustrated in FIG. Among the 65 angles, prediction modes 2, 18, 50, and 66 represent the horizontal diagonal direction, the horizontal direction, the vertical direction, and the vertical diagonal direction, respectively.
[0154] Alternatively, the intra prediction mode may include two non-directional intra prediction modes and 129 directional intra prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode and a DC intra prediction mode, and the directional intra prediction mode may include intra prediction modes numbered 2 to 130.
[0155] The prediction unit of the encoding device / decoding device may derive a reference sample from among neighboring reference samples of the current block according to an intra prediction mode of the current block, and may generate a prediction sample of the current block based on the reference sample.
[0156] For example, the prediction sample may be derived i) based on an average or interpolation of neighboring reference samples of the current block, and (ii) based on reference samples for the prediction sample existing in a specific (prediction) direction among the neighboring reference samples of the current block. Case (i) may be referred to as a non-directional mode or a non-angle mode, and case (ii) may be referred to as a directional mode or an angle mode. In addition, in an embodiment, multiple reference sample lines using one or more reference sample lines for intra prediction may be used for more accurate prediction.
[0157] According to an embodiment of the present disclosure, a method for configuring a most probable mode (MPM) is proposed to Fig.13The method described herein effectively reduces the overhead of signaling intra-prediction modes in an environment where the number of intra-prediction modes is increased compared to those implemented by conventional image compression techniques. According to an embodiment of the present disclosure, coding efficiency can be enhanced in terms of reducing signaling overhead by effectively coding intra-prediction modes. By reducing signaling overhead, better coding efficiency can be obtained in terms of BD rate / BD-PSNR.
[0158] On the encoder side, the best intra prediction mode is determined as the optimized prediction mode, taking into account both bit rate and distortion. Thereafter, the best (selected) prediction intra mode is encoded by the bitstream, and the decoder performs intra prediction using the best intra prediction mode parsed from the bitstream. However, as described above, the increased number of intra prediction modes requires efficient intra mode coding to minimize signaling overhead.
[0159] The MPM list (or MPM candidate list) may be configured using intra prediction modes of neighboring intra-coded blocks in the encoder and decoder. When the best prediction mode selected by the encoder is one of the prediction modes included in the MPM list, overhead may be minimized through MPM index signaling.
[0160] Fig.14 An example of neighboring blocks used as MPM candidates according to an embodiment of the present disclosure is illustrated.
[0161] In traditional video compression technology (e.g., HEVC), based on Fig.14 The three MPM lists are generated by using the two neighboring intra prediction modes of the positions F and G illustrated in FIG. When at least one of the following three conditions is met, F or G can be set to DC mode.
[0162] -When F or G is not available
[0163] - When F or G is not an internal compilation block
[0164] - When F or G is outside the current compilation tree unit (CTU) to which the current block belongs
[0165] In this disclosure, for the convenience of description, Fig.14 The intra prediction modes of the blocks in positions A to G illustrated in may be represented as A to G, respectively. That is, F or G refers to the intra prediction mode of the block at position F or G, respectively. If F and G are determined, three MPM lists may be configured based on the virtual codes of Table 2 below.
[0166] [Table 2]
[0167] if (if the inner patterns of F and G are equal) { if (if F's inner mode < inner mode 2) {MPM List 1} else {MPM List 2} } else { if (the internal modes of F and G are not internal mode planes) {MPM List 3} else if ((F+G)'s inner mode < inner mode 2) {MPM List 4} else {MPM List 5} }
[0168] Referring to Table 2, when F and G are the same, when F is less than 2 (or prediction mode 2, mode 2, or prediction mode 2), a first MPM list will be generated (or derived), and when F is not less than 2, a second MPM list may be generated.
[0169] In the case where F and G are not the same, when F or G is not a planar mode, a third MPM list may be generated, and when (F+G) is less than 2, a fourth list may be generated, otherwise, a fifth MPM list may be generated.
[0170] In an embodiment, the number of MPM candidates included in the MPM list may be set to be different depending on the number of intra prediction modes. Generally, as the number of intra prediction modes increases, the number of MPM candidates may increase. However, the present disclosure is not limited thereto, and for example, 35 and 67 intra modes may have various numbers (e.g., 3, 4, or 5) of MPM candidates depending on the design.
[0171] In addition, in an embodiment, when the MPM list is configured using 6 MPM candidates, the encoder / decoder may search for neighboring blocks at various positions, and may continuously perform pruning checks to exclude the same intra prediction mode. For example, the encoder / decoder may select Fig.14 The six-member MPM list is configured by sequentially searching the block D, block B, planar mode, DC mode, block E, block C, and block A illustrated in FIG. If the MPM list does not meet the maximum number of candidates, the adjacent angle mode and the predefined basic mode can be regarded as MPM candidates. Here, the adjacent angle mode represents a prediction mode adjacent to the prediction direction of the directional mode added to the MPM list. Such a search and pruning check can advantageously save bit rate, but may experience an increase in the number of hardware operation cycles for configuring the MPM list. In the worst scenario, when an image of size 3840x2160 (i.e., a 4K image) is divided into 4x4 blocks for intra-prediction of the 4K image, the increased hardware operation cycle for each 4x4 block may have a decisive impact on the throughput.
[0172] Furthermore, in one embodiment, if the intra prediction mode of a neighboring inter-coded block is known, the intra prediction mode may be used to configure the MPM list.
[0173] When 67 intra prediction modes are used for intra prediction, the 3 candidate MPM lists (or three MPM lists) used in conventional image compression techniques are not sufficient to indicate the increased diversity of intra prediction modes. In addition, the complexity of the 6-MPM list including a pruning check function and a function of searching all defined neighboring positions until a predetermined number of MPM candidates (or the length of the MPM list) is met is too high, which may greatly affect throughput.
[0174] Therefore, the present disclosure proposes a method for effectively configuring the MPM list to effectively balance complexity and compilation efficiency so as to alleviate such problems. As used herein, the term "disclosure" may be replaced by other terms, such as "document", "instruction manual" or "description".
[0175] Fig.15 is a flowchart illustrating a method of configuring an MPM list based on predefined conditions according to an embodiment of the present disclosure.
[0176] Although for ease of description reference Fig.15 The description mainly focuses on the decoder, and the method for configuring the MPM list according to the present embodiment can be applied to the encoder in substantially the same manner. In an embodiment of the present disclosure, the encoder / decoder can derive a valid MPM list by identifying predefined conditions.
[0177] Specifically, the decoder identifies whether a predetermined first condition is met (S1501). When the first condition is met, the decoder derives a first MPM list (S1502). If the first condition is not met, the decoder identifies whether a predefined second condition is met (S1503).
[0178] When the second condition is met, the decoder identifies whether a predefined third condition is met (S1504). When the third condition is met, the decoder derives a second MPM list (S1505). If the third condition is not met, the decoder identifies whether a predefined fourth condition is met (S1506). When the fourth condition is met, the decoder derives a third MPM list (S1507). If the fourth condition is not met, the decoder derives a fourth MPM list (S1508). If the second condition is not met, the decoder derives a fifth MPM list (S1509).
[0179] In an embodiment of the present disclosure, the decoder can derive five MPM lists (or five types of MPM lists) based on four predefined conditions. Fig.14The intra prediction mode of the neighboring block at the predefined position or the predefined basic (or default) intra prediction mode illustrated in can be used. The basic intra prediction mode represents an intra prediction mode that is statistically widely used. For example, the basic intra prediction mode may include a plane mode, a DC mode, a vertical mode, a horizontal mode, a horizontal diagonal mode, or a vertical diagonal mode. In other words, the intra prediction mode of the neighboring block and / or the basic intra prediction mode can be used to determine the first to fifth MPM lists according to their corresponding predefined conditions.
[0180] In addition, in an embodiment, k represents the length of the valid MPM list (or the number of MPM candidates) proposed in the present disclosure, and k can be set to various values, such as 3, 4, 5, and 6. As an example, k can be a value less than 6 (e.g., 5). In this case, before generating an MPM list (k-MPM list) including k candidates, a specific intra prediction mode can be considered as the highest candidate for intra prediction. In other words, the encoder / decoder can determine whether a specific intra prediction mode is applied to the current block, and if the specific intra prediction mode is not applied to the current block, a k-MPM list can be generated.
[0181] Fig.16 is a flowchart illustrating a method of configuring an MPM list according to a predefined condition by using a shared list according to an embodiment of the present disclosure.
[0182] Although for ease of description reference Fig.16 The description is mainly focused on the decoder, and the method for configuring the MPM list according to the present embodiment can be applied to the encoder in substantially the same manner. In an embodiment of the present disclosure, the encoder / decoder can derive a valid MPM list according to predefined conditions by using a shared list. The shared list is a list to be shared before the final MPM list is determined according to each condition, and may include at least one intra-prediction mode. In the present disclosure, the shared list may be referred to as a shared MPM list, a temporary list, or a temporary MPM list.
[0183] Specifically, the decoder identifies whether a predetermined first condition is met (S1601). When the first condition is met, the decoder derives a first MPM list (S1602). If the first condition is not met, the decoder identifies whether a predefined second condition is met (S1603).
[0184] When the second condition is met, the decoder derives the first shared list (S1604). The decoder identifies whether a predefined third condition is met (S1605). When the third condition is met, the decoder derives the second MPM list by adding (or inserting) the intra prediction mode to the first shared list (S1606).
[0185] If the third condition is not met, the decoder derives a second shared list (S1607). As an embodiment, the second shared list may be a list in which at least one intra prediction mode is added to the first shared list. The decoder identifies whether a predefined fourth condition is met (S1608). When the fourth condition is met, the decoder derives a third MPM list by adding the intra prediction mode to the second shared list (S1609). If the fourth condition is not met, the decoder derives a fourth MPM list by adding the intra prediction mode to the second shared list (S1610). If the second condition is not met, the decoder derives a fifth MPM list (S1611).
[0186] In an embodiment of the present disclosure, the decoder can derive 5 MPM lists (or 5 types of MPM lists) based on four predefined conditions. In this case, the decoder can use the following example: Fig.14 The intra prediction mode of the neighboring block at the predefined position or the predefined basic (or default) intra prediction mode exemplified in . The basic intra prediction mode represents an intra prediction mode that is statistically widely used. For example, the basic intra prediction mode may include a plane mode, a DC mode, a vertical mode, a horizontal mode, a horizontal diagonal mode, or a vertical diagonal mode. In other words, the intra prediction mode of the neighboring block and / or the basic intra prediction mode may be used to determine the first to fifth MPM lists according to their corresponding predefined conditions. In addition, the intra prediction mode in Fig.14 The first shared list and the second shared list are derived based on the intra prediction mode of the neighboring blocks in the predefined positions illustrated in FIG. 1 and / or the predefined basic (or default) intra prediction mode.
[0187] In the following, the combination Fig.16 The conditions, shared lists and final MPM lists described in the embodiments of the present invention are specific examples. Fig.16 The method proposed in is not limited to the following description Fig.17 .
[0188] Fig.17 is a flowchart illustrating a method of configuring an MPM list based on predefined conditions according to an embodiment of the present disclosure.
[0189] Although for ease of description reference Fig.17 The description is mainly focused on the decoder, and the method for configuring the MPM list according to the present embodiment can be applied to the encoder in substantially the same manner. In an embodiment of the present disclosure, the encoder / decoder can derive a valid MPM list according to predefined conditions by using a shared list.
[0190] Specifically, the decoder identifies whether the intra prediction mode (or the mode number and index of the prediction mode) of the left adjacent block and the upper adjacent block of the current block are both less than or equal to the DC mode (S1701). In the present disclosure, the intra prediction mode of the left adjacent block can be referred to as the left mode or L, and the intra prediction mode of the upper adjacent block can be referred to as the upper mode or A.
[0191] When both the left mode and the top mode are less than or equal to the DC mode, the decoder derives a first MPM list (S1702). The first MPM list may include a planar mode, a DC mode, a vertical mode, a horizontal mode, a mode having an index obtained by subtracting 4 from the vertical mode, and a mode having an index obtained by adding 4 to the vertical mode.
[0192] When neither the left mode nor the upper mode is less than or equal to the DC mode, the decoder identifies whether the left mode and the upper mode are different (S1703).
[0193] If the left mode and the top mode are different, the decoder derives a first sharing list (S1704). The first sharing list may include the left mode and the top mode.
[0194] The decoder identifies whether the left mode or the top mode is less than or equal to the DC mode (S1705). When the left mode or the top mode is less than or equal to the DC mode, the decoder derives a second MPM list by adding (or inserting) the remaining intra-prediction modes to the first shared list (S1706). In the present disclosure, the maximum mode represents a mode with a larger index value in the left mode or the top mode, and the minimum mode represents a mode with a smaller index value in the left mode or the top mode. The remaining intra-prediction modes may include a non-directional mode different from the minimum mode (i.e., a planar mode when the minimum mode is the DC mode, or a DC mode when the minimum mode is the planar mode), a mode obtained by subtracting 1 from the maximum mode, a mode obtained by adding 1 to the maximum mode, and a mode obtained by subtracting 2 from the maximum mode.
[0195] If the left mode or the top mode is not less than or equal to the DC mode, the decoder derives a second sharing list (S1707). As an embodiment, the second sharing list may be a list obtained by adding the plane mode and the DC mode to the first sharing list.
[0196] The decoder identifies whether the difference between the maximum value and the minimum value of the prediction mode index in the currently added list (i.e., the second shared list) is greater than or equal to 63 (S1708). When the condition of step S1708 is met, the decoder derives a third MPM list by adding a mode obtained by subtracting 1 from the maximum mode and a mode obtained by adding 1 to the minimum mode to the second shared list (S1709). If the condition of step S1708 is not met, the decoder derives a fourth MPM list by adding a mode obtained by subtracting 1 from the minimum mode and a mode obtained by adding 1 to the maximum mode to the second shared mode (S1710). When the left mode and the top mode are the same, the decoder derives a fifth MPM list (S1711). The fifth MPM list may include a left mode, a plane mode, a DC mode, a mode obtained by subtracting 1 from the left mode, a mode obtained by adding 1 to the left mode, and a mode obtained by subtracting 2 from the left mode.
[0197] In an embodiment, the left neighboring block of the current block used as an MPM candidate may be set to Fig.14 The neighboring block in position D, and the upper neighboring block can be set to Fig.14 The neighboring blocks in position B.
[0198] In addition, in the embodiment, a specific example of a method for generating a valid k-MPM list using a shared list is described. Assume that k is 6, and 5 MPM lists are configured using 4 predefined conditions. Fig.14 In , the inner modes in the neighboring positions B and D can be represented as L and A, respectively.
[0199] In the first step, the encoder / decoder can use the first condition of the following equation 1 to identify whether L and A are non-directional intra prediction modes. In this case, the non-directional intra prediction mode may include a plane mode and a DC mode. If the first condition is met, the encoder / decoder can perform the fifth step described below. Otherwise, the encoder / decoder can perform the next second step.
[0200] [Equation 1]
[0201] L<=DC_idx and A<=DC_idx
[0202] In the second step, the encoder / decoder may determine whether L and A are different intra-prediction modes from each other using the second condition of the following equation 2. If the second condition is met, the encoder / decoder may perform the next third step, otherwise the encoder / decoder may perform the ninth step described below.
[0203] [Equation 2]
[0204] L! =A
[0205] In the third step, the encoder / decoder may create a shared list as illustrated in Table 3 below.
[0206] [Table 3]
[0207]
[0208] Referring to Table 3, the maximum mode is represented as max_idx, and the minimum mode is represented as min_idx. In addition, the encoder / decoder can use the third condition of the following equation 3 to identify whether one of L or A is a non-directional mode. If the third condition is met, the encoder / decoder can perform the sixth step to be described below, otherwise perform the next fourth step.
[0209] [Equation 3]
[0210] L<=DC_idx or A<=DC_idx
[0211] In the fourth step, the encoder / decoder may create a shared list as illustrated in Table 4 below.
[0212] [Table 4]
[0213] mpm[2] = Planar_idx mpm[3] = DC_idx
[0214] Thereafter, the encoder / decoder may use the fourth condition of the following equation 4 to identify whether the difference between the maximum mode and the minimum mode is less than or equal to 63. If the fourth condition is satisfied, the encoder / decoder may perform the seventh step described below, otherwise, perform the eighth step described below.
[0215] [Equation 4]
[0216] mpm[max_idx]-mpm[min_idx]<=63
[0217] In the fifth step, the encoder / decoder may generate an MPM list as illustrated in Table 5 below.
[0218] [Table 5]
[0219] mpm[0]=L mpm[1]=!L mpm[2] = Vertical_idx mpm[3] = Horizontal_idx mpm[4] = Vertical_idx - 4 mpm[5] = Vertical_idx + 4
[0220] The encoder / decoder can generate the final MPM list by adding the remaining prediction modes to the shared list generated in the above steps.
[0221] Specifically, in the sixth step, the encoder / decoder may generate an MPM list as illustrated in Table 6 below.
[0222] [Table 6]
[0223] mpm[2]=]=! mpm[min_idx] mpm[3] = mpm[max_idx] - 1 mpm[4]=mpm[max_idx]+1 mpm[5]=mpm[max_idx]-2
[0224] In the seventh step, the encoder / decoder may generate an MPM list as illustrated in Table 7 below.
[0225] [Table 7]
[0226] mpm[4] = mpm[max_idx] - 1 mpm[5]=mpm[min_idx]+1
[0227] In the eighth step, the encoder / decoder may generate an MPM list as illustrated in Table 8 below.
[0228] [Table 8]
[0229] mpm[4]=mpm[min_idx]-1 mpm[5]=mpm[max_idx]+1
[0230] In the ninth step, the encoder / decoder may generate an MPM list as illustrated in Table 9 below.
[0231] [Table 9]
[0232] mpm[0]=L mpm[2] = Planar_idx mpm[3] = DC_idx mpm[3]=L-1 mpm[4]=L+1 mpm[5]=L-2
[0233] exist Fig.17 In the above-mentioned embodiments, a method of adding neighboring prediction modes as MPM candidates has been described. +1, -1, +2, and -2 modes are used to add neighboring prediction modes. However, in some cases, addition or subtraction may result in a non-directional mode that destroys the consistency of the neighboring intra-mode, or may exceed the maximum available intra-mode index. For example, the value obtained by subtracting 1 from prediction mode 2 becomes 1, which is the index of the DC mode. In addition, the mode obtained by adding 1 to prediction mode 66 exceeds the maximum intra-mode index value.
[0234] Therefore, to solve these problems, the present disclosure proposes a modular method for adding neighboring prediction modes. The encoder / decoder may apply the modular method of Equation 5 below to use prediction modes in the neighboring directions of +1, -1, and -2.
[0235] [Equation 5]
[0236] Internal mode -1: (internal mode + 62) % 65 + 2
[0237] Internal mode +1: (internal mode -1) % 65 + 2
[0238] Internal mode -2: (internal mode +61)%65+2
[0239] In Equation 5, % represents a remainder operator. The encoder / decoder can derive a -1 mode (i.e., a mode obtained by subtracting 1) by adding 62 to the intra prediction mode, dividing the result by 65, and then adding 2 to the remainder. In addition, the encoder / decoder can derive a +1 mode (i.e., a mode obtained by adding 1) by subtracting 1 from the intra prediction mode, dividing the result by 65, and then adding 2 to the remainder. In addition, the encoder / decoder can derive a -2 mode (i.e., a mode obtained by subtracting 2) by adding 61 to the intra prediction mode, dividing the result by 65, and then adding 2 to the remainder.
[0240] In an embodiment of the present disclosure, an intra-mode coding method with 6 MPMs can be used to accommodate an increased number of directional intra-modes. In this case, truncated unary binarization can be applied to the entropy coding of the 6 MPMs.
[0241] Specifically, the first three bins may be coded based on a context depending on the MPM mode associated with the currently signaled bin. The MPM mode may be classified into the following three categories: (a) whether the prediction mode is in the horizontal direction (i.e., whether the MPM mode is less than or equal to the diagonal direction), (b) whether the prediction mode is in the vertical direction (i.e., whether it is greater than the diagonal direction), and (c) whether the prediction mode is a non-directional mode (i.e., whether it is a planar mode or a DC mode).
[0242] Therefore, three contexts corresponding to the above three categories can be used to signal the MPM index.
[0243] The compilation of the remaining 61 non-MPMs is performed as follows. These 61 non-MPMs can be divided into two sets: a selected mode set; and a non-selected mode set. The selected mode set can include 16 modes. The rest (i.e., 45 modes) can be assigned to the non-selected mode set. The mode set to which the current mode belongs and the flag can be signaled in a bitstream. In addition, a code with a fixed length of 4 bits can be used to signal (or binarize) the prediction mode of the selected set, and a truncated unary binary code can be used to signal (or binarize) the prediction mode of the non-selected mode set. The selected mode set and the non-selected mode set can be generated by subsampling a total of 61 non-MPM modes using the following indexes.
[0244] - Selected pattern set = {0, 4, 8, 12, 16, 20, ..., 60}
[0245] - Non-selected pattern set = {1, 2, 3, 5, 6, 7, 9, 10, ..., 59}
[0246] Fig.18 is a flowchart illustrating a method of configuring an MPM list based on predefined conditions according to an embodiment of the present disclosure.
[0247] Although for ease of description reference Fig.18 The description is mainly focused on the decoder, and the method for configuring the MPM list according to the present embodiment can be applied to the encoder in substantially the same manner. In an embodiment of the present disclosure, the encoder / decoder can derive a valid MPM list according to predefined conditions by using a shared list.
[0248] Specifically, the decoder identifies whether the left mode and the above mode are the same (S1801). In the present disclosure, the intra prediction mode of the left neighboring block may be referred to as the left mode or L, and the intra prediction mode of the above neighboring block may be referred to as the above mode or A.
[0249] When the left mode and the upper mode are the same, the decoder identifies whether the left mode is greater than the DC mode (S1802). When the left mode is greater than the DC mode, the decoder derives a first MPM list (S1803). The first MPM list may include a left mode, a plane mode, a DC mode, a mode obtained by subtracting 1 from the left mode, a mode obtained by adding 1 to the left mode, and a mode obtained by subtracting 2 from the left mode.
[0250] If the left mode is not greater than the DC mode, the decoder derives a second MPM list (S1804). The second MPM list may include a left mode, a non-directional mode different from the left mode, a vertical mode, a horizontal mode, a mode having an index obtained by subtracting 4 from the vertical mode, and a mode having an index obtained by adding 4 to the vertical mode.
[0251] If the left mode and the top mode are different, the decoder derives a first sharing list (S1805). The first sharing list may include the left mode and the top mode.
[0252] The decoder identifies whether the left mode and the upper mode are greater than the DC mode (S1806). When the left mode and the upper mode are not greater than the DC mode, the decoder identifies whether the sum of the left mode and the upper mode is greater than or equal to 2 (S1807). If the sum of the left mode and the upper mode is less than 2, the decoder derives a second MPM list. When the sum of the left mode and the upper mode is greater than or equal to 2, the third MPM list is derived by adding (or inserting) the remaining intra-prediction modes to the first shared list (S1808). In the present disclosure, the maximum mode represents a mode with a larger index value in the left mode or the upper mode, and the minimum mode represents a mode with a smaller index value in the left mode or the upper mode. The remaining intra-prediction modes may include a non-directional mode different from the minimum mode (i.e., a planar mode when the minimum mode is a DC mode, or a DC mode when the minimum mode is a planar mode), a mode obtained by subtracting 1 from the maximum mode, a mode obtained by adding 1 to the maximum mode, and a mode obtained by subtracting 2 from the maximum mode.
[0253] When the left mode and the top mode are greater than the DC mode, the decoder derives a second sharing list (S1809). As an embodiment, the second sharing list may be a list obtained by adding the plane mode and the DC mode to the first sharing list.
[0254] The decoder identifies whether the difference between the minimum mode and the maximum mode is not 64 and not 1 (S1810). If the condition of step S1810 is not met, the decoder derives a fourth MPM list by adding a mode obtained by subtracting 2 from the maximum mode and a mode obtained by adding 2 to the maximum mode to the second shared list (S1811). If the condition of step S1810 is not met, the decoder derives a fifth MPM list by adding a mode obtained by subtracting 1 from the maximum mode and a mode obtained by adding 1 to the maximum mode to the second shared list (S1812).
[0255] In an embodiment, the left neighboring block of the current block used as an MPM candidate may be set to Fig.14 The neighboring block in position D, and the upper neighboring block can be set to Fig.14 The neighboring blocks in position B.
[0256] In addition, in the embodiment, a specific example of a method for generating a valid k-MPM list using a shared list is described. Assume that k is 6, and 5 predefined conditions are used to configure 5 MPM lists. Fig.14 In , the inner modes in the neighboring positions B and D can be represented as L and A, respectively.
[0257] In the first step, the encoder / decoder may identify whether L and A are the same using the first condition of the following equation 6. If the first condition is satisfied, the encoder / decoder may perform the next second step, and otherwise, perform the next third step.
[0258] [Equation 6]
[0259] L==A
[0260] In the second step, when L and A are the same, the encoder / decoder can use the second condition of the following equation 7 to identify whether L is greater than the DC mode. If the second condition is met, the encoder / decoder can perform the sixth step described below, otherwise, perform the seventh step described below.
[0261] [Equation 7]
[0262] L>DC_idx
[0263] In the third step, the encoder / decoder may create a shared list as illustrated in Table 10 below.
[0264] [Table 10]
[0265]
[0266] Referring to Table 10, the maximum mode is represented as max_idx, and the minimum mode is represented as min_idx. In addition, the encoder / decoder can use the third condition of the following equation 8 to identify whether one of L or A is a non-directional mode. If the third condition is met, the encoder / decoder can perform the next fourth step, otherwise the encoder / decoder can perform the fifth step described below.
[0267] [Equation 8]
[0268] L>DC_idx and A>DC_idx
[0269] In a fourth step, the encoder / decoder may create a shared list as illustrated in Table 11 below.
[0270] [Table 11]
[0271] mpm[2] = Planar_idx mpm[3] = DC_idx diff=mpm[max_idx]-mpm[min_idx]
[0272] Thereafter, the encoder / decoder may use the fourth condition of the following equation 9 to identify whether the difference between the maximum mode and the minimum mode is not 64 and not 1. If the fourth condition is satisfied, the encoder / decoder may perform the tenth step described below, otherwise, perform the ninth step described below.
[0273] [Equation 9]
[0274] diff!=64 and diff!=1
[0275] In the fifth step, the encoder / decoder may identify whether L and A are in a non-directional mode using the fifth condition of the following equation 10. If the fifth condition is satisfied, the encoder / decoder may perform the seventh step, otherwise, perform the eighth step.
[0276] [Equation 10]
[0277] L+A>=2
[0278] In the sixth step, the encoder / decoder may generate an MPM list as illustrated in Table 12 below.
[0279] [Table 12]
[0280] mpm[0]=L mpm[2] = Planar_idx mpm[3] = DC_idx mpm[3]=L-1 mpm[4]=L+1 mpm[5]=L-2
[0281] In the seventh step, the encoder / decoder may generate an MPM list as illustrated in Table 13 below.
[0282] [Table 13]
[0283] mpm[0]=L mpm[1]=!L mpm[2] = Vertical_idx mpm[3] = Horizontal_idx mpm[4] = Vertical_idx - 4 mpm[5] = Vertical_idx + 4
[0284] The encoder / decoder can generate the final MPM list by adding the remaining prediction modes to the shared list generated in the above steps.
[0285] Specifically, in the eighth step, the encoder / decoder may generate an MPM list as illustrated in Table 14 below.
[0286] [Table 14]
[0287] mpm[2] = ! mpm[min_idx] mpm[3] = mpm[max_idx] - 1 mpm[4]=mpm[max_idx]+1 mpm[5]=mpm[max_idx]-2
[0288] In the ninth step, the encoder / decoder may generate an MPM list as illustrated in Table 15 below.
[0289] [Table 15]
[0290] mpm[4] = mpm[max_idx] - 2 mpm[5]=mpm[max_idx]+2
[0291] In the tenth step, the encoder / decoder may generate an MPM list as illustrated in Table 16 below.
[0292] [Table 16]
[0293] mpm[4] = mpm[max_idx] - 1 mpm[5]=mpm[max_idx]+1
[0294] The above-described embodiments of the present disclosure may be implemented independently or in combination of one or more thereof.
[0295] Fig.19 is a flowchart illustrating a method of generating an intra prediction block according to an embodiment of the present disclosure.
[0296] refer to Fig.19 For the convenience of description, the following description mainly focuses on the decoder, but the present disclosure is not limited thereto, and the method of generating an intra prediction block according to an embodiment of the present disclosure can be performed in the same manner in the encoder and the decoder.
[0297] The decoder obtains a most probable mode (MPM) flag indicating whether the MPM is applied to the current block (S1901). Here, the MPM represents a mode in which the intra prediction mode of the current block is derived from intra prediction blocks around the current block.
[0298] When the MPM is applied to the current block, the decoder configures an MPM list based on whether the intra prediction modes of the left neighboring block and the upper neighboring block of the current block satisfy a predetermined condition (S1902).
[0299] The decoder obtains an MPM index indicating a specific intra prediction mode in the MPM list (S1903).
[0300] The decoder generates a prediction block of the current block using the intra prediction mode specified by the MPM index (S1904).
[0301] As described above, the left neighboring block may be set to a block including pixels horizontally adjacent to a lower left sample in the current block, and the upper neighboring block may be set to a block including pixels vertically adjacent to an upper right sample of the current block.
[0302] As described above, step S1902 may further include configuring one of a specific number of MPM lists based on four predefined conditions.
[0303] Specifically, as described above, step S1902 may include: when the first condition is met, configuring the first MPM list; when the second and third conditions are met but the first condition is not met, configuring the second MPM list; when the second and fourth conditions are met but the first and third conditions are not met, configuring the third MPM list; when the second condition is met but the first, third and fourth conditions are not met, configuring the fourth MPM list; and when the first and second conditions are not met, configuring the fifth MPM list.
[0304] As described above, the first condition may be set to whether the intra-prediction modes of the left neighboring block and the right neighboring block are less than or equal to the DC mode, the second condition may be set to whether the intra-prediction modes of the left neighboring block and the right neighboring block are unequal to each other, the third condition may be set to whether the intra-prediction mode of the left neighboring block or the right neighboring block is less than or equal to the DC mode, and the fourth condition may be set to whether the difference between the maximum mode indicating a mode with a larger mode index value of the intra-prediction mode of the left neighboring block and the right neighboring block and the minimum mode indicating a mode with a smaller mode index value of the intra-prediction mode of the left neighboring block and the right neighboring block is less than a predefined specific value.
[0305] Fig. 20 is a block diagram illustrating an example device for processing a video signal according to an embodiment of the present disclosure. Fig. 20 The video signal processing device may correspond to Figure 1 Encoding device or Figure 2 Decoding device. Fig. 20 The video signal processing equipment can be referenced by application Figures 1 to 19 The method described is used to process the video signal.
[0306] The video signal processing apparatus 2000 for processing a video signal may include a memory 2020 for storing the video signal and a processor 2010 coupled to the memory to process the video signal.
[0307] According to an embodiment of the present disclosure, the processor 2010 may be configured as at least one processing circuit for processing an image signal, and may execute instructions for encoding or decoding the image signal, thereby processing the image signal. In other words, the processor 2010 may encode the original image data or decode the encoded image signal by executing the above encoding or decoding method.
[0308] Fig.21 The video coding system to which the present disclosure is applied is illustrated.
[0309] The video coding system may include a source device and a receiving device. The source device may send coded video / image information or data to the receiving device in the form of a file or stream via a digital storage medium or via a network.
[0310] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device. The decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display. The display may be configured for each device or an external component.
[0311] The video source can obtain the video / image by the capture, synthesis or generation process of the video / image. The video source may include a video / image capture device and / or a video / image generation device. The video / image capture device may, for example, include one or more cameras, a video / image archive including previously captured videos / images, etc. The video / image generation device may, for example, include a computer, a tablet computer and a smart phone, and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer. In this case, the process of generating the relevant data may be replaced by a video / image capture process.
[0312] The encoding device can encode the input video / image. The encoding device can perform a series of processes for compression and coding efficiency, such as prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in the form of a bitstream.
[0313] The transmitter can send the encoded video / image information or data output in the form of a bit stream to the receiver of the receiving device in the form of a file or stream through a digital storage medium or through a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD and SSD. The transmitter may include elements for generating media files through a predefined file format, and may include elements for transmission on a broadcast / communication network. The receiver may extract the bit stream and send it to a decoding device.
[0314] The decoding device may decode a video / image by performing a series of processes such as dequantization, inverse transformation, and prediction corresponding to the operations of the encoding device.
[0315] The renderer can render the decoded video / image. The rendered video / image can be displayed by a display.
[0316] Fig. 22 The embodiment of the present disclosure is applied, and illustrates the structure of a content streaming system.
[0317] refer to Fig. 22 The content streaming system to which the present disclosure is applied may basically include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.
[0318] The encoding server is basically used to generate a bit stream by compressing the content input from a multimedia input device (such as a smartphone, camera, or camcorder) into digital data and sending the bit stream to a streaming server. For another example, if a multimedia input device such as a smartphone, camera, or camcorder directly generates a bit stream, the encoding server can be omitted.
[0319] The bitstream may be generated by applying the encoding method or bitstream generation method of the present disclosure. The stream server may temporarily store the bitstream in the process of sending or receiving the bitstream.
[0320] The streaming server sends multimedia data to the user device based on the user request through the web server. The web server acts as a medium to inform the user which services are provided. When the user requests the required service from the web server, the web server sends the request to the streaming server. The streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server is used to control the instructions / responses between the devices in the content streaming system.
[0321] The streaming server may receive content from a media storage and / or encoding server. For example, if the content is received from an encoding server, the streaming server may receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a given time.
[0322] Examples of user devices may include mobile phones, smart phones, laptop computers, terminals for digital broadcasting, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, tablet PCs, tablet PCs, ultrabooks, wearable devices (e.g., watch-type terminals (smart watches), glasses-type terminals (smart glasses), and head-mounted displays (HMDs)), digital televisions, desktop computers, and digital television signs.
[0323] The servers within the content streaming system can operate as distributed servers, in which case the data received from the servers can be distributed and processed.
[0324] As described above, the embodiments described in the present disclosure may be implemented and executed on a processor, a microprocessor, a controller or a chip. For example, the functional units illustrated in the drawings may be implemented and executed on a computer, a processor, a microprocessor, a controller or a chip.
[0325] In addition, the decoder and encoder to which the present disclosure is applied may be included in multimedia broadcast transmission and reception equipment, mobile communication terminals, home theater video equipment, digital theater video equipment, cameras for monitoring, video conversation equipment, real-time communication equipment such as video communication, mobile streaming equipment, storage media, portable cameras, video on demand (VoD) service providing equipment, over-the-top (OTT) video equipment, Internet streaming service providing equipment, three-dimensional (3D) video equipment, video phone equipment, and medical video equipment, and may be used to process video signals or data signals. For example, OTT video equipment may include game consoles, Blu-ray players, Internet access televisions, home theater systems, smart phones, tablet PCs, and digital video recorders (DVRs).
[0326] In addition, the processing method of the present disclosure can be generated in the form of a program executed by a computer, and can be stored in a computer-readable recording medium. Multimedia data with a data structure according to the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices in which computer-readable data is stored. Computer-readable recording media may include, for example, a Blu-ray disc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. In addition, computer-readable recording media include media implemented in the form of a carrier (e.g., transmission via the Internet). In addition, the bit stream generated using the encoding method can be stored in a computer-readable recording medium, or can be sent over a wired and wireless communication network.
[0327] In addition, the embodiments of the present disclosure may be implemented as a computer program product using program code. The program code may be executed by a computer according to the embodiments of the present disclosure. The program code may be stored on a computer-readable carrier.
[0328] In the aforementioned embodiments, the elements and features of the present disclosure have been combined in a particular form. Unless otherwise clearly described, each element or feature may be considered optional. Each element or feature may be implemented in a form not combined with other elements or features. In addition, some elements and / or features may be combined to form embodiments of the present disclosure. The order of the operations described in the embodiments of the present disclosure may be changed. Some elements or features of the embodiment may be included in another embodiment, or may be replaced by the corresponding elements or features of another embodiment. It is obvious that an embodiment may be constructed by combining claims that do not have a clear reference relationship in the claims, or may be included as a new claim by modification after submitting an application.
[0329] According to the embodiments of the present disclosure, various means may be used to implement, for example, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc. may be used to implement the embodiments of the present disclosure.
[0330] In the case of implementation by firmware or software, the embodiments of the present disclosure may be implemented in the form of modules, processes or functions for performing the aforementioned functions or operations. The software code may be stored in a memory and driven by a processor. The memory may be located inside or outside the processor and may exchange data with the processor in various known ways.
[0331] It is obvious to those skilled in the art that the present disclosure may be implemented in other specific forms without departing from the essential characteristics of the present disclosure. Therefore, the detailed description should not be interpreted as restrictive, but should be interpreted as illustrative in all aspects. The scope of the present disclosure should be determined by a reasonable analysis of the attached claims, and all changes within the equivalent range of the present disclosure are included in the scope of the present disclosure.
[0332] Industrial Applicability
[0333] The foregoing preferred embodiments of the present disclosure have been disclosed for illustrative purposes, and those skilled in the art may improve, change, substitute or add various other embodiments without departing from the technical spirit and scope of the present disclosure disclosed in the appended claims.
Claims
1. A method for decoding an image based on an intra-prediction mode, the method comprising: obtaining a most probable mode (MPM) flag notifying whether an MPM is applied to a current block, the MPM indicating a mode in which an intra prediction mode of the current block is derived from intra prediction blocks around the current block; When the MPM is applied to the current block, configuring the MPM list based on whether the intra prediction modes of the left neighboring block and the upper neighboring block of the current block meet a predefined condition; Obtaining an MPM index indicating a specific intra-prediction mode in the MPM list; as well as generating a prediction block of the current block based on the intra prediction mode specified by the MPM index, The configuring of the MPM list includes: deriving a first MPM list based on the intra prediction mode of the left neighboring block and the intra prediction mode of the upper neighboring block being a non-angular mode, and Among them, the first MPM list includes a horizontal intra prediction mode with a value of 18, a vertical intra prediction mode with a value of 50, an intra prediction mode with a value of 46 obtained by subtracting 4 from the value of the vertical intra prediction mode, and an intra prediction mode with a value of 54 obtained by adding 4 to the value of the vertical intra prediction mode.
2. The method according to claim 1, wherein: The left neighboring block is set to a block including pixels horizontally adjacent to the lower left sample in the current block, and The upper neighboring block is set to be a block including pixels vertically adjacent to the upper right sample in the current block.
3. The method according to claim 1, wherein: Configuring the MPM list includes configuring one of a specific number of MPM lists based on four predefined conditions.
4. The method according to claim 3, wherein: Configuring the MPM list includes: Based on satisfying the first condition, configuring a first MPM list; Based on satisfying the second condition and the third condition but not satisfying the first condition, configuring a second MPM list; configuring a third MPM list based on satisfying the second condition and the fourth condition but not satisfying the first condition and the third condition; configuring a fourth MPM list based on satisfying the second condition but not satisfying the first condition, the third condition, and the fourth condition; and Based on the fact that the first condition and the second condition are not satisfied, a fifth MPM list is configured.
5. The method according to claim 4, wherein: The first condition is set as whether the intra prediction mode of the left neighboring block and the upper neighboring block is less than or equal to the DC mode, The second condition is set to whether the intra prediction modes of the left neighboring block and the upper neighboring block are unequal to each other. The third condition is set to whether the intra prediction mode of the left neighboring block or the intra prediction mode of the upper neighboring block is less than or equal to the DC mode, and Among them, the fourth condition is set to whether the difference between the maximum mode indicating the mode with a larger mode index value among the internal prediction modes of the left neighboring block and the upper neighboring block and the minimum mode indicating the mode with a smaller mode index value among the internal prediction modes of the left neighboring block and the upper neighboring block is less than a predefined value.
6. A method for encoding an image based on an intra-prediction mode, the method comprising: signaling a most probable mode (MPM) flag notifying whether an MPM is applied to a current block, the MPM indicating a mode in which an intra prediction mode of the current block is derived from intra prediction blocks around the current block; When the MPM is applied to the current block, configuring an MPM list based on whether the intra prediction modes of the left neighboring block and the upper neighboring block of the current block meet a predefined condition; signaling an MPM index indicating a particular intra-prediction mode in the MPM list; as well as generating a prediction block of the current block based on the intra prediction mode specified by the MPM index, The configuring of the MPM list includes: deriving a first MPM list based on the intra prediction mode of the left neighboring block and the intra prediction mode of the upper neighboring block being a non-angular mode, and Among them, the first MPM list includes a horizontal intra prediction mode with a value of 18, a vertical intra prediction mode with a value of 50, an intra prediction mode with a value of 46 obtained by subtracting 4 from the value of the vertical intra prediction mode, and an intra prediction mode with a value of 54 obtained by adding 4 to the value of the vertical intra prediction mode.
7. The method according to claim 6, wherein: The left neighboring block is set to a block including pixels horizontally adjacent to the lower left sample in the current block, and The upper neighboring block is set to be a block including pixels vertically adjacent to the upper right sample in the current block.
8. The method according to claim 6, wherein: Configuring the MPM list includes configuring one of a specific number of MPM lists based on four predefined conditions.
9. The method according to claim 8, wherein: Configuring the MPM list includes: Based on satisfying the first condition, configuring a first MPM list; Based on satisfying the second condition and the third condition but not satisfying the first condition, configuring a second MPM list; configuring a third MPM list based on satisfying the second condition and the fourth condition but not satisfying the first condition and the third condition; configuring a fourth MPM list based on satisfying the second condition but not satisfying the first condition, the third condition, and the fourth condition; and Based on the fact that the first condition and the second condition are not satisfied, a fifth MPM list is configured.
10. The method according to claim 9, wherein: The first condition is set as whether the intra prediction mode of the left neighboring block and the upper neighboring block is less than or equal to the DC mode, The second condition is set to whether the intra prediction modes of the left neighboring block and the upper neighboring block are unequal to each other. The third condition is set to whether the intra prediction mode of the left neighboring block or the intra prediction mode of the upper neighboring block is less than or equal to the DC mode, and Among them, the fourth condition is set to whether the difference between the maximum mode indicating the mode with a larger mode index value among the internal prediction modes of the left neighboring block and the upper neighboring block and the minimum mode indicating the mode with a smaller mode index value among the internal prediction modes of the left neighboring block and the upper neighboring block is less than a predefined value.
11. A method for transmitting video data, the method comprising: generating a bitstream for a video, wherein the bitstream is generated based on signaling a most probable mode (MPM) flag that signals whether an MPM is applied to a current block, the MPM representing a mode in which an intra prediction mode of the current block is derived from intra prediction blocks around the current block; When the MPM is applied to the current block, configuring an MPM list based on whether the intra prediction modes of the left neighboring block and the upper neighboring block of the current block meet a predefined condition; signaling an MPM index indicating a particular intra-prediction mode in the MPM list; as well as generating a prediction block for the current block based on the intra prediction mode specified by the MPM index; as well as sending said data comprising said bit stream, The configuring the MPM list comprises: deriving a first MPM list based on that the intra prediction mode of the left neighboring block and the intra prediction mode of the upper neighboring block are non-angular modes, and Among them, the first MPM list includes a horizontal intra prediction mode with a value of 18, a vertical intra prediction mode with a value of 50, an intra prediction mode with a value of 46 obtained by subtracting 4 from the value of the vertical intra prediction mode, and an intra prediction mode with a value of 54 obtained by adding 4 to the value of the vertical intra prediction mode.
