Image / video coding method based on intra prediction, and device therefor
Patent Information
- Application Number
- IN202518122574
- Authority / Receiving Office
- IN · IN
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2018-03-30
- Filing Date
- 2025-12-05
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2039-03-27
AI Technical Summary
The increasing demand for high-resolution, high-quality images and videos, particularly in virtual and augmented reality, necessitates a more efficient image/video compression technique to reduce transmission and storage costs while accommodating diverse content characteristics.
An image/video coding method and apparatus that enhances coding efficiency through adaptive intra prediction, utilizing a Most Probable Mode (MPM) list to derive intra prediction modes without parsing an MPM flag for specific types, reducing data transmission and computational complexity.
This approach improves image/video compression efficiency by decreasing the amount of data required for residual processing and signaling, thereby lowering transmission costs and computational demands.
Smart Images

Figure 00000064_0000
Abstract
Description
BACKGROUND OF THE DISCLOSUREField of the Disclosure[1] The present disclosure relates to an image coding technology, and morespecifically, to an image / video coding method and apparatus based on intra prediction.Related Art[2] Recently, the demand for high resolution, high quality image / video such as4K, 8K or more Ultra High Definition (UHD) image / video is increasing in variousfields. As the image / video resolution or quality becomes higher, relatively moreamount of information or bits are transmitted than for conventional image / video data.Therefore, if image / video data are transmitted via a medium such as an existingwired / wireless broadband line or stored in a legacy storage medium, costs fortransmission and storage are readily increased.[3] Moreover, interests and demand are growing for virtual reality (VR) andartificial reality (AR) contents, and immersive media such as hologram; andbroadcasting of images / videos exhibiting image / video characteristics different fromthose of an actual image / video, such as game images / videos, are also growing.[4] Therefore, a highly efficient image / video compression technique is requiredto effectively compress and transmit, store, or play high resolution, high qualityimages / videos showing various characteristics as described above.SUMMARY[5] An object of the present disclosure is to provide a method and an apparatusfor enhancing image coding efficiency.[6] Another object of the present disclosure is to provide an efficient intraprediction method and apparatus.[7] Still another object of the present disclosure is to provide a method and anapparatus for efficiently signaling an intra prediction mode.[8] Yet another object of the present disclosure is to provide a method and anapparatus for adaptively deriving an intra prediction mode according to an intraprediction type.[9] An exemplary embodiment of the present disclosure provides an imagedecoding method performed by a decoding apparatus. The method includes: acquiringinformation about an intra prediction type of a current block from a bitstream, derivingthe intra prediction type of the current block based on the information about the intraprediction type, acquiring information about an intra prediction mode of the currentblock from the bitstream, configuring a most probable mode (MPM) list includingcandidate intra prediction modes of the current block based on neighboring blocks ofthe current block, deriving the intra prediction mode of the current block based on theMPM list and the information about the intra prediction mode, generating a predictedblock of the current block based on the intra prediction type and the intra predictionmode, acquiring residual information from the bitstream, generating a residual blockbased on the residual information, and generating a reconstructed block configuring areconstructed picture based on the predicted block and the residual block, in which ifthe intra prediction type represents a specific intra prediction type, the informationabout the intra prediction mode includes: an MPM index, and if the intra predictiontype represents the specific intra prediction type, the MPM index is parsed withoutparsing an MPM flag from the bitstream.
[10] Another exemplary embodiment of the present disclosure provides a decodingapparatus for performing an image decoding. The decoding apparatus includes: anentropy decoder for acquiring information about an intra prediction type, informationabout an inter prediction mode, and residual information of a current block from abitstream, a predictor for deriving the intra prediction type of the current block basedon the information about the intra prediction type, configuring a most probable mode(MPM) list including candidate intra prediction modes of the current block based onneighboring blocks of the current block, deriving an intra prediction mode of thecurrent block based on the MPM list and the information about the intra predictionmode, and generating a predicted block of the current block based on the intraprediction type and the intra prediction mode, and a residual processor for generatinga residual block based on the residual information, in which if the intra prediction typerepresents a specific intra prediction type, the information about the intra predictionmode includes: an MPM index, and if the intra prediction type represents the specificintra prediction type, the MPM index is parsed without parsing an MPM flag from thebitstream.
[11] Still another exemplary embodiment of the present disclosure provides animage encoding method performed by an encoding apparatus. The method includes:determining an intra prediction type of a current block, generating information aboutthe intra prediction type of the current block, configuring a most probable mode (MPM)list including candidate intra prediction modes of the current block based onneighboring blocks of the current block, deriving an intra prediction mode of thecurrent block based on the MPM list, generating a predicted block of the current blockbased on the intra prediction type and the intra prediction mode, generating predictedinformation of the current block, the predicted information including the informationabout the intra prediction type and the information about the intra prediction mode,deriving a residual block based on the predicted block, generating residual informationbased on the residual block, and outputting a bitstream by encoding image informationincluding the predicted information and the residual information, in which theinformation about the intra prediction mode is generated based on the intra predictiontype, if the intra prediction type represents a specific intra prediction type, theinformation about the intra prediction mode includes: an MPM index, and the intraprediction mode of the current block is limited to one of candidate intra predictionmodes within the MPM list.
[12] Yet exemplary embodiment of the present disclosure provides an imageencoding apparatus for performing an image encoding. The image encoding apparatusincludes: a predictor for determining an intra prediction type of a current block,generating information about the intra prediction type of the current block, configuringa most probable mode (MPM) list including candidate intra prediction modes of thecurrent block based on neighboring blocks of the current block, deriving an intraprediction mode of the current block based on the MPM list, generating a predictedblock of the current block based on the intra prediction type and the intra predictionmode, and generating prediction information of the current block, the predictioninformation including the information about the intra prediction type and theinformation about the intra prediction mode, a residual processor for deriving aresidual block based on the predicted block, and generating residual information basedon the residual block, and an entropy encoder for outputting a bitstream by encodingimage information including the prediction information and the residual information,in which the predictor generates the information about the intra prediction mode basedon the intra prediction type, if the intra prediction type represents a specific intraprediction type, the information about the intra prediction mode includes: an MPMindex, and the intra prediction mode of the current block is limited to one of thecandidate intra prediction modes within the MPM list.
[13] The present disclosure may enhance the overall image / video compressionefficiency.
[14] The present disclosure may decrease the amount of data to be transmittednecessary for the residual processing through the efficient prediction.
[15] The present disclosure may decrease the computational complexity andefficiently derive the intra prediction mode.
[16] The present disclosure may decrease the amount of data necessary forsignaling the intra prediction mode.BRIEF DESCRIPTION OF THE DRAWINGS
[17] FIG. 1 schematically illustrates an example of a video / image coding systemto which the present disclosure is applicable.
[18] FIG. 2 is a diagram for schematically explaining a configuration of avideo / image encoding apparatus to which the present disclosure is applicable.
[19] FIG. 3 is a diagram for schematically explaining a configuration of avideo / image decoding apparatus to which the present disclosure is applicable.
[20] FIG. 4 illustrates an example of neighboring reference samples which may beused for a case where intra prediction is performed.
[21] FIGS. 5 and 6 exemplarily illustrate intra prediction modes including 33directional prediction modes and 2 non-directional prediction modes.
[22] FIG. 7 exemplarily illustrates 65 intra directional modes of predictiondirections
[23] FIG. 8 exemplarily illustrates LIP.
[24] FIG. 9 illustrates an example of a method for generating a bottom rightneighboring sample of a current block.
[25] FIG. 10 illustrates an example of a method for generating bottom neighboringsamples and right neighboring samples using the bottom right neighboring sample.
[26] FIG. 11 illustrates an example of the MRL.
[27] FIG. 12 illustrates an example of the split according to the size of the currentblock (e.g., CU / CB).
[28] FIG. 13 illustrates an example of a method for determining an intra predictionmode in the encoding apparatus.
[29] FIG. 14 exemplarily illustrates a specific intra prediction mode to which theLIP is to be applied.
[30] FIG. 15 illustrates an example of various non-square blocks.
[31] FIG. 16 illustrates a method for signaling an intra prediction mode accordingto an exemplary embodiment of the present disclosure.
[32] FIG. 17 exemplarily illustrates an intra prediction mode derivation and intraprediction procedure based on an LIP type.
[33] FIG. 18 exemplarily illustrates an intra prediction mode derivation and intraprediction procedure based on an MRL intra prediction type.
[34] FIG. 19 exemplarily illustrates an intra prediction mode derivation and intraprediction procedure based on an ISP intra prediction type.
[35] FIG. 20 illustrates a method for signaling an intra prediction mode accordingto another exemplary embodiment of the present disclosure.
[36] FIGS. 21 and 22 schematically illustrate an example of a video / imageencoding method including the intra prediction method according to the exemplaryembodiment of the present disclosure and related components.
[37] FIGS. 23 and 24 schematically illustrate an example of the intra predictionmethod according to the exemplary embodiment of the present disclosure and relatedcomponents.DESCRIPTION OF EXEMPLARY EMBODIMENTS
[38] An exemplary embodiment of the present disclosure provides an imageinformation decoding method performed by a decoding apparatus. The methodincludes: decoding a non-separable secondary transform (NSST) index from abitstream, if an NSST is applied to a target block, decoding information abouttransform coefficients for the target block from the bitstream, based on the decodedNSST index, and deriving the transform coefficients for the target block based on thedecoded information about the transform coefficients, in which the NSST index isdecoded prior to the information about the transform coefficients for the target block.
[39] The present disclosure may be modified in various forms, and specificembodiments thereof will be described and illustrated in the drawings. However, theembodiments are not intended for limiting the disclosure. The terms used in thefollowing description are used to merely describe specific embodiments, but are notintended to limit the disclosure. An expression of a singular number includes anexpression of the plural number, so long as it is clearly read differently. The terms suchas "include" and "have" are intended to indicate that features, numbers, steps,operations, elements, components, or combinations thereof used in the followingdescription exist and it should be thus understood that the possibility of existence oraddition of one or more different features, numbers, steps, operations, elements,components, or combinations thereof is not excluded.
[40] Hereinafter, examples of the present embodiment will be described in detailwith reference to the accompanying drawings. In addition, like reference numerals areused to indicate like elements throughout the drawings, and the same descriptions onthe like elements will be omitted.
[41] This document relates to video / image coding. For example, themethods / embodiments disclosed in this document may be applied to a method whichis disclosed in a versatile video coding (VVC) standard, or a next generationvideo / image coding standard.
[42] This document suggests various embodiments of video / image coding, and theabove embodiments may also be performed in combination with each other unlessotherwise specified.
[43] In this document, a video may refer to a series of images over time. A picturegenerally refers to the unit representing one image at a particular time frame, and aslice / tile refers to the unit constituting a part of the picture in terms of coding. Aslice / tile may include one or more coding tree units (CTUs). One picture may consistof one or more slices / tiles. One picture may consist of one or more tile groups. Onetile group may include one or more tiles.
[44] A pixel or a pel may mean a smallest unit constituting one picture (or image).Also, 'sample' may be used as a term corresponding to a pixel. A sample may generallyrepresent a pixel or a value of a pixel, and may represent only a pixel / pixel value of aluma component or only a pixel / pixel value of a chroma component.
[45] A unit may represent a basic unit of image processing. The unit may includeat least one of a specific region of the picture and information related to the region.One unit may include one luma block and two chroma (ex. cb, cr) blocks. The unitmay be used interchangeably with terms such as block or area in some cases. In ageneral case, an MxN block may include samples (or sample arrays) or a set (or array)of transform coefficients of M columns and N rows.
[46] In this document, the term " / " and "," should be interpreted to indicate "and / or."For instance, the expression "A / B" may mean "A and / or B." Further, "A, B" maymean "A and / or B." Further, "A / B / C" may mean "at least one of A, B, and / or C."Also, "A / B / C" may mean "at least one of A, B, and / or C."
[47] Further, in the document, the term "or" should be interpreted to indicate"and / or." For instance, the expression "A or B" may comprise 1) only A, 2) only B,and / or 3) both A and B. In other words, the term "or" in this document should beinterpreted to indicate "additionally or alternatively."
[48] FIG. 1 illustrates an example of a video / image coding system to which thepresent disclosure may be applied.
[49] Referring to FIG. 1, a video / image coding system may include a source deviceand a reception device. The source device may transmit encoded video / imageinformation or data to the reception device through a digital storage medium ornetwork in the form of a file or streaming.
[50] The source device may include a video source, an encoding apparatus, and atransmitter. The receiving device may include a receiver, a decoding apparatus, and arenderer. The encoding apparatus may be called a video / image encoding apparatus,and the decoding apparatus may be called a video / image decoding apparatus. Thetransmitter may be included in the encoding apparatus. The receiver may be includedin the decoding apparatus. The renderer may include a display, and the display may beconfigured as a separate device or an external component.
[51] The video source may acquire video / image through a process of capturing,synthesizing, or generating the video / image. The video source may include avideo / image capture device and / or a video / image generating device. The video / imagecapture device may include, for example, one or more cameras, video / image archivesincluding previously captured video / images, and the like. The video / image generatingdevice may include, for example, computers, tablets and smart phones, and may(electronically) generate video / images. For example, a virtual video / image may begenerated through a computer or the like. In this case, the video / image capturingprocess may be replaced by a process of generating related data.
[52] The encoding apparatus may encode input video / image. The encodingapparatus may perform a series of procedures such as prediction, transform, andquantization for compaction and coding efficiency. The encoded data (encodedvideo / image information) may be output in the form of a bitstream.
[53] The transmitter may transmit the encoded image / image information or dataoutput in the form of a bitstream to the receiver of the receiving device through a digitalstorage medium or a network in the form of a file or streaming. The digital storagemedium may include various storage mediums such as USB, SD, CD, DVD, Blu-ray,HDD, SSD, and the like. The transmitter may include an element for generating amedia file through a predetermined file format and may include an element fortransmission through a broadcast / communication network. The receiver mayreceive / extract the bitstream and transmit the received bitstream to the decodingapparatus.
[54] The decoding apparatus may decode the video / image by performing a seriesof procedures such as dequantization, inverse transform, and prediction correspondingto the operation of the encoding apparatus.
[55] The renderer may render the decoded video / image. The rendered video / imagemay be displayed through the display.
[56] FIG. 2 schematically illustrates a configuration of a video / image encodingapparatus to which the present disclosure is applicable. Hereinafter, a video encodingapparatus may include an image encoding apparatus.
[57] Referring to FIG. 2, an encoding apparatus 200 may be configured to includean image partitioner 210, a subtractor 215, a transformer 220, a quantizer 230, adequantizer 240, an inverse transformer 250, an adder 255, a filter 260, a memory 270,an inter predictor 280, an intra predictor 285, and an entropy encoder 290. The interpredictor 280 and the intra predictor 285 may be collectively called a predictor. Thatis, the predictor may include the inter predictor 280 and the intra predictor 285. Thetransformer 220, the quantizer 230, the dequantizer 240, and the inverse transformer250 may be included in a residual processor. The residual processor may furtherinclude the subtractor 215. The aforementioned image partitioner 210, subtractor 215,transformer 220, quantizer 230, dequantizer 240, inverse transformer 250, adder 255,filter 260, inter predictor 280, intra predictor 285, and entropy encoder 290 may beconfigured by one hardware component (e.g., an encoder chipset or a processor)according to an exemplary embodiment. In addition, the memory 270 may include adecoded picture buffer (DPB), and may also be configured by a digital storage medium.The hardware component may further include the memory 270 as an internal / externalcomponent.
[58] The image partitioner 210 may split an input image (or, picture, frame) inputto the encoding apparatus 200 into one or more processing units. As an example, theprocessing unit may be called a coding unit (CU). In this case, the coding unit may berecursively split according to a Quad-tree binary-tree ternary-tree (QTBTTT) structurefrom a coding tree unit (CTU) or the largest coding unit (LCU). For example, onecoding unit may be split into a plurality of coding units of a deeper depth based on aquad-tree structure, a binary-tree structure, and / or a ternary-tree structure. In this case,for example, the quad-tree structure is first applied and the binary-tree structure and / orthe ternary-tree structure may be later applied. Alternatively, the binary-tree structuremay also be first applied. A coding procedure according to the present disclosure maybe performed based on a final coding unit which is not split any more. In this case,based on coding efficiency according to image characteristics or the like, the maximumcoding unit may be directly used as the final coding unit, or as necessary, the codingunit may be recursively split into coding units of a deeper depth, such that a codingunit having an optimal size may be used as the final coding unit. Here, the codingprocedure may include a procedure such as prediction, transform, and reconstructionto be described later. As another example, the processing unit may further include aprediction unit (PU) or a transform unit (TU). In this case, each of the prediction unitand the transform unit may be split or partitioned from the aforementioned final codingunit. The prediction unit may be a unit of sample prediction, and the transform unitmay be a unit for inducing a transform coefficient and / or a unit for inducing a residualsignal from the transform coefficient.
[59] The unit may be interchangeably used with the term such as a block or an areain some cases. Generally, an MxN block may represent samples composed of Mcolumns and N rows or a group of transform coefficients. The sample may generallyrepresent a pixel or a value of the pixel, and may also represent only the pixel / pixelvalue of a luma component, and also represent only the pixel / pixel value of a chromacomponent. The sample may be used as the term corresponding to a pixel or a pelconfiguring one picture (or image).
[60] The encoding apparatus 200 may generate a residual signal (residual block,residual sample array) by subtracting a predicted signal (predicted block, predictionsample array) output from the inter predictor 280 or the intra predictor 285 from theinput image signal (original block, original sample array), and the generated residualsignal is transmitted to the transformer 220. In this case, as illustrated, the unit forsubtracting the predicted signal (predicted block, prediction sample array) from theinput image signal (original block, original sample array) within an encoder 200 maybe called the subtractor 215. The predictor may perform prediction for a block to beprocessed (hereinafter, referred to as a current block), and generate a predicted blockincluding prediction samples of the current block. The predictor may determinewhether intra prediction is applied or inter prediction is applied in units of the currentblock or the CU. The predictor may generate various information about prediction,such as prediction mode information, to transfer the generated information to theentropy encoder 290 as described later in the description of each prediction mode. Theinformation about prediction may be encoded by the entropy encoder 290 to be outputin a form of the bitstream.
[61] The intra predictor 285 may predict a current block with reference to sampleswithin a current picture. The referenced samples may be located neighboring to thecurrent block, or may also be located away from the current block according to theprediction mode. The prediction modes in the intra prediction may include a pluralityof non-directional modes and a plurality of directional modes. The non-directionalmode may include, for example, a DC mode or a planar mode. The directional modemay include, for example, 33 directional prediction modes or 65 directional predictionmodes according to the fine degree of the prediction direction. However, this isillustrative and the directional prediction modes which are more or less than the abovenumber may be used according to the setting. The intra predictor 285 may alsodetermine the prediction mode applied to the current block using the prediction modeapplied to the neighboring block.
[62] The inter predictor 280 may induce a predicted block of the current blockbased on a reference block (reference sample array) specified by a motion vector on areference picture. At this time, in order to decrease the amount of motion informationtransmitted in the inter prediction mode, the motion information may be predicted inunits of a block, a sub-block, or a sample based on the correlation of the motioninformation between the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0 prediction, L1prediction, Bi prediction, or the like) information. In the case of the inter prediction,the neighboring block may include a spatial neighboring block existing within thecurrent picture and a temporal neighboring block existing in the reference picture. Thereference picture including the reference block and the reference picture including thetemporal neighboring block may also be the same as each other, and may also bedifferent from each other. The temporal neighboring block may be called the namesuch as a collocated reference block, a collocated CU (colCU), or the like, and thereference picture including the temporal neighboring block may also be called acollocated picture (colPic). For example, the inter predictor 280 may configure amotion information candidate list based on the neighboring blocks, and generateinformation indicating what candidate is used to derive the motion vector and / or thereference picture index of the current block. The inter prediction may be performedbased on various prediction modes, and for example, in the case of a skip mode and amerge mode, the inter predictor 280 may use the motion information of theneighboring block as the motion information of the current block. In the case of theskip mode, the residual signal may not be transmitted unlike the merge mode. A motionvector prediction (MVP) mode may indicate the motion vector of the current block byusing the motion vector of the neighboring block as a motion vector predictor, andsignaling a motion vector difference.
[63] The predictor may generate a predicted signal based on various predictionmethods to be described later. For example, the predictor may not only apply the intraprediction or the inter prediction for predicting one block, but also simultaneouslyapply the intra prediction and the inter prediction. This may be called a combined interand intra prediction (CIIP). In addition, the predictor may also perform an intra blockcopy (IBC) for predicting the block. The intra block copy may be used to code contentimage / video of a game or the like such as a screen content coding (SCC), for example.The IBC basically performs prediction within the current picture but may be performedsimilar to the inter prediction in that the reference block is derived within the currentpicture. That is, the IBC may use at least one of the inter prediction techniquesdescribed in this document.
[64] The predicted signal generated through the predictor (including the interpredictor 280 and / or the intra predictor 285) may be used to generate a reconstructedsignal or used to generate a residual signal. The transformer 220 may generatetransform coefficients by applying the transform technique to the residual signal. Forexample, the transform technique may include at least one of a discrete cosinetransform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform(KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT).Here, when the relationship information between pixels is illustrated as a graph, theGBT means the transform obtained from the graph. The CNT means the transformwhich is acquired based on a predicted signal generated by using all previouslyreconstructed pixels. In addition, the transform process may also be applied to a pixelblock having the same size of the square, and may also be applied to the block havinga variable size rather than the square.
[65] The quantizer 230 may quantize the transform coefficients to transmit thequantized transform coefficients to the entropy encoder 290, and the entropy encoder290 may encode the quantized signal (information about the quantized transformcoefficients) to the encoded quantized signal to the bitstream. The information aboutthe quantized transform coefficients may be called residual information. The quantizer230 may rearrange the quantized transform coefficients having a block form in a onedimensional vector form based on a coefficient scan order, and also generate theinformation about the quantized transform coefficients based on the quantizedtransform coefficients of the one dimensional vector form. The entropy encoder 290may perform various encoding methods, for example, such as an exponential Golombcoding, a context-adaptive variable length coding (CAVLC), and a context-adaptivebinary arithmetic coding (CABAC). The entropy encoder 290 may also encodeinformation (e.g., values of syntax elements and the like) necessary for reconstructingvideo / image other than the quantized transform coefficients together or separately. Theencoded information (e.g., encoded video / image information) may be transmitted orstored in units of network abstraction layer (NAL) unit in a form of the bitstream. Thevideo / image information may further include information about various parameter setssuch as an adaptation parameter set (APS), a picture parameter set (PPS), a sequenceparameter set (SPS), or a video parameter set (VPS). In addition, the video / imageinformation may further include general constraint information. Thesignaled / transmitted information and / or syntax elements to be described later in thisdocument may be encoded through the aforementioned encoding procedure and thusincluded in the bitstream. The bitstream may be transmitted through a network, orstored in a digital storage medium. Here, the network may include a broadcastingnetwork and / or a communication network, or the like, and the digital storage mediummay include various storage media such as USB, SD, CD, DVD, Blue-ray, HDD, andSSD. A transmitter (not illustrated) for transmitting the signal output from the entropyencoder 290 and / or a storage (not illustrated) for storing the signal may be configuredas the internal / external elements of the encoding apparatus 200, or the transmitter mayalso be included in the entropy encoder 290.
[66] The quantized transform coefficients output from the quantizer 230 may beused to generate a predicted signal. For example, the dequantizer 240 and the inversetransformer 250 apply dequantization and inverse transform to the quantized transformcoefficients, such that the residual signal (residual block or residual samples) may bereconstructed. The adder 255 adds the reconstructed residual signal to the predictedsignal output from the inter predictor 280 or the intra predictor 285, such that thereconstructed signal (reconstructed picture, reconstructed block, reconstructed samplearray) may be generated. As in the case where the skip mode is applied, if there is noresidual for the block to be processed, the predicted block may be used as thereconstructed block. The adder 255 may be called a reconstructor or a reconstructedblock generator. The generated reconstructed signal may be used for the intraprediction of the next block to be processed within the current picture, and as describedlater, also used for the inter prediction of the next picture through filtering.
[67] Meanwhile, a luma mapping with chroma scaling (LMCS) may also beapplied in a picture encoding and / or reconstruction process.
[68] The filter 260 may apply filtering to the reconstructed signal, therebyimproving subjective / objective image qualities. For example, the filter 260 may applyvarious filtering methods to the reconstructed picture to generate a modifiedreconstructed picture, and store the modified reconstructed picture in the memory 270,specifically, the DPB of the memory 270. Various filtering methods may include, forexample, a deblocking filtering, a sample adaptive offset, an adaptive loop filter, abilateral filter, and the like. The filter 260 may generate various filtering-relatedinformation to transfer the generated information to the entropy encoder 290, asdescribed later in the description of each filtering method. The filtering-relatedinformation may be encoded by the entropy encoder 290 to be output in a form of thebitstream.
[69] The modified reconstructed picture transmitted to the memory 270 may beused as the reference picture in the inter predictor 280. If the inter prediction is appliedby the inter predictor, the encoding apparatus may avoid the prediction mismatchbetween the encoding apparatus 200 and the decoding apparatus, and also improvecoding efficiency.
[70] The DPB of the memory 270 may store the modified reconstructed picture tobe used as the reference picture in the inter predictor 280. The memory 270 may storemotion information of the block in which the motion information within the currentpicture is derived (or encoded) and / or motion information of the blocks within thepreviously reconstructed picture. The stored motion information may be transferred tothe inter predictor 280 to be utilized as motion information of the spatial neighboringblock or motion information of the temporal neighboring block. The memory 270 maystore the reconstructed samples of the reconstructed blocks within the current picture,and transfer the reconstructed samples to the intra predictor 285.
[71] FIG. 3 is a diagram for schematically explaining a configuration of avideo / image decoding apparatus to which the present disclosure is applicable.
[72] Referring to FIG. 3, a decoding apparatus 300 may be configured to includean entropy decoder 310, a dequantizer 320, an inverse transformer 330, an adder 335,a filter 340, a memory 350, an inter predictor 360, and an intra predictor 365. The interpredictor 360 and the intra predictor 365 may be collectively called a predictor. Thatis, the predictor may include the inter predictor 360 and the intra predictor 365. Thedequantizer 320 and the inverse transformer 330 may be collectively called a residualprocessor. That is, the residual processor may include the dequantizer 320 and theinverse transformer 330. The aforementioned entropy decoder 310, dequantizer 320,inverse transformer 330, adder 335, filter 340, inter predictor 360, and intra predictor365 may be configured by one hardware component (e.g., a decoder chipset or aprocessor) according to the exemplary embodiment. In addition, the memory 350 mayinclude the decoded picture buffer (DPB), and may also be configured by a digitalstorage medium. The hardware component may further include the memory 350 as aninternal / external component.
[73] When the bitstream including the video / image information is input, thedecoding apparatus 300 may reconstruct the image in response to a process in whichthe video / image information is processed in the encoding apparatus illustrated in FIG.2. For example, the decoding apparatus 300 may derive the units / blocks based on blocksplit-related information acquired from the bitstream. The decoding apparatus 300 mayperform decoding using the processing unit applied to the encoding apparatus.Therefore, the processing unit for the decoding may be, for example, a coding unit,and the coding unit may be split according to the quad-tree structure, the binary-treestructure, and / or the ternary-tree structure from the coding tree unit or the maximumcoding unit. One or more transform units may be derived from the coding unit. Inaddition, the reconstructed image signal decoded and output through the decodingapparatus 300 may be reproduced through a reproducing apparatus.
[74] The decoding apparatus 300 may receive the signal output from the encodingapparatus illustrated in FIG. 2 in a form of the bitstream, and the received signal maybe decoded through the entropy decoder 310. For example, the entropy decoder 310may derive information (e.g., video / image information) necessary for the imagereconstruction (or picture reconstruction) by parsing the bitstream. The video / imageinformation may further include information about various parameter sets such as anadaptation parameter set (APS), a picture parameter set (PPS), a sequence parameterset (SPS), and a video parameter set (VPS). In addition, the video / image informationmay further include general constraint information. The decoding apparatus maydecode the picture further based on the information about the parameter set and / or thegeneral constraint information. The signaled / received information and / or syntaxelements to be described later in this document may be decoded through the decodingprocedure and acquired from the bitstream. For example, the entropy decoder 310 maydecode information within the bitstream based on a coding method such as anexponential Golomb coding, a CAVLC, or a CABAC, and output a value of the syntaxelement necessary for the image reconstruction, and the quantized values of theresidual-related transform coefficient. More specifically, the CABAC entropydecoding method may receive a bin corresponding to each syntax element from thebitstream, determine a context model using syntax element information to be decodedand decoding information of the neighboring block and the block to be decoded orinformation of the symbol / bin decoded in the previous stage, and generate a symbolcorresponding to a value of each syntax element by predicting the probability ofgeneration of the bin according to the determined context model to perform thearithmetic decoding of the bin. At this time, the CABAC entropy decoding methodmay determine the context model and then update the context model using theinformation of the decoded symbol / bin for a context model of a next symbol / bin. Theinformation about prediction among the information decoded by the entropy decoder310 may be provided to the predictor (the inter predictor 360 and the intra predictor365), and a residual value at which the entropy decoding is performed by the entropydecoder 310, that is, the quantized transform coefficients and the related parameterinformation may be input to the dequantizer 320. In addition, the information aboutfiltering among the information decoded by the entropy decoder 310 may be providedto the filter 340. Meanwhile, a receiver (not illustrated) for receiving the signal outputfrom the encoding apparatus may be further configured as the internal / external elementof the decoding apparatus 300, or the receiver may also be a component of the entropydecoder 310. Meanwhile, the decoding apparatus according to this document may becalled a video / image / picture decoding apparatus, and the decoding apparatus may alsobe classified into an information decoder (video / image / picture information decoder)and a sample decoder (video / image / picture sample decoder). The information decodermay include the entropy decoder 310, and the sample decoder may include at least oneof the dequantizer 320, the inverse transformer 330, the adder 335, the filter 340, thememory 350, the inter predictor 360, and the intra predictor 365.
[75] The dequantizer 320 may dequantize the quantized transform coefficients tooutput the transform coefficients. The dequantizer 320 may rearrange the quantizedtransform coefficients in a two-dimensional block form. In this case, the rearrangementmay be performed based on a coefficient scan order performed by the encodingapparatus. The dequantizer 320 may perform dequantization for the quantizedtransform coefficients using a quantization parameter (e.g., quantization step sizeinformation), and acquire the transform coefficients.
[76] The inverse transformer 330 inversely transforms the transform coefficientsto acquire the residual signal (residual block, residual sample array).
[77] The predictor may perform the prediction of the current block, and generate apredicted block including the prediction samples of the current block. The predictormay determine whether the intra prediction is applied or the inter prediction is appliedto the current block based on the information about prediction output from the entropydecoder 310, and determine a specific intra / inter prediction mode.
[78] The predictor may generate the predicted signal based on various predictionmethods to be described later. For example, the predictor may not only apply the intraprediction or the inter prediction for the prediction of one block, but also apply theintra prediction and the inter prediction at the same time. This may be called acombined inter and intra prediction (CIIP). In addition, the predictor may also performan intra block copy (IBC) for the prediction of the block. The intra block copy may beused for the content image / video coding of a game or the like, such as a screen contentcoding (SCC). The IBC may basically perform the prediction within the current picturebut be performed similar to the inter prediction in that the reference block is derivedwithin the current picture. That is, the IBC may use at least one of the inter predictiontechniques described in this document.
[79] The intra predictor 365 may predict the current block with reference to thesamples within the current picture. The referenced samples may be locatedneighboring to the current block according to the prediction mode, or may also belocated away from the current block. The prediction modes in the intra prediction mayinclude a plurality of non-directional modes and a plurality of directional modes. Theintra predictor 365 may also determine the prediction mode applied to the current blockusing the prediction mode applied to the neighboring block.
[80] The inter predictor 360 may induce the predicted block of the current blockbased on the reference block (reference sample array) specified by the motion vectoron the reference picture. At this time, in order to decrease the amount of the motioninformation transmitted in the inter prediction mode, the motion information may bepredicted in units of a block, a sub-block, or a sample based on the correlation of themotion information between the neighboring block and the current block. The motioninformation may include a motion vector and a reference picture index. The motioninformation may further include inter prediction direction (L0 prediction, L1prediction, Bi prediction, or the like) information. In the case of the inter prediction,the neighboring block may include a spatial neighboring block existing within thecurrent picture and a temporal neighboring block existing in the reference picture. Forexample, the inter predictor 360 may configure a motion information candidate listbased on the neighboring blocks, and derive the motion vector and / or the referencepicture index of the current block based on received candidate selection information.The inter prediction may be performed based on various prediction modes, and theinformation about the prediction may include information indicating the mode of theinter prediction of the current block.
[81] The adder 335 may add the acquired residual signal to the predicted signal(predicted block, prediction sample array) output from the predictor (including theinter predictor 360 and / or the intra predictor 365) to generate the reconstructed signal(reconstructed picture, reconstructed block, reconstructed sample array). As in the casewhere the skip mode is applied, if there is no residual for the block to be processed,the predicted block may be used as the reconstructed block.
[82] The adder 335 may be called a reconstructor or a reconstructed blockgenerator. The generated reconstructed signal may be used for the intra prediction ofa next block to be processed within the current picture, and as described later, may alsobe output through filtering or may also be used for the inter prediction of a next picture.
[83] Meanwhile, a luma mapping with chroma scaling (LMCS) may also beapplied in the picture decoding process.
[84] The filter 340 may apply filtering to the reconstructed signal, therebyimproving the subjective / objective image qualities. For example, the filter 340 mayapply various filtering methods to the reconstructed picture to generate a modifiedreconstructed picture, and transmit the modified reconstructed picture to the memory350, specifically, the DPB of the memory 350. Various filtering methods may include,for example, a deblocking filtering, a sample adaptive offset, an adaptive loop filter, abidirectional filter, and the like.
[85] The (modified) reconstructed picture stored in the DPB of the memory 350may be used as the reference picture in the inter predictor 360. The memory 350 maystore motion information of the block in which the motion information within thecurrent picture is derived (decoded) and / or motion information of the blocks withinthe previously reconstructed picture. The stored motion information may betransferred to the inter predictor 360 to be utilized as motion information of the spatialneighboring block or motion information of the temporal neighboring block. Thememory 350 may store the reconstructed samples of the reconstructed blocks withinthe current picture, and transfer the stored reconstructed samples to the intra predictor365.
[86] In the present specification, the exemplary embodiments described in the filter260, the inter predictor 280, and the intra predictor 285 of the encoding apparatus 200may be applied equally to or to correspond to the filter 340, the inter predictor 360,and the intra predictor 365 of the decoding apparatus 300, respectively.
[87] Meanwhile, as described above, in performing video coding, prediction isperformed to improve compression efficiency. Through this, a predicted blockincluding prediction samples for a current block as a block to be coded (i.e., a codingtarget block) may be generated. Here, the predicted block includes prediction samplesin a spatial domain (or pixel domain). The predicted block is derived in the samemanner in an encoding apparatus and a decoding apparatus, and the encodingapparatus may signal information (residual information) on residual between theoriginal block and the predicted block, rather than an original sample value of anoriginal block, to the decoding apparatus, thereby increasing image coding efficiency.The decoding apparatus may derive a residual block including residual samples basedon the residual information, add the residual block and the predicted block to generatereconstructed blocks including reconstructed samples, and generate a reconstructedpicture including the reconstructed blocks.
[88] The residual information may be generated through a transform andquantization procedure. For example, the encoding apparatus may derive a residualblock between the original block and the predicted block, perform a transformprocedure on residual samples (residual sample array) included in the residual blockto derive transform coefficients, perform a quantization procedure on the transformcoefficients to derive quantized transform coefficients, and signal related residualinformation to the decoding apparatus (through a bit stream). Here, the residualinformation may include value information of the quantized transform coefficients,location information, a transform technique, a transform kernel, a quantizationparameter, and the like. The decoding apparatus may perform dequantization / inversetransform procedure based on the residual information and derive residual samples (orresidual blocks). The decoding apparatus may generate a reconstructed picture basedon the predicted block and the residual block. Also, for reference for inter predictionof a picture afterward, the encoding apparatus may also dequantize / inverse-transformthe quantized transform coefficients to derive a residual block and generate areconstructed picture based thereon.
[89] Meanwhile, if the intra prediction is applied to the current block, as describedabove, the encoding apparatus / decoding apparatus may derive the intra predictionmode of the current block, and derive the predicted block including prediction samplesof the current block based on the intra prediction mode. That is, the encodingapparatus / decoding apparatus may derive the prediction samples of the current blockby applying the intra prediction mode (e.g., directional mode or non-directional mode)based on neighboring reference samples of the current block.
[90] FIG. 4 illustrates an example of the neighboring reference samples which maybe used in performing the intra prediction.
[91] Referring to FIG. 4, as neighboring (reference) samples for the intra predictionof a current block 400, left neighboring samples p[-1][2H-1]...p[-1][0], top leftneighboring samples p[-1][-1], and top neighboring samples p[0][-1]...p[2W-1][-1]may be derived. Here, p[m][n] represents a sample (or pixel) of a sample position (m,n), which may represent a relative sample position, when considering the top leftsample position of the current block as (0, 0). In addition, here, W and H represent thewidth and height of the current block 400, respectively. The current block 400 maycorrespond to a coding unit (coding block) or a transform unit (transform block).
[92] Meanwhile, if there is a sample not available for the intra prediction amongthe neighboring samples p[-1][2H-1]...p[-1][-1]...p[2W-1][-1], the corresponding non-available sample may be filled with an available sample through a substitution orpadding procedure. In this case, for example, the non-available sample may besubstituted or padded with another neighboring sample adjacent to the correspondingsample.
[93] Here, as an example, if the corresponding sample is located outside the picture,the corresponding sample may be a non-available sample. For example, if the currentblock 400 is located on the edge of the picture, some of the neighboring samples maybe not available. As another example, if another CU including the correspondingsample is not coded yet, the corresponding sample may be a non-available sample.
[94] The substitution or padding procedure may be performed in the followingorder, for example.
[95] 1) If the neighboring samples p[-1][2H-1] are not available, search isperformed sequentially from the neighboring samples p[-1][2H-1] (or the neighboringsamples p[-1][2H-2] to p[-1][1]) to p[-1][-1], and then sequentially from p[0][-1] top[2W-1][-1], and a value of the available neighboring sample first found may beassigned to the neighboring sample p[-1][2H-1].
[96] 2) Search is performed sequentially from x = -1, y = 2H-2 to x = -1, y = -1,and if p[x][y] is not available, the value of the non-available p[x][y] may be substitutedwith a value of p[x][y+1].
[97] 3) Search is performed sequentially from x = 0, y = -1 to x = 2W-1, y = -1,and if p[x][y] is not available, the value of the non-available p[x][y] may be substitutedwith a value of p[x-1][y].
[98] If the intra prediction is applied, a prediction sample is derived using at leastone neighboring sample according to an intra prediction mode based on a current block.
[99] In this case, as an example, the intra prediction modes may include two nondirectional (or non-angular) intra prediction modes and 33 directional (or angular) intraprediction modes as follows.
[100] Table 1
[101] Here, a 0th intra prediction mode represents an intra planar mode, and a firstintra prediction mode represents an intra DC mode. Second to 34th intra predictionmodes represent an intra angular 2 mode... an intra angular 34 mode, respectively.
[102] Here, the intra planar mode and the intra DC mode are non-directionalprediction modes, and the intra angular 2 mode to the intra angular 34 mode aredirectional prediction modes.
[103] FIGS. 5 and 6 exemplarily illustrate intra prediction modes including 33directional prediction modes and two non-directional prediction modes.
[104] Referring to FIGS. 5 and 6, an intra prediction mode having horizontaldirectionality and an intra prediction mode having vertical directionality may beclassified with respect to a 18th intra prediction mode having a top left diagonalprediction direction. The numbers of -32 to 32 in FIG. 4 represent the vertical orhorizontal displacement of a unit of 1 / 32 on a sample grid position. Second to 17thintra prediction modes have horizontal directionality, and 18th to 34th intra predictionmodes have vertical directionality. 10th intra prediction mode and 26th intra predictionmode represent a horizontal intra prediction mode and a vertical intra prediction mode,respectively and may express the prediction direction of the angular intra mode withrespect to the 10th intra prediction mode and the 26th intra prediction mode as an angle.That is, a relative angle corresponding to each of the intra prediction modes may beexpressed with respect to a horizontal reference angle 0° corresponding to the 10thintra prediction mode, and a relative angle corresponding to each of the intra predictionmodes may be expressed with respect to a vertical reference angle 0° corresponding tothe 26th intra prediction mode.
[105] As another example, the intra prediction modes may include two nondirectional (or non-angular) intra prediction modes and 65 directional (or angular) intraprediction modes as follows. The non-directional intra prediction modes may includea planar intra prediction mode which is a 0th non-directional intra prediction mode,and a DC intra prediction mode which is a first non-directional intra prediction mode,and the directional intra prediction modes may include 65 intra prediction modeswhich are second to 66th directional intra prediction modes. However, this isillustrative and the present disclosure may also be applied to a case where the numberof intra prediction modes is different.
[106] Table 2
[107] FIG. 7 exemplarily illustrates intra directional modes of 65 predictiondirections.
[108] Referring to FIG. 7, an intra prediction mode having horizontal directionalityand an intra prediction mode having vertical directionality may be classified withrespect to a 34th intra prediction mode having a top left diagonal prediction direction.In FIG. 3 H and V mean horizontal directionality and vertical directionality,respectively, and the numbers of -32 to 32 represent the displacement of a unit of 1 / 32on the sample grid position. Second to 33th intra prediction modes have horizontaldirectionality, and 34th to 66th intra prediction modes have vertical directionality. 18thintra prediction mode and 50th intra prediction mode represent a horizontal intraprediction mode and a vertical intra prediction mode, respectively, and the second intraprediction mode may be called a bottom left diagonal intra prediction mode, the 34thintra prediction mode may be called a top left diagonal intra prediction mode, and the66th intra prediction mode may be called a top rightt diagonal intra prediction mode.
[109] Meanwhile, if the intra prediction is applied to the current block, an intraprediction mode applied to the current block may be derived based on the intraprediction mode of the neighboring block of the current block. For example, thedecoding apparatus may derive a most probable mode (MPM) list based on the intraprediction mode of the neighboring block (e.g., left neighboring block and / or topneighboring block) of the current block and additional candidate modes, and select oneof the MPM candidates within the derived MPM list based on a received MPM index,and alternatively, select one of the remaining intra prediction modes not included inthe MPM candidates based on remaining intra prediction mode information. Whetherthe intra prediction mode of the current block exists within the MPM list may beindicated based on an MPM flag.
[110] The MPM list may also be referred to as an intra prediction mode candidatelist, and may also be expressed as candModeList.
[111] Here, for example, the MPM list may include three MPM candidates or sixMPM candidates. As an example, the MPM list may include candidates derived basedon the intra prediction mode of the neighboring block, the derived intra predictionmode and / or a default intra prediction mode. The encoding apparatus / decodingapparatus may search for the neighboring blocks of the current block according to aspecific order, and derive the intra prediction mode of the neighboring block as theMPM candidate in the derived order. For example, the neighboring blocks may includea left neighboring block, a top neighboring block, a bottom left neighboring block, atop rightt neighboring block, and a top left neighboring block, and the encodingapparatus / decoding apparatus may search for the intra prediction mode of the leftneighboring block, the intra prediction mode of the top neighboring block, the planarintra prediction mode, the DC intra prediction mode, the intra prediction mode of thebottom left neighboring bock, the intra prediction mode of the top rightt neighboringblock, and the intra prediction mode of the top left neighboring block in the order toderive an MPM candidate and configure the MPM list of the current block. Meanwhile,after searching, if six MPM candidates are not derived, the MPM candidate may bederived based on the intra prediction mode derived as the MPM candidate. Forexample, if the intra prediction mode derived as the MPM candidate is an Nth intraprediction mode, the encoding apparatus / decoding apparatus may derive the N+1thintra prediction mode and / or the N-1th intra prediction mode as the MPM candidate ofthe current block. A detailed description of the neighboring blocks will be made later.
[112] In addition, for example, whether the intra prediction mode applied to thecurrent block is included in the MPM candidates, or included in the remaining intraprediction modes may be derived based on the MPM flag. Specifically, if the value ofthe MPM flag is 1, the MPM flag may represent that the intra prediction mode of thecurrent block is included in the MPM candidates (MPM list), and if the value of theMPM flag is 0, the MPM flag may represent that the intra prediction mode of thecurrent block is not included in the MPM candidates (MPM list) and included in theresidual intra prediction modes. Meanwhile, the MPM index may be signaled in a formof mpm_idx or intra_luma_mpm_idx syntax elements, and the remaining intraprediction mode information may be signaled in a form of rem_intra_luma_pred_modeor intra_luma_mpm_remainder syntax elements. In addition, the MPM flag may besignaled in a form of an intra_luma_mpm_flag syntax element, and if the MPM flag isnot signaled, the value of the MPM flag may also be regarded as 1. In addition, forexample, the remaining intra prediction mode information may index the remainingintra prediction modes not included in the MPM candidates among all intra predictionmodes in the order of the prediction mode number to indicate one among them. Theintra prediction mode may be an intra prediction mode for a luma component (sample).Hereinafter, the intra prediction mode information may include at least one of theMPM flag, the MPM index, and the remaining intra prediction mode information. Inaddition, the MPM list may also be called various terms such as MPM candidate listand candModeList.
[113] Meanwhile, the encoding apparatus may derive the MPM list of the currentblock based on the intra prediction mode of the neighboring block of the current blockand the additional candidate modes, determine the intra prediction mode of the currentblock, and encode the intra prediction mode information of the current block to storeand / or transmit the encoded intra prediction mode information.
[114] Meanwhile, various intra prediction methods may be applied for the intraprediction. As an example, the prediction sample may also be generated through theinterpolation between the second neighboring sample and the first neighboring samplelocated in the opposite direction of the prediction direction of the intra prediction modeof the current block with respect to the prediction sample of the current block amongthe neighboring reference samples. The aforementioned case may be called a linearinterpolation intra prediction (LIP) or a linear interpolation intra prediction. As anotherexample, the prediction sample may be derived using the reference sample located inthe prediction direction in the corresponding line by selecting a reference sample linehaving the highest prediction accuracy among neighboring multi-reference samplelines of the current block and at this time, the intra prediction coding may be performedin a method for indicating (signaling) the used reference sample line to the decodingapparatus. The aforementioned case may be called multi-reference line (MRL) intraprediction or MRL-based intra prediction. As still another example, the intra predictionis performed based on the same intra prediction mode by dividing the current blockinto the vertical or horizontal sub-partitions, and the neighboring reference samplesmay be derived and used in a unit of the sub-partition. That is, in this case, the intraprediction mode of the current block is equally applied to the sub-partitions, and theneighboring reference sample is derived and used in a unit of the sub-partition, therebyenhancing intra prediction performance in some cases. Such a prediction method maybe called an intra sub-partitions (ISP) or an ISP-based intra prediction. Theaforementioned intra prediction methods may be called an intra prediction typedifferently from the intra prediction mode. The intra prediction type may be calledvarious terms such as an intra prediction technique or an additional intra predictionmode. For example, the intra prediction type (or additional intra prediction mode orthe like) may include at least one of the aforementioned LIP, MRL intra prediction, orISP intra prediction. A general intra prediction method other than specific intraprediction types such as the LIP, MRL intra prediction, and ISP intra prediction maybe called a normal intra prediction type. The normal intra prediction type may begenerally applied if the aforementioned specific intra prediction type is not applied,and if the normal intra prediction type is applied, the intra prediction may be performedusing the aforementioned intra prediction mode and the reference sample line adjacentto the current block as illustrated in FIG. 3.
[115] The aforementioned LIP, MRL intra prediction, and ISP intra prediction maybe described in detail as follows.
[116] FIG. 8 exemplarily illustrates the LIP.
[117] Generally, since the intra prediction simply copies the neighboring referencesample in order to derive the prediction sample, there is a tendency of increasing anerror as a distance between the prediction sample and the reference sample is far away.The LIP may generate the prediction sample through the linear interpolation betweenthe first neighboring reference sample located in the intra prediction direction and thesecond neighboring reference sample located in the opposite direction of the intraprediction direction in order to decrease the error generated in the intra predictioncoding. To this end, a right buffer and a bottom buffer may be additionally generated.The right buffer may include the right neighboring reference samples of the currentblock, and the bottom buffer may include the bottom neighboring reference samplesof the current block. The right buffer or the bottom buffer may include the bottom rightneighboring reference sample of the current block. That is, the LIP may derive theright neighboring reference samples, the bottom right neighboring reference sample,and the bottom neighboring reference samples of the current block, and then performthe linear interpolation with the conventional reference samples (the left neighboringreference samples, the top left neighboring reference sample, and the top neighboringreference samples) to generate the prediction samples. Generally, since the blockswithin the picture are encoded / decoded in a raster scan order, the right block, thebottom block, and the bottom right block with respect to the current block to becurrently decoded are not decoded, such that the right neighboring reference samples,the bottom right neighboring reference samples, and the bottom neighboring referencesamples of the current block have not been reconstructed, and the right neighboringreference samples, the bottom right neighboring reference samples, and the bottomneighboring reference samples of the current block are required to be derived.
[118] That is, in order to perform the LIP, first, the neighboring reference samplesof the current block are required to be derived. In this case, the left neighboringreference samples, the top left neighboring reference sample, and the top neighboringreference samples may use the reconstructed samples of the corresponding locations,and if there are some non-available samples, the corresponding non-available samplemay be filled with an available sample through a substitution or padding procedure. Inthis case, for example, the non-available sample may be substituted or padded withanother neighboring reference sample adjacent to the corresponding sample.
[119] Meanwhile, since the right neighboring reference samples, the bottom rightneighboring reference sample, and the bottom neighboring reference samples amongthe neighboring reference samples are located in a block not decoded yet in thedecoding order, the reconstructed sample may not exist at the corresponding location,and the present disclosure may derive the right neighboring reference samples, thebottom right neighboring reference sample, and the bottom neighboring referencesamples through various methods. As an example, the right neighboring referencesamples, the bottom right neighboring reference sample, and the bottom neighboringreference samples among the neighboring reference samples may be generated usingthe previously derived top neighboring reference samples and left neighboringreference samples. In this case, the bottom right neighboring reference sample may befirst derived based on at least one of the top neighboring reference samples and at leastone of the left neighboring reference samples, and the bottom neighboring referencesamples and the right neighboring reference samples may be derived using the derivedbottom right neighboring reference sample, the left neighboring reference sample, andthe top neighboring reference sample. In this case, the bottom neighboring referencesamples and the right neighboring reference samples may be derived through the linearinterpolation according to the distance. Alternatively, values of the bottom neighboringreference samples and the right neighboring reference samples may also be derived bycopying values of the corresponding left neighboring reference samples and topneighboring reference samples according to the locations of the bottom neighboringreference samples and the right neighboring reference samples.
[120] Hereinafter, a method for generating the right neighboring reference samples,the bottom right neighboring reference samples, and the bottom neighboring referencesamples not reconstructed yet in the decoding order will be described. Although thepresent example is described based on a square block shape, the method according tothe present example may be applied thereto equally / similarly in order to enhance intraprediction efficiency even if the current block has a non-square block shape.Hereinafter, the neighboring reference sample may be called a neighboring sample.
[121] First, the bottom right neighboring reference sample of the current block maybe generated in the following method.
[122] FIG. 9 illustrates an example of a method for generating a bottom rightneighboring sample of the current block.
[123] Referring to (a) of FIG. 9, a bottom right neighboring sample BR may begenerated using a top rightt neighboring sample TR located on the top right corner ofthe current block, and a bottom left neighboring sample BL located on the bottom leftcorner of the current block. For example, when the top left sample position of thecurrent block is regarded as (0, 0), the top right neighboring sample TR may beexpressed as p[W][-1], the bottom left neighboring sample BL may be expressed asp[-1][H], and the bottom right neighboring sample BR may be expressed as p[W][H].Here, the W and H correspond to the width and height of the current block, respectively.
[124] Referring to (b) of FIG. 9, a bottom right neighboring sample BR may begenerated using a neighboring sample MTR located on the rightmost among the topneighboring samples of the current block and a neighboring sample MBL located onthe lowermost among the left neighboring samples thereof. For example, the topneighboring samples and the left neighboring samples may have the number by thenumber of W+H, respectively, and when the top left sample position of the currentblock is regarded as (0, 0), the neighboring sample MTR may be expressed asp[W+H][-1], the neighboring sample MBL may be expressed as p[-1][H+W], and thebottom right neighboring sample BR may be expressed as p[W][H].
[125] The bottom right neighboring sample BR may also be derived based on anaverage value as in the following equations, and alternatively, derived based on adistance ratio.
[126] Equation 1
[127] Equation 2
[128] Equation 3
[129] Equation 4
[130] Here, as described above, BR corresponds to the bottom right neighboringsample, TR corresponds to the top rightt neighboring sample located on the top righttcorner of the current block, BL corresponds to the bottom left neighboring samplelocated on the bottom left corner of the current block, MTR corresponds to theneighboring sample located on the rightmost among the top neighboring samples ofthe current block, MBL corresponds to the neighboring sample located on thelowermost among the left neighboring samples, and W and H correspond to the widthand height of the current block, respectively.
[131] Meanwhile, the bottom neighboring samples and the right neighboringsamples as well as the bottom right neighboring sample of the current block arerequired to be generated for the LIP. If the blocks are coded in the raster scan order, inthe coding time point of the current block, the bottom, bottom right, and right block ofthe current block have not been coded yet. Therefore, the bottom neighboring samplesand the right neighboring samples are required to be generated for the LIP. In this case,the bottom right neighboring sample derived according to the aforementioned methodmay be used. Specifically, the bottom neighboring samples and right neighboringsamples of the non-square current block may be generated using the derived bottomright neighboring sample, and the previously decoded top neighboring sample and leftneighboring sample.
[132] FIG. 10 illustrates an example of a method for generating the bottomneighboring samples and the right neighboring samples using the bottom rightneighboring sample.
[133] Referring to FIG. 10, the bottom neighboring samples may be generated bylinearly interpolating the bottom right neighboring sample BR of the current block andthe bottom left neighboring sample BL located on the bottom left corner of the currentblock according to the distance. In addition, the right neighboring samples may begenerated by linearly interpolating the bottom right neighboring sample BR of thecurrent block and the top rightt neighboring sample TR located on the top rightt cornerof the current block according to the distance. For example, if the top righttneighboring sample TR is expressed as p[W][-1], the bottom left neighboring sampleBL is expressed as p[-1][H], and the bottom right neighboring sample BR is expressedas p[W][H], the bottom neighboring samples may be expressed as p[0][H]...p[W-1][H],and the right neighboring samples may be expressed as p[W][0]...p[W][H-1]. In thiscase, for example, the bottom neighboring sample p[x][H] may be derived based onthe interpolation according to the distances of the p[-1][H] and the p[W][H] for thecorresponding sample location (x, H). In addition, for example, the right neighboringsample p[W][y] may be derived based on the interpolation according to the distancebetween the p[W][-1] and the p[W][H] for the corresponding sample location (W, y).Meanwhile, the bottom neighboring samples and the right neighboring samples mayalso be generated using the left neighboring samples and the top neighboring samplesof the current block without using the bottom right neighboring sample of the currentblock.
[134] After generating the bottom neighboring samples and the right neighboringsamples, the LIP may be performed using the generated bottom neighboring samplesand right neighboring samples. As in FIG. 8 described above, the method forgenerating a current prediction sample C using the LIP may be performed as follows,for example. FIG. 8 illustrates a prediction mode as a vertical series of mode havingpositive directionality as an example.
[135] 1) Left neighboring samples are copied to the lower end sample buffer and abottom buffer is generated using the generated bottom neighboring sample.
[136] 2) A prediction sample value P is generated by interpolating a referencesample A and a reference sample B of the top reference buffer using a reconstructedvalue.
[137] 3) A prediction sample P' is generated by interpolating a reference sample A'and a reference sample B' of the bottom reference buffer newly generated.
[138] 4) A final prediction value C is generated by linearly interpolating thegenerated P and P'.
[139] Equation 5
[140] A prediction sample value may be generated by applying the methods 2) to 4)to all samples within the current block. The LIP method may be applied to alldirectionality modes other than a planar mode and a DC mode whose directionality donot exist.
[141] Meanwhile, the MRL intra prediction may be performed as follows.
[142] The conventional intra prediction uses only the neighboring samples of thefirst top line and the neighboring samples of the first left line of the current block asthe reference samples for the intra prediction. However, the Multiple-reference line(MRL) method may perform the intra prediction using the neighboring sampleslocated in the sample line separated by one or three sample distances with respect tothe top and / or left of the current block as the reference samples.
[143] FIG. 11 illustrates an example of the MRL, and here, a multi-reference lineindex represents whether any line is used for the intra prediction with respect to thecurrent block. The multi-reference line index may be called an MRL index, andconfigured in a form of an intra_luma_ref_idx syntax element. If a value of the multireference line index is 0, it may be indicated that only the neighboring referencesamples of the conventional first line are used as the reference samples for the intraprediction, and the values of the multi-reference line index larger than 0 may indicatethat the neighboring reference samples of the lines other than the conventional firstline are used as the reference samples for the intra prediction. For example, the multi-reference line index may indicate the following intra reference sample line.
[144] Table 3
[145] That is, if the MRL intra prediction is applied to the current block, the intraprediction may be performed (derivation of the predicted block) based on the intraprediction mode of the current block and the neighboring reference samples of theneighboring reference sample line indicated by the value of the MRL index.
[146] Meanwhile, the ISP intra prediction may be performed as follows.
[147] The normal intra prediction performs prediction without split by regarding theblock to be currently coded as one processing unit. However, the ISP intra predictionmethod performs the intra prediction by splitting the block to be currently codedhorizontally or vertically. At this time, the reconstructed block is generated byperforming the encoding / decoding in a unit of the split block and (the reconstructedsamples of) the reconstructed block is used as the reference block (reference samples)of the block to be next split.
[148] Table 4 and FIG. 12 illustrate an example of the split according to the size ofthe current block (e.g., CU / CB).
[149] Table 4
[150] If the ISP intra prediction type is applied, an optimal mode may be generatedby generating an MPM list according to the respective split methods (horizontal splitand vertical split) in order to decrease coding complexity and comparing a suitableprediction mode among the prediction modes within the generated MPM list in viewof rate distortion optimization (RDO). In addition, if the intra prediction of theaforementioned multi-reference line (MRL) is used, the aforementioned intra subpartition method may be limited not to be used. That is, in this case, the intra subpartition method may be applied only in the case of using the first line (0th) referenceline (i.e., a value of intra_luma_ref_idx is 0).
[151] The ISP intra prediction method first transmits whether to apply the intra subpartition in a unit of block and if the current block uses the intra sub-partition(intra_subpartitions_mode_flag), information (intra_subpartitions_split_flag) aboutwhether it is horizontal split or vertical split may be signaled.
[152] If the ISP intra prediction method is applied, the intra prediction mode of thecurrent block is equally applied to the sub-partitions, and the neighboring referencesample may be derived in a unit of the sub-partition and used, thereby enhancing intraprediction performance. That is, if the intra sub-partition method is applied, a residualsample processing procedure is performed in a unit of sub-partition. That is, the intraprediction samples are derived for the respective sub-partitions, and a residual signal(residual samples) for the corresponding sub-partition is added to the intra predictionsamples, such that the reconstructed samples are acquired. The residual signal (residualsamples) may be derived through dequantization / inverse transform, or the like basedon residual information (quantized transform coefficient information or residualcoding syntax) within the aforementioned bit stream. That is, the prediction samplesand the residual samples for a first sub-partition may be derived, and the reconstructedsamples for the first sub-partition may be derived based on the derived predictionsamples and residual samples. In this case, when the prediction samples for a secondsub-partition are derived, some (e.g., the left or top neighboring reference samples ofthe second sub-partition) of the reconstructed samples within the first sub-partitionmay be used as the neighboring reference samples for the second sub-partition.Likewise, the prediction samples and the residual samples may be derived for thesecond sub-partition, and the reconstructed samples for the second sub-partition maybe derived based on the derived prediction samples and residual samples. In this case,when the prediction samples for a third sub-partition are derived, some (e.g., the leftor top neighboring reference samples of the third sub-partition) of the reconstructedsamples within the second sub-partition may be used as the neighboring referencesamples for the third sub-partition. The following is the same.
[153] Meanwhile, a number of intra prediction modes are currently considered forenhancing intra prediction performance, and the content in which the encodingapparatus sequentially performs the intra prediction for all intra prediction modes todetermine the optimal intra prediction mode based on the rate-distortion optimization(RDO) is not preferable in terms of computational complexity or a load. Therefore, inconsideration of such a point, the optimal intra prediction mode may be, for example,determined as follows.
[154] FIG. 13 illustrates an example of a method for determining the intra predictionmode in the encoding apparatus.
[155] Referring to FIG. 13, if the intra prediction is applied to the current block, acandidate even-numbered mode is first determined. For example, the candidate evennumbered mode for full rate-distortion (RD) may be determined through a rough modedecision method for the even-numbered modes. At this time, the rough mode decisionmethod determines a cost value based on a difference between a prediction block andan original block and a bit necessary for simply coding mode information anddetermines a mode with a small cost value as a candidate mode. Next, the rough modedecision method re-determines a candidate mode for the Full RD for odd-numberedmodes obtained by adding and subtracting 1 to and from the determined evennumbered mode (e.g., if the selected even-numbered mode is 20, the odd-numberedmode obtained by adding and subtracting 1 to and from the even-numbered mode area 19th mode and a 21th mode) through the rough mode decision method. Afterdetermining the candidate mode through the rough mode determination, the roughmode decision method searches for a similar mode neighboring to the current block(i.e., the intra prediction mode of the neighboring block) using the most probable mode(MPM) method and adds the similar mode to the candidate mode. Lastly, the roughmode decision method may determine an optimal intra prediction mode through theFull RD in view of the rate-distortion optimization (RDO).
[156] Meanwhile, whether the aforementioned specific prediction type (e.g., LIP,MRL, and / or ISP) is applied may be determined based on the size of the block and / orthe intra prediction mode. The following description is made based on the LIP but thisis illustrative and the following method may also be applied to the MRL, the SIP, andthe like.
[157] For example, a linear interpolation intra prediction (LIP) may be variablyapplied according to the size of the current block. Generally, the LIP is efficient for acase where the image with a complicated shape exists within the block. Therefore, itis efficient not to use a linear interpolation intra prediction in the block including asimple image area.
[158] In the case of the video / image coding, generally the block including thecomplicated image area is split and thus the size of the processing unit block isgradually decreased. On the other hand, the block including the simple image area isnot split and coded in units of the large-sized block. Therefore, whether the LIP isapplied may be variably determined based on the size of the block to be currentlycoded. At this time, whether the LIP is applied may be 1) determined in considerationof only the width of the current block, 2) determined in consideration of only the heightthereof, 3) determined in consideration of both the width and the height thereof, and 4)determined in consideration of the area (width x height) of the block as factors ofdetermining whether the LIP is applied.
[159] One example of defining a criterion of the size of the block for determiningwhether the LIP is applied is as follows. This example defines a reference length forapplying the LIP as 16. The 16 defined in this example may be variably determinedand any case of four cases below may be selected and used.
[160] 1) A case of determining whether the LIP is applied in consideration of onlythe width: if the width of the block < 16, the LIP is applied.
[161] 2) A case of determining whether the LIP is applied in consideration of onlythe height: if the height of the block < 16, the LIP is applied.
[162] 3) A case of determining whether the LIP is applied in consideration of boththe width and the height: if the width of the block < 16 and the height of the block <16, the LIP is applied.
[163] 4) A case of determining whether the LIP is applied in consideration of thearea of the block: if the area of the block < (16x16), the LIP is applied.
[164] Meanwhile, the LIP may also be variably applied according to the intraprediction mode of the current block. Generally, the vertical direction mode, thehorizontal direction mode, and the diagonal direction mode other than the planar modeand the DC mode have the high frequency of occurrence compared to other generalmodes. Therefore, the exemplary embodiment proposes a method in which the LIP isapplied to only a specific intra prediction mode having the high frequency ofoccurrence.
[165] The method for applying the LIP to the specific intra prediction modegenerally includes 1) a method for applying the LIP to only the specific mode havingthe high frequency of occurrence, and 2) a method for applying the LIP including themode neighboring to the specific mode having the high frequency of occurrence. Atthis time, the specific intra prediction mode to which the LIP is to be applied may alsobe defined as the vertical direction mode, the horizontal directional mode, and / or thediagonal direction mode, or the like in advance, and may also be adaptively signaled.Alternatively, the specific intra prediction mode may be adaptively derived inconsideration of the frequency of occurrence.
[166] FIG. 14 exemplarily illustrates a specific intra prediction mode to which theLIP is to be applied.
[167] As illustrated in FIG. 14, the linear interpolation intra prediction mode maybe applied to only the neighboring mode including the vertical direction mode and thehorizontal directional mode. FIG. 14 illustrates an example of applying the LIP to thevertical direction mode and neighboring four modes (from the vertical direction mode+ 2 modes to the vertical direction mode - 2 modes) in the vertical direction mode andthe LIP to the horizontal directional mode and neighboring four modes (from thehorizontal directional mode + 2 modes to the horizontal directional mode - 2 modes)in the horizontal directional mode. Although the example illustrated in FIG. 14 hasrepresented the vertical direction mode and the horizontal directional mode as theexample of the specific intra prediction mode, the specific intra prediction mode mayalso be selected as other different modes. In addition, although the example illustratedin FIG. 7 has limited a range in which the linear interpolation intra prediction mode isapplied up to ±2 modes neighboring to the specific intra prediction mode, this isillustrative, and the application range may be determined as ±n (n refers to a nonnegative integer) fixedly or variably.
[168] In addition, whether the LIP is applied may also be determined further basedon the shape of the current block. For example, the block having a non-square shapevariably applies the LIP. The current video coding may perform the coding using thenon-square blocks having various shapes in order to enhance coding efficiency.
[169] FIG. 15 illustrates an example of various non-square blocks.
[170] Referring to FIG. 15, a first non-square block (N / 4xN) or a third non-squareblock (N / 2xN) has a small width and a large height. In this case, a horizontal predictionmode for performing the prediction mainly using a left reference sample is used, andin this case, a distance between the prediction sample and the reference sample is short,thereby causing few prediction errors. Conversely, since the vertical direction modeperforms the prediction using a top reference sample, the distance between thereference sample and the prediction sample is long, thereby causing many predictionerrors. Therefore, the first non-square block (N / 4xN) does not apply the LIP to themode having the horizontal directionality and conversely, may apply the LIP to onlythe mode having the vertical directionality. If the same method is applied to a secondnon-square block (NxN / 4) and a fourth non-square block (NxN / 2), the non-squareblock having the long width and the short height applies the LIP to the mode havingthe horizontal directionality and does not apply the LIP to the mode having the verticaldirectionality. Table 5 expresses a method for applying the LIP based on thecomparison of the width and height of the current block, and the intra prediction mode,if the non-square block is the current block.
[171] Table 5
[172] In addition, the exemplary embodiment of the present disclosure maydetermine an optimal intra prediction mode in consideration of the aforementionedspecific intra prediction type (e.g., LIP, MRL, and / or ISP) and efficiently signal theoptimal intra prediction mode to the decoding apparatus. According to theaforementioned exemplary embodiment, the specific intra prediction type (e.g., LIP,MRL, and ISP) may be limitedly applied to only the candidate modes generated in theMPM list. That is, if the specific intra prediction type is applied, one of the candidatemodes within the MPM list may be limited to be used for the current block, and in thiscase, an MPM flag may be omitted from the information about the intra predictionmode, thereby decreasing the number of bits to be signaled. Considering that onepicture is split into a considerable number of blocks and coded, the decrease in thenumber of bits may enhance the overall coding efficiency.
[173] FIG. 16 illustrates a method for signaling the intra prediction mode accordingto the exemplary embodiment of the present disclosure.
[174] Referring to FIG. 16, the decoding apparatus acquires intra prediction modeinformation from a bitstream (S1600). As described above, the intra prediction modeinformation may include at least one of the MPM flag, the MPM index, and theremaining intra prediction mode information.
[175] The decoding apparatus configures an MPM list (S1610). The MPM list isconfigured to be the same as the MPM list configured in the encoding apparatus. Thatis, the MPM list may also include the intra prediction mode of the neighboring block,and may further include specific intra prediction modes derived according to theaforementioned method. The MPM list may also be variously configured according tothe intra prediction type, or the MPM list may also be configured regardless of theintra prediction type of the current block.
[176] Although it has been illustrated that the S1610 is performed after the S1600,this is illustrative, and the S1610 may also be performed prior to the S1600 and theS1610 and the S1600 may also be performed simultaneously.
[177] The decoding apparatus may determine the intra prediction mode of thecurrent block based on the MPM list and the intra prediction mode information.
[178] The decoding apparatus determines whether the specific intra prediction typeis applied to the current block (S1620). The specific intra prediction type may includethe aforementioned LIP type, MRL intra prediction type, and / or ISP prediction type.For example, the decoding apparatus may acquire the information about the intraprediction type from the bitstream.
[179] If the specific intra prediction type is applied, the decoding apparatus maydetermine that the intra prediction mode of the current block exists in the MPM list. Inthis case, the decoding apparatus may derive a candidate intra prediction mode withinthe MPM list indicated by an MPM index as the intra prediction mode of the currentblock by parsing and decoding the MPM index without parsing the MPM flag (S1630).That is, in this case, the information about the intra prediction mode of the currentblock does not include the MPM flag. In this case, the decoding apparatus mayestimate or regard a value of the MPM flag as 1 even without parsing the MPM flag,and the encoding apparatus may not encode the value of the MPM flag upon the intraprediction encoding as described above.
[180] Meanwhile, if the specific intra prediction type is not applied, that is, if anormal intra prediction is applied, the information about the intra prediction modeincludes the MPM flag, and in this case, the decoding apparatus confirms whether thevalue of the MPM flag is 1 (S1640).
[181] If the value of the MPM flag is 1, the decoding apparatus may derive thecandidate intra prediction mode within the MPM list indicated by the MPM index asthe intra prediction mode of the current block.
[182] Meanwhile, if the value of the MPM flag is 0, the decoding apparatus mayderive the intra prediction mode of the current block among the remaining intraprediction modes not included in the MPM list based on the remaining intra predictionmode information included in the information about the intra prediction mode (S1650).
[183] As described above, a predicted block may be generated based on the derivedintra prediction mode, and the reconstructed block / picture may be generated based onthe predicted block.
[184] Meanwhile, the intra prediction mode derivation and intra predictionprocedure according to each prediction type may be specifically performed, forexample, as follows.
[185] FIG. 17 exemplarily illustrates the intra prediction mode derivation and intraprediction procedure based on the LIP type.
[186] Referring to FIG. 17, the decoding apparatus determines whether the LIP isapplied to the current block. For example, the decoding apparatus may acquire an LIPflag from a bitstream, and if a value of the LIP flag is 1, the decoding apparatus maydetermine that the LIP is applied to the current block.
[187] If the LIP is applied to the current block, the decoding apparatus may derivethe intra prediction mode for the LIP from the MPM list by parsing the MPM indexfrom the bitstream. In this case, the decoding apparatus may directly parse the MPMindex without having to parse the MPM flag. The decoding apparatus may generatethe predicted block by performing the intra prediction based on the LIP type and theintra prediction mode.
[188] Meanwhile, if the LIP is not applied to the current block, the decodingapparatus may parse the MPM flag to the bitstream, and if the value of the MPM flagis 1, the decoding apparatus may derive the intra prediction mode for the normal intraprediction by parsing the MPM index. Meanwhile, if the value of the MPM flag is 0,the decoding apparatus may parse the remaining intra mode information from thebitstream, and derive the intra prediction mode of the current block among the intraprediction modes other than the candidate intra prediction modes included in the MPMlist based on the remaining intra mode information.
[189] The decoding apparatus may generate the predicted block by performing theintra prediction based on the intra prediction mode.
[190] FIG. 18 exemplarily illustrates the intra prediction mode derivation and intraprediction procedure based on an MRL intra prediction type.
[191] Referring to FIG. 18, the decoding apparatus determines whether the MRLintra prediction is applied to the current block. For example, the decoding apparatusmay acquire a multi-reference line index (e.g., intra_luma_ref_idx) from the bitstream,and determine that the MRL intra prediction is applied if a value of the multi-referenceline index is larger than 0. Meanwhile, if the value of the multi-reference line index is0, the decoding apparatus may determine that the normal intra prediction is applied.
[192] If the MRL intra prediction is applied to the current block, the decodingapparatus may derive the intra prediction mode for the MRL intra prediction from theMPM list by parsing the MPM index from the bitstream. In this case, the decodingapparatus may directly parse the MPM index without having to parse the MPM flag.The decoding apparatus may generate a predicted block by performing the intraprediction based on the MRL intra prediction type and the intra prediction mode.
[193] Meanwhile, if the MRL intra prediction is not applied to the current block, thedecoding apparatus may parse the MPM flag to the bitstream, and derive the intraprediction mode for the normal intra prediction by parsing the MPM index if the valueof the MPM flag is 1. Meanwhile, if the value of the MPM flag is 0, the decodingapparatus may parse the remaining intra mode information from the bitstream, andderive the intra prediction mode of the current block among the intra prediction modesother than the candidate intra prediction modes included in the MPM list based on theremaining intra mode information.
[194] The decoding apparatus may generate the predicted block by performing theintra prediction based on the intra prediction mode.
[195] FIG. 19 exemplarily illustrates the intra prediction mode derivation and intraprediction procedure based on an ISP intra prediction type.
[196] Referring to FIG. 19, the decoding apparatus determines whether an ISP intraprediction is applied to the current block. For example, the decoding apparatus mayacquire an ISP flag from the bitstream, and determine that the ISP intra prediction isapplied if a value of the ISP flag is 1. Meanwhile, if the value of the ISP flag is 0, thedecoding apparatus may determine that the ISP intra prediction is not applied.
[197] If the ISP intra prediction is applied to the current block, the decodingapparatus may derive the intra prediction mode for the ISP intra prediction from theMPM list by parsing the MPM index from the bitstream. In this case, the decodingapparatus may directly parse the MPM index without having to parse the MPM flag.The decoding apparatus may generate a predicted block by performing the intraprediction based on the ISP intra prediction type and the intra prediction mode.
[198] Meanwhile, if the ISP intra prediction is not applied to the current block, thedecoding apparatus may parse the MPM flag to the bitstream, and derive the intraprediction mode for the normal intra prediction by parsing the MPM index if the valueof the MPM flag is 1. Meanwhile, if the value of the MPM flag is 0, the decodingapparatus may parse the remaining intra mode information from the bitstream, andderive the intra prediction mode of the current block among the intra prediction modesother than the candidate intra prediction modes included in the MPM list based on theremaining intra mode information.
[199] The decoding apparatus may generate a predicted block by performing theintra prediction based on the intra prediction mode.
[200] The specific intra prediction type in which the signaling of the MPM flag isunnecessary may include the LIP, MRL intra prediction, and ISP intra prediction asdescribed above.
[201] As another example, as illustrated in FIG. 20, after the MPM flag is first parsed,whether the specific intra prediction type (LIP, MRL, ISP) is applied may also beconfirmed according to whether the value of the MPM flag is 1. In this case, the MPMflag may be signaled to all intra blocks, but when the MPM is not applied to the currentblock (MPM flag off), whether the specific intra prediction type is applied is notconsidered, such that in this case, there is an advantage in that the informationindicating the specific intra prediction type (e.g., the LIP flag, the MRL index, the ISPflag, or the like) may not be signaled / parsed.
[202] FIGS. 21 and 22 schematically illustrate an example of the video / imageencoding method including the intra prediction method according to the exemplaryembodiment of the present disclosure and related components. The method illustratedin FIG. 21 may be performed by the encoding apparatus illustrated in FIG. 2.Specifically, for example, S2100 to S2120 illustrated in FIG. 21 may be performed bythe predictor (specifically, the intra predictor 285) of the encoding apparatus, andS2130 illustrated in FIG. 21 may be performed by the entropy encoder 290 of theencoding apparatus. The method illustrated in FIG. 21 may include the aforementionedexemplary embodiments in this document.
[203] Referring to FIG. 21, the encoding apparatus determines an intra predictiontype of a current block (S2100). The encoding apparatus may determine whether aspecific intra prediction type in which the signaling of an MPM flag is unnecessary isapplied to the current block. As described above, the specific intra prediction type mayinclude at least one of the LIP, the MRL intra prediction, and the ISP intra prediction.Alternatively, the encoding apparatus may also determine whether a normal intraprediction type in which the signaling of the MPM flag is necessary is applied to thecurrent block. The encoding apparatus may generate information about the intraprediction type of the current block.
[204] The encoding apparatus configures an MPM list of the current block (S2110).The MPM list may be configured in various methods as described above in thisdocument. The MPM list may be derived based on intra prediction modes ofneighboring blocks of the current block and predetermined default modes. The MPMlist may include (MPM) candidate intra prediction modes of the current block.
[205] The encoding apparatus derives the intra prediction mode of the current blockbased on the MPM list (S2120). The encoding apparatus may derive an optimal intraprediction mode as the intra prediction mode of the current block based on RDO. Theencoding apparatus may generate information about the intra prediction mode of thecurrent block. Procedures of configuring the MPM list and deriving the intra predictionmode may include the aforementioned procedures in FIGS. 12 to 15 or the like. Theencoding apparatus may derive the intra prediction mode based on the intra predictiontype and generate the information about the intra prediction mode.
[206] The encoding apparatus may encode image information including theinformation about the intra prediction type and the information about the intraprediction mode (S2130). The encoded image information may be output in a form ofthe bitstream. The bitstream may be transmitted to the decoding apparatus through anetwork or a storage medium. The prediction-related information may include theinformation about the intra prediction type and the information about the intraprediction mode. As described above, the information about the intra prediction typemay include the LIP flag, the multi-reference line index, and / or the ISP flag. Theinformation about the intra prediction mode may include at least one of theaforementioned MPM flag, MPM index, and remaining intra mode information. Theimage information may include the residual information, as described above. Theresidual information may represent (quantized) transform coefficients about theresidual samples of the current block.
[207] For example, if the intra prediction type represents the specific intra predictiontype, a bit about the MPM flag may not be included in the bitstream.
[208] For example, if the intra prediction type represents the normal intra predictiontype, one or more bits for each of the MPM flag and the MPM index may be includedin the bitstream.
[209] For example, if the intra prediction type represents the specific intra predictiontype, the encoding apparatus may search for the intra prediction mode of the currentblock only among the candidate intra prediction modes within the MPM list.
[210] The specific intra prediction type may include at least one of the linearinterpolation intra prediction (LIP) type, the multi-reference line (MRL) intraprediction type, or the intra sub-partitions (ISP) intra prediction type.
[211] For example, the specific intra prediction type may include the LIP type, theinformation about the intra prediction type may include the LIP flag, and the value ofthe LIP flag, which is 1, may indicate that the LIP type is applied to the current block.
[212] For example, the specific intra prediction type may include the MRL intraprediction type, the information about the intra prediction type may include the multireference line index (e.g., intra_luma_ref_idx), and the value of the multi-referenceline index larger than 0 may indicate that the MRL intra prediction type is applied tothe current block.
[213] For example, the specific intra prediction type may include the ISP intraprediction type, the information about the intra prediction type may include the ISPflag, and the value of the ISP flag, which is 1, may indicate that the ISP intra predictiontype is applied to the current block.
[214] If the intra prediction type represents the specific intra prediction type, theintra prediction mode of the current block may be limited to one of the candidate intraprediction modes within the MPM list.
[215] FIGS. 23 and 24 schematically illustrate an example of the intra predictionmethod according to the exemplary embodiment of the present disclosure and relatedcomponents. The method illustrated in FIG. 23 may be performed by the decodingapparatus illustrated in FIG. 3. Specifically, for example, S2300 illustrated in FIG. 23may be performed by the entropy decoder 310 of the decoding apparatus, and S2310to S2350 illustrated in FIG. 23 may be performed by the predictor (specifically, theintra predictor 285) of the decoding apparatus. The method illustrated in FIG. 23 mayinclude the aforementioned exemplary embodiments in this document.
[216] Referring to FIG. 23, the decoding apparatus acquires information about anintra prediction type of a current block from a bitstream (S2300). As described above,the information about the intra prediction type may include the LIP flag, the multi-reference line index, and / or the ISP flag.
[217] The decoding apparatus derives the intra prediction type of the current blockbased on the information about the intra prediction type (S2310).
[218] The decoding apparatus acquires information about an intra prediction modeof the current block from the bitstream (S2320). The information about the intraprediction mode may include at least one of the aforementioned MPM flag, MPMindex, and remaining intra mode information.
[219] The decoding apparatus configures a most probable mode (MPM) listincluding candidate intra prediction modes of the current block based on neighboringblocks of the current block (S2330). The MPM list may be configured in variousmethods as described above in this document. The MPM list may be derived based onthe intra prediction modes of the neighboring blocks of the current block andpredetermined default modes. The MPM list may include (MPM) candidate intraprediction modes of the current block.
[220] The decoding apparatus derives the intra prediction mode of the current blockbased on the MPM list and the information about the intra prediction mode (S2340).
[221] The decoding apparatus generates a predicted block of the current block basedon the intra prediction type and the intra prediction mode (S2350). The predicted blockincludes prediction samples of the current block. As described above, a reconstructedblock / picture may be generated based on the predicted block. As described above, thedecoding apparatus may acquire residual information from the bitstream, and thereconstructed block / picture may be generated further based on the residual information.Thereinafter, as necessary, in order to improve the subjective / objective image qualities,the in-loop filtering procedure such as a deblocking filter, a SAO, and / or ALFprocedure may be applied to the reconstructed picture.
[222] For example, if the intra prediction type represents the specific intra predictiontype, the decoding apparatus may parse the MPM index without parsing the MPM flagfrom the bitstream. For example, if the intra prediction type represents the specificintra prediction type, a value of the MPM flag may be derived as 1 without parsing theMPM flag from the bitstream. As another example, if the intra prediction typerepresents the normal intra prediction type, the decoding apparatus may parse theMPM flag from the bitstream, and parse the MPM index if the value of the MPM flagis 1.
[223] For example, the specific intra prediction type may include at least one of thelinear interpolation intra prediction (LIP) type, the multi-reference line (MRL) intraprediction type, or the intra sub-partitions (ISP) intra prediction type.
[224] For example, the specific intra prediction type may include the LIP type, theinformation about the intra prediction type may include the LIP flag, and if the valueof the LIP flag is 1, the LIP type may be derived as the intra prediction type of thecurrent block.
[225] For example, the specific intra prediction type may include the MRL intraprediction type, the information about the intra prediction type may include the multireference line index, and if the value of the multi-reference line index is larger than 0,the MRL intra prediction type may be derived as the intra prediction type of the currentblock.
[226] For example, the specific intra prediction type may include the ISP intraprediction type, the information about the intra prediction type may include the ISPflag, and if the value of the ISP flag is 1, the ISP intra prediction type may be derivedas the intra prediction type of the current block.
[227] For example, if the intra prediction type represents the specific intra predictiontype, the intra prediction mode of the current block may be limited to one of thecandidate intra prediction modes within the MPM list.
[228] For example, if the intra prediction type represents the normal intra predictiontype and the value of the MPM flag is 0, the information about the intra predictionmode may further include remaining intra mode information, and the intra predictionmode of the current block may be derived as one of the remaining intra predictionmodes other than the candidate intra prediction modes within the MPM list based onthe remaining intra mode information.
[229] In the aforementioned exemplary embodiments, while the methods aredescribed based on the flowcharts shown as a series of steps or blocks, the presentdisclosure is not limited to the order of steps, and a certain step may occur in differentorder from or simultaneously with a step different from that described above. Inaddition, those skilled in the art will understand that the steps illustrated in theflowchart are not exclusive, and other steps may be included or one or more steps inthe flowcharts may be deleted without affecting the scope of the present disclosure.
[230] The aforementioned method according to the present disclosure may beimplemented in a software form, and the encoding apparatus and / or the decodingapparatus according to the present disclosure may be, for example, included in anapparatus for performing the image processing, such as a TV, a computer, a smartphone, a set top box, or a display apparatus.
[231] When the exemplary embodiments of the present disclosure are implementedin software, the aforementioned method may be implemented by modules (processes,functions, and so on) for performing the aforementioned functions. Such modules maybe stored in a memory and executed by a processor. The memory may be internal orexternal to the processor, and the memory may be coupled to the processor usingvarious well known means. The processor may include an application-specificintegrated circuit (ASIC), other chipsets, a logic circuit, and / or a data processingdevice. The memory may include a read-only memory (ROM), a random accessmemory (RAM), a flash memory, a memory card, a storage medium, and / or otherstorage device. That is, the exemplary embodiments described in the present disclosuremay be implemented and performed on a processor, a microprocessor, a controller, ora chip. For example, the functional units illustrated in each drawing may beimplemented and performed on a computer, a processor, a microprocessor, a controller,or a chip.
[232] In addition, the decoding apparatus and encoding apparatus to which thepresent document is applied may be included in a multimedia broadcastingtransmitting and receiving device, a mobile communication terminal, a home cinemavideo device, a digital cinema video device, a surveillance camera, a video chat device,a real time communication device such as video communication, a mobile streamingdevice, a storage medium, a camcorder, a Video on Demand (VoD) service provider,an Over the top video (OTT) device, an Internet streaming service provider, a 3D videodevice, a video telephony video device, a medical video device, or the like, and maybe used to process video signals or data signals. For example, the Over the top video(OTT) device may include a game console, a Blu-ray player, an Internet access TV, ahome theater system, a smartphone, a tablet PC, a digital video recorder (DVR), andthe like.
[233] In addition, the processing method to which the present disclosure is appliedmay be produced in the form of a program executed by a computer, and may be storedin a computer-readable recording medium. Multimedia data having a data structureaccording to the present disclosure may also be stored in the computer-readablerecording medium. The computer-readable recording medium includes all kinds ofstorage devices and distributed storage devices in which computer-readable data isstored. The computer-readable recording medium may include, for example, a Blu-raydisc (BD), a universal serial bus (USB), a ROM, a PROM, an EPROM, an EEPROM,a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device.In addition, the computer-readable recording medium also includes mediaimplemented in the form of a carrier wave (e.g., transmission over the Internet). Inaddition, the bitstream generated by the encoding method may be stored in thecomputer-readable recording medium or transmitted through a wired and wirelesscommunication network. In addition, the exemplary embodiment of the presentdisclosure may be implemented by a computer program product by a program code,and the program code may be performed in a computer by the exemplary embodimentof the present disclosure. The program code may be stored on a computer-readablecarrier.
[234] In addition, the contents streaming system to which the present disclosure isapplied may largely include an encoding server, a streaming server, a web server, amedia storage, a user device, and a multimedia input device.
[235] The encoding server performs the role of generating a bitstream bycompressing contents input from multimedia input devices such as a smartphone, acamera, and a camcorder into digital data and transmitting the generated bitstream tothe streaming server. As another example, if multimedia input devices such as asmartphone, a camera, and a camcorder directly produce a bitstream, the encodingserver may be omitted. The bitstream may be generated by an encoding method or amethod for generating a bitstream to which the present disclosure is applied, and thestreaming server may temporarily store the bitstream while the bitstream is transmittedor received.
[236] The streaming server performs the role of transmitting multimedia data to auser device based on a user request through a web server, and the web server performsthe role of a medium informing the user of which services are available. If the userrequests a desired service from the web server, the web server transfers the request tothe streaming server, and the streaming server transmits multimedia data to the user.At this time, the contents streaming system may include a separate control server, andin this case, the control server performs the role of controlling commands / responsesbetween devices within the contents streaming system.
[237] The streaming server may receive contents from a media storage and / or anencoding server. For example, if contents are received from the encoding server, thecontents may be received in real-time. In this case, to provide a smooth streamingservice, the streaming server may store the bitstream for a predetermined time.
[238] Examples of the user device may include a mobile phone, a smartphone, alaptop computer, a digital broadcast terminal, a personal digital assistant (PDA), aportable multimedia player (PMP), a navigation terminal, a slate PC, a tablet PC, anultra-book, a wearable device (e.g., a smart watch or a smart glass), head mounteddisplay (HMD), a digital TV, a desktop computer, a digital signage, or the like. Eachindividual server within the contents streaming system may be operated as a distributedserver, and in this case, data received by each server may be processed in a distributedmanner.
Claims
1. An image decoding method performed by a decoding apparatus comprising: receiving information about an intra prediction type of a current block from a bitstream; deriving the intra prediction type of the current block as a specific intra prediction type based on the information about the intra prediction type; acquiring information about an intra prediction mode of the current block from the bitstream; configuring a most probable mode (MPM) list comprising candidate intra prediction modes of the current block based on neighboring blocks of the current block; deriving the intra prediction mode of the current block based on the MPM list and the information about the intra prediction mode; generating a predicted block of the current block by performing intra prediction based on the specific intra prediction type and the derived intra prediction mode; acquiring residual information from the bitstream; generating a residual block based on the residual information; and generating a reconstructed block based on the residual block and the predicted block; wherein for the specific intra prediction type, the intra prediction is performed based on neighboring reference samples in a specific sample line which is not adjacent to a top boundary and a left boundary of the current block.
2. The image decoding method of claim 1, wherein based on the intra prediction type representing a normal intra prediction type rather than the specific intra prediction type, the information about the intra prediction mode comprises an MPM flag information and an MPM index information.
3. The image decoding method of claim 1, wherein based on the intra prediction type representing the specific intra prediction type, a value of an MPM flag information is derived as 1 without parsing the MPM flag information from the bitstream.
4. The image decoding method of claim 2, wherein based on the intra prediction type representing the normal intra prediction type, the MPM flag information is parsed from the bitstream, and based on a value of the MPM flag information being equal to 1, the MPM index information is parsed.
5. The image decoding method of claim 3, wherein based on the intra prediction type representing the specific intra prediction type, the information about the intra prediction mode comprises an MPM index information; and the MPM index information is parsed without parsing the MPM flag information from the bitstream.
6. An image encoding method performed by an image encoding apparatus comprising: determining an intra prediction type of a current block as a specific intra prediction type; generating information about the intra prediction type of the current block; configuring a most probable mode (MPM) list comprising candidate intra prediction modes of the current block based on neighboring blocks of the current block; deriving an intra prediction mode of the current block based on the MPM list; generating a predicted block of the current block by performing intra prediction based on the specific intra prediction type and the derived intra prediction mode; generating predicted information of the current block, the predicted information comprising the information about the intra prediction type and the information about the intra prediction mode; deriving a residual block based on the predicted block; generating residual information based on the residual block; and outputting a bitstream by encoding image information comprising the predicted information and the residual information, wherein the information about the intra prediction mode is generated based on the intra prediction type, wherein for the specific intra prediction type, the intra prediction is performed based on neighboring reference samples in a specific sample line which is not adjacent to a top boundary and a left boundary of the current block.
7. The image encoding method of claim 6, wherein based on the intra prediction type representing a normal intra prediction type rather than the specific intra prediction type, the information about the intra prediction mode comprises: an MPM flag information and an MPM index information.
8. The image encoding method of claim 6, wherein based on the intra prediction type representing the normal intra prediction type, one or more bits for each of an MPM flag information and an MPM index information are comprised in the bitstream.
9. The image encoding method of claim 6, wherein based on the intra prediction type representing the specific intra prediction type, the intra prediction mode of the current block is searched only among the candidate intra prediction modes within the MPM list.
10. The image encoding method of claim 6, wherein based on the intra prediction type representing the specific intra prediction type, a bit for an MPM flag information of the current block is not comprised in the bitstream.
11. The image encoding method of claim 10, wherein based on the intra prediction type representing the specific intra prediction type, the information about the intra prediction mode comprises an MPM index information; and the MPM index information is configured to be parsed without parsing the MPM flag information from the bitstream.
12. A non-transitory computer-readable storage medium storing a bitstream generated by a method, the method comprising: determining an intra prediction type of a current block as a specific intra prediction type; generating information about the intra prediction type of the current block; configuring a most probable mode (MPM) list comprising candidate intra prediction modes of the current block based on neighboring blocks of the current block; deriving an intra prediction mode of the current block based on the MPM list; generating a predicted block of the current block by performing intra prediction based on the specific intra prediction type and the derived intra prediction mode; generating predicted information of the current block, the predicted information comprising the information about the intra prediction type and the information about the intra prediction mode; deriving a residual block based on the predicted block; generating residual information based on the residual block; and generating the bitstream by encoding image information comprising the predicted information and the residual information, wherein the information about the intra prediction mode is generated based on the intra prediction type, wherein for the specific intra prediction type, the intra prediction is performed based on neighboring reference samples in a specific sample line which is not adjacent to a top boundary and a left boundary of the current block.
