Image encoding / decoding method and apparatus and method of transmitting bitstream

Abstract

An image encoding / decoding method and apparatus and a method of transmitting a bitstream are provided. An image decoding method according to the disclosure can include determining whether a prediction mode of a current block is an intra prediction mode based on information about the prediction mode of the current block, determining whether an intra sub-partition (ISP) is available for the current block when the prediction mode of the current block is the intra prediction mode, decoding an ISP application indicator indicating whether the ISP is applied to the current block when the ISP is available for the current block, and generating a prediction block of the current block by applying the ISP to the current block when the ISP is determined to be applied to the current block based on the ISP application indicator.

Description

[0001] This application is a divisional application of patent application No. 202080019891.9 (PCT / KR2020 / 003359), filed on September 9, 2021, with the title of "Image encoding / decoding method and apparatus and method of transmitting bitstream", and a filing date of March 11, 2020. TECHNICAL FIELD

[0002] The present disclosure relates to an image encoding / decoding method and apparatus and a method of transmitting a bitstream, and more particularly, to a method and apparatus of encoding / decoding an image by applying intra-subblock partitioning (ISP) to an image having various chroma formats, and a method of transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure. BACKGROUND

[0003] Recently, the demand for high-resolution and high-quality images, such as high definition (HD) images and ultra-high definition (UHD) images, is increasing in various fields. As the resolution and quality of image data increase, the amount of information or bits to be transmitted increases relatively compared to existing image data. The increase in the amount of transmitted information or bits results in an increase in transmission and storage costs.

[0004] Therefore, an efficient image compression technique is required to effectively transmit, store, and reproduce information about high-resolution and high-quality images. SUMMARY

[0005] TECHNICAL PROBLEM

[0006] An object of the present disclosure is to provide an image encoding / decoding method and apparatus having improved encoding / decoding efficiency.

[0007] An object of the present disclosure is to provide a method and apparatus of encoding / decoding an image by applying ISP to an image having various chroma formats.

[0008] Another object of the present disclosure is to provide a method of transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0009] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0010] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0011] The technical problems solved by the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not described herein will be clearly understood by those skilled in the art from the following description.

[0012] Technical Solution

[0013] An image decoding method performed by an image decoding apparatus according to an aspect of the disclosure can include determining whether a prediction mode of a current block is an intra prediction mode based on information about the prediction mode of the current block, determining whether an intra sub-partition (ISP) is available for the current block when the prediction mode of the current block is the intra prediction mode, decoding an ISP application indicator indicating whether the ISP is applied to the current block when the ISP is available for the current block, and generating a prediction block for the current block by applying the ISP to the current block when it is determined that the ISP is applied to the current block based on the ISP application indicator.

[0014] In the image decoding method of the disclosure, it can be determined whether the ISP is available for the current block based on a comparison between a size of a luma component block of the current block and a predetermined threshold.

[0015] In the image decoding method of the disclosure, the predetermined threshold can be a maximum transform size.

[0016] In the image decoding method of the disclosure, it can be determined that the ISP is available for the current block when a width and a height of the luma component block of the current block are equal to or smaller than the predetermined threshold.

[0017] In the image decoding method of the disclosure, generating the prediction block for the current block by applying the ISP to the current block can include generating a prediction block for a luma component block of the current block by applying the ISP to the luma component block, determining whether to apply the ISP to a chroma component block of the current block, and generating a prediction block for the chroma component block by applying the ISP to the chroma component block when it is determined that the ISP is applied to the chroma component block.

[0018] In the image decoding method of the disclosure, the determination of whether to apply the ISP to the chroma component block of the current block can be performed based on at least one of a size of the chroma component block or a chroma format of the current block.

[0019] In the image decoding method of the disclosure, it can be determined that the ISP is applied to the chroma component block of the current block when the chroma format of the current block is 4:2:2 or 4:4:4 and at least one of a width or a height of the chroma component block is greater than a maximum transform size.

[0020] In the image decoding method of the disclosure, when the ISP is applied to the chroma component block, a split direction and a number of sub-partitions for the chroma component block can be determined based on a split direction and a number of sub-partitions for the luma component block.

[0021] In the image decoding method of the disclosure, the number of partition directions and sub-partitions for the luma component block can be equal to the number of partition directions and sub-partitions for the chroma component block, respectively.

[0022] In the image decoding method of the disclosure, generating the prediction block for the current block by applying the ISP to the current block can include generating the prediction block for the luma component block by applying the ISP to the luma component block of the current block and generating the prediction block for the chroma component block by applying the ISP to the chroma component block of the current block.

[0023] In the image decoding method of the disclosure, the number of partition directions and sub-partitions for the chroma component block can be determined based on the number of partition directions and sub-partitions for the luma component block.

[0024] In the image decoding method of the disclosure, the number of partition directions and sub-partitions for the chroma component block can be equal to the number of partition directions and (the number of sub-partitions / 2) for the luma component block, respectively.

[0025] An image decoding apparatus according to another aspect of the disclosure can include a memory and at least one processor. The at least one processor can determine whether a prediction mode of a current block is an intra prediction mode based on information about the prediction mode of the current block, determine whether an intra-sub-partition (ISP) is available for the current block when the prediction mode of the current block is the intra prediction mode, decode an ISP application indicator indicating whether the ISP is applied to the current block when the ISP is available for the current block, and generate a prediction block for the current block by applying the ISP to the current block when it is determined that the ISP is applied to the current block based on the ISP application indicator.

[0026] An image encoding method according to another aspect of the disclosure can include determining whether a prediction mode of a current block is an intra prediction mode, determining whether an intra-sub-partition (ISP) is available for the current block when the prediction mode of the current block is the intra prediction mode, determining whether the ISP is applied to the current block when the ISP is available for the current block, generating a prediction block for the current block by applying the ISP to the current block when the ISP is available for the current block, and encoding information about the prediction mode of the current block and information about whether the ISP is applied to the current block.

[0027] A transmission method according to another aspect of the disclosure can transmit a bitstream generated by the image encoding apparatus or the image encoding method of the disclosure.

[0028] A computer-readable recording medium according to another aspect of the disclosure can store a bitstream generated by the image encoding apparatus or the image encoding method of the disclosure.

[0029] A computer-readable recording medium according to another aspect of the disclosure can store a bitstream received and decoded by an image decoding apparatus and used for reconstructing an image. The bitstream can include information on a prediction mode of a current block and an ISP application indicator indicating whether to apply an ISP to the current block, the information on the prediction mode of the current block can be used to determine whether the prediction mode of the current block is an intra prediction mode, the ISP application indicator can be used to determine whether to apply the ISP to the current block when the prediction mode of the current block is the intra prediction mode and it is determined that the ISP is available for the current block, and a prediction block of the current block is generated by performing the ISP for the current block when it is determined that the ISP is applied to the current block.

[0030] The features described above with respect to the brief summary of the disclosure are merely exemplary aspects of the following detailed description of the disclosure and do not limit the scope of the disclosure.

[0031] Advantageous Effects

[0032] According to the disclosure, it is possible to provide an image encoding / decoding method and apparatus having improved encoding / decoding efficiency.

[0033] Further, according to the disclosure, it is possible to provide a method and apparatus of encoding / decoding an image by applying an ISP to an image having various chroma formats.

[0034] Further, according to the disclosure, it is possible to provide a method of transmitting a bitstream generated by an image encoding method or apparatus according to the disclosure.

[0035] Further, according to the disclosure, it is possible to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to the disclosure.

[0036] Further, according to the disclosure, it is possible to provide a recording medium storing a bitstream received, decoded by an image decoding apparatus according to the disclosure and used for reconstructing an image.

[0037] Those skilled in the art will understand that the effects achievable by the disclosure are not limited to what has been particularly described hereinabove and other advantages of the disclosure will be more clearly understood from the detailed description. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a view schematically showing a video encoding system to which embodiments of the disclosure are applicable.

[0039] Figure 2 is a view schematically showing an image encoding apparatus to which embodiments of the disclosure are applicable.

[0040] Figure 3 is a view schematically showing an image decoding apparatus to which embodiments of the disclosure are applicable.

[0041] Figure 4 is a flowchart illustrating a video / image encoding method based on intra prediction.

[0042] Figure 5 is a view illustrating a configuration of an intra predictor 185 according to the present disclosure.

[0043] Figure 6 is a flowchart illustrating a video / image decoding method based on intra prediction.

[0044] Figure 7 is a view illustrating a configuration of an intra predictor 265 according to the present disclosure.

[0045] Figure 8a and 8b is a view illustrating intra prediction directions according to an embodiment of the present disclosure.

[0046] Figure 9 is a view illustrating a conventional method of performing encoding / decoding of a current block according to an ISP mode.

[0047] FIG. 10 is a view illustrating a partitioning example of an ISP for a current block having various sizes.

[0048] FIG. 11 is a view illustrating a relationship between a luma component block (luma component array) and a chroma component block (chroma component array) according to a chroma format.

[0049] Figure 12 is a view illustrating a size of a chroma component block according to a chroma format when a luma component block is a 64x128 block.

[0050] Figure 13 is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[0051] Figure 14 is a view illustrating an example of partitioning a chroma component block according to an ISP application method of Figure 13 .

[0052] Figure 15 is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[0053] Figure 16 is a view illustrating an example of partitioning a chroma component block according to an ISP application method of Figure 15 .

[0054] Figure 17 is a view illustrating an example of a structure of a bitstream in which a modified ISP availability condition is reflected.

[0055] Figure 18is a view illustrating another example of a structure of a bitstream in which an ISP availability condition reflecting a modification is reflected.

[0056] Figure 19 is a view showing a content streaming system to which embodiments of the present disclosure are applicable. DETAILED DESCRIPTION

[0057] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so as to be easily implemented by those skilled in the art. However, the present disclosure can be implemented in various different forms and is not limited to the embodiments described herein.

[0058] In describing the present disclosure, if it is determined that a detailed description of a related known function or configuration makes the scope of the present disclosure unnecessarily obscure, a detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and like reference numerals are assigned to like parts.

[0059] In the present disclosure, when a component is "connected", "coupled", or "linked" to another component, it can include not only a direct connection relationship but also an indirect connection relationship in which a middle component exists. In addition, when a component "includes" or "has" another component, it means that it can further include the other component unless otherwise specified, rather than excluding the other component.

[0060] In the present disclosure, the terms first, second, and the like are used only for the purpose of distinguishing one component from other components, and do not limit the order or importance of the components unless otherwise specified. Accordingly, in the scope of the present disclosure, a first component in one embodiment can be referred to as a second component in another embodiment, and similarly, a second component in one embodiment can be referred to as a first component in another embodiment.

[0061] In the present disclosure, components distinguished from each other are intended to clearly describe each feature, and do not mean that the components must be separated. That is, a plurality of components can be integrated in one hardware or software unit, or one component can be distributed and implemented in a plurality of hardware or software units. Therefore, even if not specifically described, embodiments in which these components are integrated or distributed are included in the scope of the present disclosure.

[0062] In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some components can be optional components. Therefore, embodiments consisting of a subset of the components described in the embodiments are also included in the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

[0063] The disclosure relates to encoding and decoding of a picture, and unless redefined in the disclosure, the terms used in the disclosure can have the general meaning commonly used in the technical field to which the disclosure belongs.

[0064] In the disclosure, a "picture" generally refers to a unit representing one image for a specific time period, while a slice / tile is a coding unit constituting a part of a picture, and one picture can be composed of one or more slices / tiles. In addition, a slice / tile can include one or more coding tree units (CTU).

[0065] In the disclosure, "pixel" or "pel" can mean the smallest individual constituting one picture (or image). In addition, "sample" can be used as a term corresponding to a pixel. The sample can generally represent a pixel or a value of a pixel, and can represent only a pixel / pixel value of a luminance component or only a pixel / pixel value of a chrominance component.

[0066] In the disclosure, "unit" can mean a basic unit of image processing. The unit can include at least one of a specific area of a picture and information related to the area. In some cases, the unit can be used interchangeably with terms such as "sample array", "block", or "region". In general, an MxN block can include a set (or array) of M columns and N rows of transform coefficients or samples (or sample array).

[0067] In the disclosure, "current block" can mean one of "current coding block", "current coding unit", "coding target block", "decoding target block", or "processing target block". When performing prediction, "current block" can mean "current prediction block" or "prediction target block". When performing transform (inverse transform) / quantization (dequantization), "current block" can mean "current transform block" or "transform target block". When performing filtering, "current block" can mean "filtering target block".

[0068] In the disclosure, a slash " / " or "and" can be interpreted to mean "and / or". For example, "A / B" and "A, B" can mean "A and / or B". In addition, "A / B / C" and "A / B / C" can mean "at least one of A, B, and / or C".

[0069] In the disclosure, the term "or" should be interpreted to mean "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in the disclosure, the word "or" should be interpreted to mean "additionally or alternatively".

[0070] Video encoding system overview

[0071] Figure 1is a view schematically illustrating a video encoding system according to the disclosure.

[0072] A video encoding system according to an embodiment can include an encoding apparatus 10 and a decoding apparatus 20. The encoding apparatus 10 can deliver encoded video and / or image information or data in the form of a file or a stream to the decoding apparatus 20 via a digital storage medium or a network.

[0073] The encoding apparatus 10 according to an embodiment can include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding apparatus 20 according to an embodiment can include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 can be referred to as a video / image encoding unit, and the decoding unit 22 can be referred to as a video / image decoding unit. The transmitter 13 can be included in the encoding unit 12. The receiver 21 can be included in the decoding unit 22. The renderer 23 can include a display and the display can be configured as a separate device or an external component.

[0074] The video source generator 11 can acquire a video / image through a process of capturing, synthesizing, or generating a video / image. The video source generator 11 can include a video / image capturing device and / or a video / image generating device. The video / image capturing device can include, for example, one or more cameras, a video / image archive including previously captured videos / images, or the like. The video / image generating device can include, for example, a computer, a tablet, and a smartphone, and can generate a video / image (electronically). For example, a virtual video / image can be generated through a computer or the like. In this case, the video / image capturing process can be replaced by a process of generating related data.

[0075] The encoding unit 12 can encode an input video / image. For compression and encoding efficiency, the encoding unit 12 can perform a series of processes, such as prediction, transformation, and quantization. The encoding unit 12 can output encoded data (encoded video / image information) in the form of a bitstream.

[0076] The transmitter 13 can transmit encoded video / image information in the form of a file or a stream or data output in the form of a bitstream to the receiver 21 of the decoding apparatus 20 through a digital storage medium or a network. The digital storage medium can include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, or the like. The transmitter 13 can include an element for generating a media file through a predetermined file format and can include an element for transmission through a broadcasting / communication network. The receiver 21 can extract / receive a bitstream from a storage medium or a network and transmit the bitstream to the decoding unit 22.

[0077] The decoding unit 22 can decode the video / picture by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transform, and prediction.

[0078] The renderer 23 can render the decoded video / picture. The rendered video / picture can be displayed through a display.

[0079] Image encoding device overview

[0080] Figure 2 is a view schematically showing an image encoding apparatus to which embodiments of the present disclosure are applicable.

[0081] As shown in Figure 2 , the image encoding apparatus 100 can include an image partitioner 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-predictor 180, an intra-predictor 185, and an entropy encoder 190. The inter-predictor 180 and the intra-predictor 185 can be collectively referred to as a “predictor”. The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 can be included in a residual processor. The residual processor can further include the subtractor 115.

[0082] In some embodiments, all or at least some of the plurality of components configuring the image encoding apparatus 100 can be configured by one hardware component (e.g., an encoder or a processor). Also, the memory 170 can include a decoded picture buffer (DPB) and can be configured by a digital storage medium.

[0083] The image partitioner 110 can partition an input image (or picture or frame) input to the image encoding apparatus 100 into one or more processing units. For example, the processing units can be referred to as coding units (CUs). The coding units can be obtained by recursively partitioning a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree binary tree ternary (QT / BT / TT) structure. For example, one coding unit can be partitioned into multiple coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the partitioning of the coding units, the quadtree structure can be applied first, and then the binary tree structure and / or the ternary tree structure can be applied. The coding process according to the disclosure can be performed based on a final coding unit that is not partitioned any more. The final coding unit can be a maximum coding unit or a coding unit of a deeper depth obtained by partitioning the maximum coding unit. Here, the coding process can include a process of prediction, transform, and reconstruction that will be described later. As another example, the processing unit of the coding process can be a prediction unit (PU) or a transform unit (TU). The prediction unit and the transform unit can be split or partitioned from the final coding unit. The prediction unit can be a sample prediction unit, and the transform unit can be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0084] The predictor (inter prediction 180 or intra prediction 185) can perform prediction on a block to be processed (current block) and generate a prediction block including prediction samples of the current block. The predictor can determine whether to apply intra prediction or inter prediction on a basis of the current block or CU. The predictor can generate various information related to prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0085] The intra predictor 185 can predict the current block by referring to samples in the current picture. The reference samples can be located in the neighbors of the current block or can be placed separately according to the intra prediction mode and / or the intra prediction technique. The intra prediction mode can include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes can include, for example, a DC mode and a planar mode. The directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes according to the level of detail of the prediction direction. However, this is merely an example, and more or less directional prediction modes can be used according to settings. The intra predictor 185 can determine a prediction mode applied to the current block by using a prediction mode applied to a neighboring block.

[0086] The inter predictor 180 can derive a prediction block of a current block based on a reference block (a reference sample array) specified by a motion vector on a reference picture. In this case, to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of a block, a sub-block, or a sample based on the correlation of the motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block can be the same or different. The temporal neighboring block can be referred to as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block can be referred to as a collocated picture (colPic). For example, the inter predictor 180 can configure a motion information candidate list based on the neighboring blocks and generate information indicating which candidate to use to derive the motion vector and / or the reference picture index of the current block. The inter prediction can be performed based on various prediction modes. For example, in the case of a skip mode and a merge mode, the inter predictor 180 can use the motion information of the neighboring blocks as the motion information of the current block. In the case of the skip mode, unlike the merge mode, a residual signal can not be transmitted. In the case of a motion vector prediction (MVP) mode, the motion vector of the neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding a motion vector difference and an indicator of the motion vector predictor. The motion vector difference can mean a difference between the motion vector of the current block and the motion vector predictor.

[0087] The predictor can generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the predictor can not only apply intra prediction or inter prediction, but also simultaneously apply both intra prediction and inter prediction to predict the current block. The prediction method of simultaneously applying both intra prediction and inter prediction to predict the current block can be referred to as combined inter and intra prediction (CIIP). In addition, the predictor can perform intra block copy (IBC) to predict the current block. Intra block copy can be used for content image / video encoding of games, etc., for example, screen content encoding (SCC). IBC is a method of predicting a current picture using a reference block previously reconstructed in the current picture at a position separated by a predetermined distance. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance.

[0088] The prediction signal generated by the predictor can be used to generate a reconstructed signal or to generate a residual signal. The subtractor 115 can generate a residual signal (a residual block or a residual sample array) by subtracting the prediction signal (a prediction block or a prediction sample array) output from the predictor from the input image signal (an original block or an original sample array). The generated residual signal can be transmitted to the transformer 120.

[0089] The transformer 120 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique can include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when relationship information between pixels is represented by a graph. The CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Further, the transform process can be applied to a square pixel block having the same size or can be applied to a block having a variable size other than a square.

[0090] The quantizer 130 can quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 can encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients can be referred to as residual information. The quantizer 130 can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0091] The entropy encoder 190 can perform various encoding methods, such as exponential golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), and the like. The entropy encoder 190 can encode information (e.g., values of syntax elements, etc.) required for video / image reconstruction other than the quantized transform coefficients together or individually. The encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of a network abstraction layer (NAL). The video / image information can further include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Further, the video / image information can further include general constraint information. The information signaled, the information transmitted, and / or the syntax elements described in the disclosure can be encoded through the above-described encoding processes and included in the bitstream.

[0092] The bitstream can be transmitted through a network or can be stored in a digital storage medium. The network can include a broadcasting network and / or a communication network, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) that transmits a signal output from the entropy encoder 190 and / or a storage unit (not shown) that stores the signal can be included as an internal / external element of the image encoding apparatus 100. Alternatively, the transmitter can be provided as a component of the entropy encoder 190.

[0093] The quantized transform coefficients output from the quantizer 130 can be used to generate a residual signal. For example, the quantized transform coefficients can be applied with dequantization and inverse transform by the dequantizer 140 and the inverse transformer 150 to reconstruct a residual signal (a residual block or a residual sample).

[0094] The adder 155 adds the reconstructed residual signal to a prediction signal output from the inter-predictor 180 or the intra-predictor 185 to generate a reconstructed signal (a reconstructed image, a reconstructed block, a reconstructed sample array). If there is no residual for a block to be processed, for example, in the case of applying a skip mode, a prediction block can be used as a reconstructed block. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-prediction of a next block to be processed in the current picture, and can be used for inter-prediction of a next picture by filtering as described below.

[0095] Meanwhile, as described below, luminance mapping with chroma scaling (LMCS) can be applied in a picture encoding process.

[0096] The filter 160 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. The various filtering methods can include, for example, a deblocking filter, a sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filter 160 can generate various information related to filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to filtering can be encoded by the entropy encoder 190 and output in the form of a bitstream.

[0097] The modified reconstructed picture transmitted to the memory 170 can be used as a reference picture in the inter-predictor 180. When inter-prediction is applied by the image encoding apparatus 100, prediction mismatch between the image encoding apparatus 100 and an image decoding apparatus can be avoided and encoding efficiency can be improved.

[0098] The DPB of memory 170 can store modified reconstructed images for use as reference images in inter-frame predictor 180. Memory 170 can store motion information of blocks from which motion information in the current image is derived (or encoded) and / or motion information of already reconstructed blocks in the image. The stored motion information can be transmitted to inter-frame predictor 180 and used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current image and can transmit the reconstructed samples to intra-frame predictor 185.

[0099] Image decoding device overview

[0100] Figure 3 This is a schematic view illustrating an image decoding apparatus to which embodiments of the present disclosure may be applied.

[0101] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as "predictors". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0102] According to an embodiment, all or at least some of the components of the image decoding device 200 may be configured by hardware components (e.g., a decoder or a processor). Furthermore, the memory 250 may include a decoded image buffer (DPB) or may be configured by a digital storage medium.

[0103] The image decoding device 200, having received a bitstream including video / image information, can perform operations related to... Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding apparatus 100. For example, the image decoding apparatus 200 can perform decoding using a processing unit applied in the image encoding apparatus. Therefore, the decoding processing unit can be, for example, a compilation unit. The compilation unit can be obtained by partitioning a compilation tree unit or a maximum compilation unit. The reconstructed image signal decoded and output by the image decoding apparatus 200 can be reproduced by a reproduction apparatus (not shown).

[0104] Image decoding device 200 can receive data in bitstream form from... Figure 2The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse a bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information can further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information can further include general constraint information. The image decoding apparatus can further decode a picture based on the parameter set information and / or the general constraint information. The information and / or the syntax elements described to be signaled / received in the disclosure can be decoded through a decoding process and obtained from the bitstream. For example, the entropy decoder 210 decodes information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and outputs values of syntax elements required for image reconstruction and quantized values of transform coefficients of a residual. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information of a decoded target syntax element, decoding information of a neighboring block and a block being decoded, or a symbol / bin decoded in a previous stage, generate a symbol corresponding to a value of each syntax element according to a probability of occurrence of the bin predicted according to the determined context model. In this case, the CABAC entropy decoding method can update the context model by using information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Among the information decoded by the entropy decoder 210, information related to prediction can be provided to the predictor (inter-predictor 260 and intra-predictor 265), and a residual value, i.e., a quantized transform coefficient and related parameter information, on which entropy decoding is performed in the entropy decoder 210, can be input to the dequantizer 220. In addition, information on filtering among the information decoded by the entropy decoder 210 can be provided to the filter 240. Meanwhile, a receiver (not shown) for receiving a signal output from the image encoding apparatus can be further configured as an internal / external element of the image decoding apparatus 200, or the receiver can be a component of the entropy decoder 210.

[0105] Meanwhile, the image decoding apparatus according to the disclosure can be referred to as a video / image / picture decoding apparatus. The image decoding apparatus can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoder 210. The sample decoder can include at least one of the dequantizer 220, the inverse transformer 230, the adder 235, the filter 240, the memory 250, the inter-predictor 160, or the intra-predictor 265.

[0106] The dequantizer 220 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 can rearrange the quantized transform coefficients in the form of a two-dimensional block. In this case, the rearrangement can be performed based on a coefficient scanning order performed in the image encoding apparatus. The dequantizer 220 can perform dequantization on the quantized transform coefficients by using a quantization parameter (e.g., quantization step length information) and obtain the transform coefficients.

[0107] The inverse transformer 230 can inverse-transform the transform coefficients to obtain a residual signal (a residual block, a residual sample array).

[0108] The predictor can perform prediction on the current block and generate a prediction block including predicted samples of the current block. The predictor can determine whether to apply intra prediction or inter prediction to the current block based on information about prediction output from the entropy decoder 210, and can determine a specific intra / inter prediction mode (prediction technique).

[0109] The same as described in the predictor of the image encoding apparatus 100, the predictor can generate a prediction signal based on various prediction methods (techniques) described later.

[0110] The intra predictor 265 can predict the current block by referring to samples in the current picture. The description of the intra predictor 185 is equally applicable to the intra predictor 265.

[0111] The inter predictor 260 can derive a prediction block of the current block based on a reference block (a reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of a block, a sub-block, or a sample based on the correlation of the motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter predictor 260 can configure a motion information candidate list based on the neighboring blocks, and derive a motion vector and / or a reference picture index of the current block based on received candidate selection information. The inter prediction can be performed based on various prediction modes, and information about prediction can include information indicating an inter prediction mode of the current block.

[0112] The adder 235 can generate a reconstructed signal (a reconstructed picture, a reconstructed block, a reconstructed sample array) by adding the obtained residual signal to a prediction signal (a prediction block, a prediction sample array) output from the predictor (including the inter predictor 260 and / or the intra predictor 265). The description of the adder 155 is equally applicable to the adder 235.

[0113] Meanwhile, as described below, luma mapping with chroma scaling (LMCS) can be applicable in a picture decoding process.

[0114] The filter 240 can improve subjective / objective picture quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and store the modified reconstructed picture in the memory 250, particularly, in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0115] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-predictor 260. The memory 250 can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in the picture that has been reconstructed. The stored motion information can be transferred to the inter-predictor 260 to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory 250 can store reconstructed samples of a reconstructed block in the current picture and transfer the reconstructed samples to the intra-predictor 265.

[0116] In the disclosure, the embodiments described in the filter 160, the inter-predictor 180, and the intra-predictor 185 of the image encoding apparatus 100 can be equally or correspondingly applied to the filter 240, the inter-predictor 260, and the intra-predictor 265 of the image decoding apparatus 200.

[0117] Intra prediction overview

[0118] Hereinafter, an intra-prediction method according to an embodiment will be described.

[0119] Intra-prediction can refer to prediction in which a prediction sample for a current block is generated based on reference samples in a picture to which the current block belongs (hereinafter, referred to as a current picture). When intra-prediction is applied to the current block, neighboring reference samples to be used for the intra-prediction of the current block can be derived. The neighboring reference samples of the current block can include samples neighboring a left boundary of the current block having a size of nWxnH and a total of 2xnH samples neighboring a lower left, samples neighboring an upper boundary of the current block and a total of 2xnW samples neighboring an upper right, and one sample neighboring an upper left of the current block having a size of nWxnH. Alternatively, the neighboring reference samples of the current block can include a plurality of columns of top neighboring samples and a plurality of rows of left neighboring samples. Further, the neighboring reference samples of the current block can include a total of nH samples neighboring a right boundary of the current block having a size of nWxnH, a total of nW samples neighboring a lower boundary of the current block, and one sample neighboring a lower right of the current block.

[0120] Some neighboring reference samples of the current block are not yet decoded or can not be available. In this case, the decoder can construct the neighboring reference samples for prediction by replacing the unavailable samples with available samples. Alternatively, the neighboring reference samples to be used for prediction can be constructed using interpolation of available samples.

[0121] When deriving the neighboring reference samples, (i) the prediction samples can be derived based on an average or an interpolation of the neighboring reference samples of the current block, and (ii) the prediction samples can be derived based on the reference samples present in a certain (prediction) direction relative to the prediction samples among the neighboring reference samples of the current block. The case of (i) can be referred to as non-directional mode or non-angular mode and the case of (ii) can be referred to as directional mode or angular mode.

[0122] Further, the prediction samples can be generated by interpolating with a first neighboring sample located in a prediction direction of an intra prediction mode of the current block and a second neighboring sample located in an opposite direction, based on the prediction target samples of the current block among the neighboring reference samples. The above case can be referred to as linear interpolation intra prediction (LIP).

[0123] Further, the chroma prediction samples can be generated based on the luma samples using a linear model. This case can be referred to as linear model (LM) mode.

[0124] In addition, the temporary prediction samples of the current block can be derived based on the filtered neighboring reference samples, and the prediction samples of the current block can be derived by weighted sum of the temporary prediction samples and at least one reference sample derived from the intra prediction mode among the existing neighboring reference samples (i.e., the unfiltered neighboring reference samples). This case can be referred to as position dependent intra prediction (PDPC).

[0125] In addition, the reference sample line with the highest prediction accuracy can be selected from a plurality of neighboring reference sample lines of the current block to derive the prediction samples using the reference samples located in the prediction direction of the corresponding line, at which time the information about the used reference sample line (e.g., intra_luma_ref_idx) is coded and signaled in a bitstream. This case can be referred to as multi-reference line (MRL) intra prediction or MRL-based intra prediction.

[0126] In addition, the current block can be partitioned into vertical or horizontal sub-partitions to perform intra prediction with respect to each sub-partition based on the same intra prediction mode. At this time, neighboring reference samples for intra prediction can be derived in units of sub-partitions. That is, reconstructed samples of previous sub-partitions in coding / decoding order can be used as neighboring reference samples for the current sub-partition. In this case, the intra prediction mode of the current block is equally applied to the sub-partitions, and neighboring reference samples are derived and used in units of sub-partitions, thereby improving the intra prediction performance. Such a prediction method can be referred to as intra sub-partition (ISP) or ISP-based intra prediction.

[0127] The intra prediction technique can be referred to as various terms, such as an intra prediction type or an additional intra prediction mode, to distinguish from directional or non-directional intra prediction modes. For example, the intra prediction technique (intra prediction type or additional intra prediction mode) can include at least one of LIP, LM, PDPC, MRL, ISP, or MIP. Meanwhile, if necessary, post-filtering can be performed on the derived prediction samples.

[0128] Specifically, the intra prediction process can include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra prediction mode / type. In addition, if necessary, post-filtering can be performed on the derived prediction samples.

[0129] Figure 4 is a flowchart illustrating a video / image encoding method based on intra prediction.

[0130] Figure 4 The image encoding method of Figure 2 may be performed by the image encoding apparatus of . Specifically, step S410 can be performed by the intra predictor 185, and step S420 can be performed by the residual processor. Specifically, step S420 can be performed by the subtractor 115. Step S430 can be performed by the entropy encoder 190. The prediction information of step S430 can be derived by the intra predictor 185, and the residual information of step S430 can be derived by the residual processor. The residual information is information about residual samples. The residual information can include information about quantized transform coefficients of the residual samples. As described above, the residual samples can be derived as transform coefficients by the transformer 120 of the image encoding apparatus, and the transform coefficients can be derived as quantized transform coefficients by the quantizer 130. The information about the quantized transform coefficients can be encoded by the entropy encoder 190 through a residual coding process.

[0131] The image encoding apparatus can perform intra prediction for the current block (S410). The image encoding apparatus can determine an intra prediction mode / type for the current block, derive neighboring reference samples for the current block, and generate prediction samples in the current block based on the intra prediction mode / type and the neighboring reference samples. Here, the intra prediction mode / type determination, the neighboring reference samples derivation, and the prediction samples generation process can be performed simultaneously, or any one process can be performed before the other processes.

[0132] Figure 5 is a view illustrating a configuration of an intra predictor 185 according to the disclosure.

[0133] As Figure 5 shown in the above

[0134] The image encoding apparatus can determine a mode / type applied to the current block among a plurality of intra prediction modes / types. The image encoding apparatus can compare rate-distortion (RD) costs of the intra prediction modes / types, and determine a best intra prediction mode / type for the current block.

[0135] Meanwhile, the image encoding apparatus can perform a prediction sample filtering process. The prediction sample filtering can be referred to as post-filtering. Through the prediction sample filtering process, some or all of the prediction samples can be filtered. In some cases, the prediction sample filtering process can be omitted.

[0136] Referring again to Figure 4 , the image encoding apparatus can generate residual samples for the current block based on the prediction samples or the filtered prediction samples (S420). The image encoding apparatus can derive the residual samples by subtracting the prediction samples from original samples of the current block. That is, the image encoding apparatus can derive residual sample values by subtracting corresponding prediction sample values from original sample values.

[0137] The image encoding apparatus can encode image information including information about intra prediction (prediction information) and residual samples. The prediction information can include intra prediction mode information and / or intra prediction technique information. The image encoding apparatus can output the encoded image information in the form of a bitstream. The output bitstream can be transmitted to an image decoding apparatus through a storage medium or a network.

[0138] The residual information can include residual coding syntax, which will be described later. The image encoding apparatus can transform / quantize the residual samples and derive quantized transform coefficients. The residual information can include information about the quantized transform coefficients.

[0139] Meanwhile, as described above, the image encoding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the image encoding apparatus can perform dequantization / inverse transform on the quantized transform coefficients and derive (modified) residual samples. The reason for transforming / quantizing the residual samples and then performing dequantization / inverse transform is to derive the same residual samples as those derived by the image decoding apparatus. The image encoding apparatus can generate a reconstructed block including reconstructed samples for the current block based on the prediction samples and the (modified) residual samples. Based on the reconstructed blocks, a reconstructed picture for the current picture can be generated. As described above, an in-loop filtering process is further applied to the reconstructed picture.

[0140] Figure 6 is a flowchart illustrating a video / image decoding method based on intra prediction.

[0141] The image decoding apparatus can perform operations corresponding to those performed by the image encoding apparatus.

[0142] Figure 6 The image decoding apparatus of Figure 3 may be performed by the image decoding apparatus of The steps S610 to S630 can be performed by the intra predictor 265, and the prediction information of step S610 and the residual information of step S640 can be obtained by the entropy decoder 210 from the bitstream. The residual processor of the image decoding apparatus can derive the residual samples for the current block based on the residual information (S640). Specifically, the dequantizer 220 of the residual processor can perform dequantization based on the dequantized transform coefficients derived from the residual information to derive transform coefficients, and the inverse transformer 230 of the residual processor can perform inverse transform on the transform coefficients to derive the residual samples for the current block. The step S650 can be performed by the adder 235 or the reconstructor.

[0143] Specifically, the image decoding apparatus can derive an intra prediction mode / type for the current block based on the received prediction information (intra prediction mode / type information) (S610). The image decoding apparatus can derive neighboring reference samples for the current block (S620). Based on the intra prediction mode / type and the neighboring reference samples, the image decoding apparatus can generate prediction samples in the current block (S630). In this case, the image decoding apparatus can perform a prediction sample filtering process. The prediction sample filtering can be referred to as post-filtering. Through the prediction sample filtering process, some or all of the prediction samples can be filtered. In some cases, the prediction sample filtering process can be omitted.

[0144] The image decoding device can generate residual samples of the current block based on the received residual information (S640). The image decoding device can generate reconstructed samples for the current block based on the prediction samples and the residual samples and derive a reconstructed block including the reconstructed samples (S650). Based on the reconstructed block, a reconstructed picture for the current picture can be generated. As described above, the in-loop filtering process is further applied to the reconstructed picture.

[0145] Figure 7 is a view illustrating a configuration of the intra predictor 265 according to this disclosure.

[0146] As Figure 7 The intra predictor 265 of the image decoding device can include an intra prediction mode / type determination unit 266, a reference sample derivation unit 267, and a prediction sample derivation unit 268 as shown in FIG. 2. The intra prediction mode / type determination unit 266 can determine an intra prediction mode / type for the current block based on the intra prediction mode / type information generated and signaled by the intra prediction mode / type determination unit 186 of the image encoding device, and the reference sample derivation unit 267 can derive neighboring reference samples of the current block from a reconstructed reference region in the current picture. The prediction sample derivation unit 268 can derive prediction samples of the current block. Meanwhile, although not shown, when the above-described prediction sample filtering process is performed, the intra predictor 265 can further include a prediction sample filter (not shown).

[0147] The intra prediction mode information can include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether a most probable mode (MPM) or a remaining mode is applied to the current block, and when the MPM is applied to the current block, the intra prediction mode information can further include index information (e.g., intra_luma_mpm_idx) indicating one of intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) can consist of an MPM candidate list or an MPM list. In addition, when the MPM is not applied to the current block, the intra prediction mode information can further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of remaining intra prediction modes other than the intra prediction mode candidates (MPM candidates). The image decoding device can determine an intra prediction mode of the current block based on the intra prediction mode information. The MPM candidate modes can include intra prediction modes of neighboring blocks (e.g., left neighboring block and upper neighboring block) of the current block and additional candidate modes.

[0148] Figure 8aIntra prediction directions are shown according to an embodiment of the disclosure. To capture any edge direction presented in natural videos, as Figure 8a As shown in the middle, the intra prediction modes can include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes can include a planar intra prediction mode (planar mode) and a DC intra prediction mode (DC mode), and the directional intra prediction modes can include the 2nd to 66th intra prediction modes.

[0149] Meanwhile, in addition to the above intra prediction modes, the intra prediction modes can further include a cross component linear model (CCLM) mode for chroma samples. The CCLM mode can be divided into L_CCLM, T_CCLM, LT_CCLM according to whether to consider a left sample, an upper sample, or both, and can be applied only to a chroma component.

[0150] For example, the intra prediction modes can be indexed, for example, as shown in the following table.

[0151] [Table 1]

[0152]

[0153] Figure 8b Intra prediction directions are shown according to another embodiment of the disclosure. In Figure 8b In the middle, a dotted line direction represents a wide angle mode applied only to a non-square block. As Figure 8b As shown in the middle, to capture any edge direction presented in natural videos, the intra prediction modes according to an embodiment can include two non-directional intra prediction modes and 93 directional intra prediction modes. The non-directional intra prediction modes can include a planar mode and a DC mode, and the directional intra prediction modes can include the 2nd to 80th and -1st to -14th intra prediction modes, as Figure 8bThe planar mode can be denoted by INTRA PLANAR, and the DC mode can be denoted by INTRA DC. In addition, the directional intra prediction modes can be denoted by INTRA_ANGULAR-14 to INTRA_ANGULAR-1 and INTRA_ANGULAR2 to INTRA_ANGULAR80. Furthermore, the intra prediction technique information can be implemented in various forms. For example, the intra prediction technique information can include intra prediction type index information indicating one of the intra prediction techniques. As another example, the intra prediction technique information can include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether to apply the MRL to the current block and which reference sample line to use if applied, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether to apply the ISP to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating the partitioning type of the sub-partitions when the ISP is applied, flag information indicating whether to apply the PDPC, or flag information indicating whether to apply the LIP. In the disclosure, the ISP flag information can be referred to as an ISP application indicator.

[0154] The intra prediction mode information and / or the intra prediction technique information can be encoded / decoded by the encoding method described in the disclosure. For example, the intra prediction mode information and / or the intra prediction technique information can be encoded / decoded by entropy encoding (e.g., CABAC, CAVLC) based on a truncated (Rice) binary code.

[0155] Before determining whether to apply the predetermined intra prediction technique to the current block, it can be first determined whether the corresponding intra prediction technique is available for the current block. For example, whether the corresponding intra prediction technique is available for the current block can be determined based on the encoding parameters of the current block. In this case, the encoding parameters can include the size (width and / or height) of the current block, the position of the current block, the color component of the current block, or whether another intra prediction technique is applied.

[0156] In addition, the determination as to whether the corresponding intra prediction technique is available for the current block can be performed based on information signaled at a higher level of the current block, such as a sequence, a picture, a slice, and a CTU. For example, when information transmitted at the sequence level indicates that the predetermined intra prediction technique is not available, it can be determined that the corresponding intra prediction technique is not available for a block belonging to the corresponding sequence.

[0157] Upon determining that the predetermined intra prediction technique is applicable to the current block, the image encoding apparatus can use various methods to determine whether to apply the corresponding intra prediction technique to the current block. For example, the image encoding apparatus can determine whether to apply the corresponding intra prediction technique based on RDO.

[0158] The image encoding apparatus can encode and signal the intra prediction technique information in the bitstream. The intra prediction technique information can be encoded according to the corresponding intra prediction technique in various forms. For example, when the intra prediction technique is MRL, the intra prediction technique information can be an index (e.g., intra_luma_ref_idx) indicating a reference line used for prediction of the current block among multiple reference lines. When the intra prediction technique is ISP, the intra prediction technique information can be flag information (e.g., intra_subpartitions_mode_flag) indicating whether to apply ISP to the current block. In addition, when ISP is applied, the intra prediction technique information can further include information on a partition direction (e.g., intra_subpartitions_split_flag). In addition, the intra prediction technique information can include flag information indicating whether to apply PDPC, flag information indicating whether to apply LIP, or flag information indicating whether to apply LM mode.

[0159] Upon determining that the predetermined intra prediction technique is applicable to the current block, the image decoding apparatus can determine whether to apply the corresponding intra prediction technique to the current block based on the signaled intra prediction technique information.

[0160] As another example, in addition to the explicitly signaled intra prediction technique information, the image encoding apparatus and the image decoding apparatus can implicitly derive whether to apply the predetermined intra prediction technique to the current block based on a coding parameter of the current block. In this case, the coding parameter can include a size (width and / or height) of the current block, a position of the current block, a color component of the current block, or whether to apply another intra prediction technique.

[0161] Hereinafter, an ISP mode according to the disclosure will be described in detail.

[0162] The current block can include a luma component block (array) and a corresponding chroma component block (array). In the disclosure, "current block" or "luma component block" can mean "luma component block of the current block", and "chroma component block" or "corresponding chroma component block" can mean "chroma component block of the current block". In addition, the "luma component block" can be referred to as the term "luma block", "luma component block", or "luma block", and the "chroma component block" can be referred to as the term "chroma block", "color component block", or "color block".

[0163] In the conventional intra prediction, a current encoding / decoding target block (current block) is regarded as one unit and thus encoding / decoding is performed without partitioning. However, when the ISP mode is applied, the current block is partitioned in a horizontal or vertical direction to perform intra prediction encoding / decoding. In this case, encoding / decoding is performed in units of partitioned sub-partitions to generate reconstructed sub-partitions, and the reconstructed sub-partitions are used as a reference block for a next partitioned sub-partition.

[0164] Whether the ISP is available for the current block can be determined based on the following conditions of Table 2. The following conditions can be determined based on the luma component block of the current block. That is, in the following cases, the width, height, and position of the current block can mean the width, height, and position of the luma component block of the current block, respectively.

[0165] [Table 2]

[0166]

[0167] For example, when all of the above conditions for the current block are satisfied, it can be determined that the ISP is available for the current block. After determining that the ISP is available for the current block, it can be determined whether to apply the ISP to the current block. In the ISP availability condition, (x0, y0) is a coordinate indicating the position of the left-top sample of the current block. In addition, intra_luma_ref_idx[x0][y0] is information indicating a reference line for prediction of the current block. According to the ISP availability condition, when intra_luma_ref_idx is 0, that is, when the reference line for prediction of the current block is line 0 (a reference line immediately adjacent to the current block), it can be determined that the ISP is available for the current block. When the reference line for prediction of the current block is a line other than line 0, it can be determined that the ISP is not available for the current block.

[0168] In the ISP availability condition, cbWidth and cbHeight correspond to a width and a height of the current block, respectively. In addition, MaxTbSizeY and MinTbSizeY can indicate a maximum transform size and a minimum transform size, respectively. As described above, the residual processing can include a transform or an inverse transform. In this case, a size of a transform block in which the transform or the inverse transform is available can be predefined or signaled through a bitstream. That is, the maximum transform size means a maximum size of a transform block in which the transform or the inverse transform can be performed. In addition, the minimum transform size means a minimum size of a transform block in which the transform or the inverse transform can be performed. For example, when a size of the current block is greater than the maximum transform size, the current block can be split into two or more transform blocks. In addition, the current block can not be split into a transform block having a size smaller than the minimum transform size. The maximum transform size and / or the minimum transform size can be predefined in the image encoding apparatus and the image decoding apparatus, or can be derived based on information signaled at a higher level of the block.

[0169] According to the ISP availability condition, it can be determined that the ISP is available for the current block only when at least one of cbWidth or cbHeight is equal to or smaller than MaxTbSizeY. That is, when both cbWidth and cbHeight are greater than MaxTbSizeY, it can be determined that the ISP is not available for the current block. When cbWidth is greater than MaxTbSizeY, cbHeight is equal to or smaller than MaxTbSizeY, and the ISP is applicable to the current block, the ISP split direction can be determined as a vertical direction as described below. When cbHeight is greater than MaxTbSizeY, cbWidth is equal to or smaller than MaxTbSizeY, and the ISP is applicable to the current block, the ISP split direction can be determined as a horizontal direction as described below.

[0170] According to the ISP availability condition, it can be determined that the ISP is available for the current block when cbWidth cbHeight is greater than MinTbSizeY MinTbSizeY. cbWidth cbHeight can mean an area of the current block or a number of samples included in the current block. For example, if MinTbSizeY is 4, it can be determined that the ISP is available for the current block only when the number of samples included in the current block is greater than 16 (4 4). For example, if MinTbSizeY is 4, it can be determined that the ISP is available for the current block only when the number of samples included in the current block is greater than 16 (4

[0171] Figure 9 is a view illustrating a conventional method of performing encoding / decoding of a current block according to an ISP mode.

[0172] When the current block to be encoded / decoded is input (S910), it can be determined whether to apply the ISP to the current block (S920). The determination of step S920 can include determining whether the ISP is available for the current block and / or determining whether to apply the ISP to the current block. The determination as to whether the ISP is available for the current block can be made based on the ISP availability condition described above. When the ISP is available, the image encoding apparatus can determine whether to apply the ISP to the current block based on various methods as described above, and can encode the result of the determination as intra prediction technique information in a bitstream. When the ISP is available, the image decoding apparatus can determine whether to apply the ISP to the current block based on the signaled intra prediction technique information.

[0173] When the ISP is not applicable to the current block, TU tiling can be performed with respect to the current block (S930). TU tiling means a process of partitioning the current block into a plurality of transform blocks such that the width and height of the current block are equal to or smaller than a maximum transform size as a transformable size. As described above, the processing of the residual signal includes a transform, and the maximum transform size means the maximum size of the transform block for which the transform process can be performed. Accordingly, when the width or height of the current block is greater than the maximum transform size, by partitioning the current block through TU tiling, the width and height of the partitioned block can both be equal to or smaller than the maximum transform size. For example, when the maximum transform size is a size of 64 samples and the current block has a size of 128x128, the current block can be partitioned into four 64x64 blocks. Alternatively, when the maximum transform size is a size of 64 samples and the current block has a size of 64x128 or 128x64, the current block can be partitioned into two 64x64 blocks. For example, when the maximum transform size is a size of 64 samples and the size of the current block has a size of 64x64 or smaller, TU tiling can not be performed.

[0174] Thereafter, encoding / decoding can be performed for the current block or each block partitioned through the TU tiling step (S960). The encoding of step S960 can include intra prediction, residual processing, and / or encoding of the prediction information and the residual information. The encoding of step S960 can include intra prediction, residual sample derivation, and / or reconstructed block generation.

[0175] When the ISP is applied to the current block in step S920, a partition direction and a partitioning number (a sub-partition number) can be determined (S940).

[0176] In step S920, the segmentation direction can be derived based on information signaled via a bitstream (e.g., intra_subpartitions_split_flag). Alternatively, the segmentation direction can be derived implicitly based on the size of the current block. For example, as described above, when the width of the current block is greater than the maximum transform size, the segmentation direction of the ISP can be derived in the vertical direction. Additionally, when the height of the current block is greater than the maximum transform size, the segmentation direction of the ISP can be derived in the horizontal direction. Furthermore, when both the width and height of the current block are greater than the maximum transform size, restrictions can be imposed to prevent the ISP from being used on the current block. For example, when the maximum transform size is 64 samples and the current block to which the ISP applies is a 128x64 block, the segmentation direction of the ISP is derived in the vertical direction, and the width (128 / 4) and height (64) of the subpartition are determined to be the maximum transform size (64) or smaller. Similarly, when the maximum transform size is 64 samples and the current block to which the ISP applies is a 64x128 block, the segmentation direction of the ISP is derived in the horizontal direction, and the width (64) and height (128 / 4) of the sub-partitions are determined to be the maximum transform size (64) or smaller. As described above, when the ISP is applied to the current block, the current block is segmented such that the width and height of the sub-partitions are equal to or less than the maximum transform size. For this reason, it is not necessary to perform TU tiling in step S930 with respect to the current block to which the ISP applies.

[0177] In step S940, the number of splits can be implicitly derived based on the size of the current block. Specifically, the current block can be split according to its size, as shown in Table 3.

[0178] [Table 3]

[0179]

[0180] Figure 10a This is a view illustrating an example of ISP segmentation for 4x8 or 8x4 blocks. (Example:) Figure 10a As shown in the diagram, a 4x8 block or an 8x4 block can be divided into two sub-partitions. When the current block is a 4x8 block and is divided horizontally, it can be divided into two 4x4 sub-partitions. When the current block is a 4x8 block and is divided vertically, it can be divided into two 2x8 sub-partitions. When the current block is an 8x4 block and is divided horizontally, it can be divided into two 8x2 sub-partitions. When the current block is an 8x4 block and is divided vertically, it can be divided into two 4x4 sub-partitions.

[0181] Figure 10b This is a view illustrating an example of ISP segmentation for blocks of 8x8 size or larger.

[0182] As Figure 10b illustrated, a block with a size of 8x8 or larger can be split into four sub-partitions. When the current block is a WxH block, W and H are both 8 or larger, and the current block is split in the horizontal direction, the current block can be split into four Wx(H / 4) sub-partitions. When the current block is a WxH block, W and H are both 8 or larger, and the current block is split in the vertical direction, the current block can be split into four (W / 4)xH sub-partitions.

[0183] Referring again to Figure 9 When the split direction and the split number are determined in step S940, the luma component block of the current block can be split based thereon (S950).

[0184] Thereafter, encoding / decoding can be performed for each split sub-partition (S960). As described above, the encoding of step S960 can include intra prediction, residual processing, and / or encoding of the prediction information and the residual information. In addition, the decoding of step S960 can include intra prediction, residual sample derivation, and / or reconstructed block generation. Specifically, when ISP is applied, the intra prediction mode for the current block is applied to the sub-partitions as well, and the neighboring reference samples are derived and used in the unit of sub-partition, thereby improving the intra prediction performance. That is, when ISP is applied, the residual sample processing procedure is performed in the unit of sub-partition. In other words, the intra prediction samples are derived for each sub-partition, and the residual signal (residual samples) of the corresponding sub-partition is added thereto, thereby obtaining the reconstructed samples. The residual signal (residual samples) can be derived through the dequantization / inverse transform procedure based on the residual information (quantized transform coefficient information or residual coding syntax) in the bitstream described above. That is, the prediction sample derivation and the residual sample derivation for the first sub-partition can be performed, and the reconstructed samples for the first sub-partition can be derived based thereon. In this case, when the prediction samples for the second sub-partition are derived, some of the reconstructed samples in the first sub-partition (e.g., the reconstructed samples adjacent to the left or top side of the second sub-partition) can be used as the neighboring reference samples for the second sub-partition. Similarly, the prediction sample derivation and the residual sample derivation for the second sub-partition can be derived, and the reconstructed samples for the second sub-partition can be derived based thereon. In this case, when the prediction samples for the third sub-partition are derived, some of the reconstructed samples in the second sub-partition (e.g., the reconstructed samples adjacent to the left or top side of the third sub-partition) can be used as the neighboring reference samples for the third sub-partition. Similarly, some of the reconstructed samples in the third sub-partition can be used as the neighboring reference samples for the fourth sub-partition.

[0185] The order of the encoding / decoding among the plurality of sub-partitions is top-down when the split direction is the horizontal direction, and left-to-right when the split direction is the vertical direction. For example, in Figure 10bWhen the split direction is a horizontal direction, the sub-partitions can be sequentially encoded / decoded from the uppermost sub-partition to the lowermost sub-partition. In addition, when the split direction is a vertical direction, the sub-partitions can be sequentially encoded / decoded from the leftmost sub-partition to the rightmost sub-partition.

[0186] When the ISP is applied to the current block, in order to reduce encoding complexity, the MPM list is generated according to each split method (horizontal split and vertical split), and a suitable prediction mode among the prediction modes in the generated MPM list is compared in terms of rate-distortion optimization (RDO) to generate the best mode. In addition, when multi-reference line (MRL) intra prediction is used, a restriction can be imposed such that the ISP is not used. That is, when the 0th reference line (e.g., intra_luma_ref_idx == 0) is used, the ISP is applicable. In addition, when the ISP is applied, a restriction can be imposed such that the PDPC is not used. That is, when the ISP is applied, the PDPC can not be used.

[0187] When the ISP is applied, as the intra prediction technique information, information (intra_subpartitions_mode_flag) indicating whether the ISP is applied can be transmitted in units of the current block, and if the current block uses the ISP (e.g., when the intra_subpartitions_mode_flag is 1), information (intra_subpartitions_split_flag) about the split method (horizontal split or vertical split) can be transmitted.

[0188] According to the reference Figure 9 The conventional method described above, in a single tree structure in which a luma component block of a current block and a chroma component block of the current block are split into the same tree structure, when the ISP is applied to the current block, the luma component block is split into a plurality of sub-partitions, but the ISP is not applicable to the chroma component block. In this case, the width or height of the chroma component block becomes greater than the maximum transform size according to the chroma format, and thus the transform or inverse transform of the chroma component block can become impossible.

[0189] Hereinafter, a relationship between the size of the luma component block according to the chroma format and the size of the chroma component block will be described.

[0190] FIG. 11 is a view illustrating a relationship between a luma component block (luma component array) and a chroma component block (chroma component array) according to a chroma format.

[0191] A source or a coded picture / image can include a luma component (Y) block and two chroma component (cb, cr) blocks. That is, one pixel of a picture / image can include a luma sample and two chroma samples (cb, cr). A color format can represent a configuration format of a luma sample and chroma samples (cb, cr), and can be referred to as a chroma format. The chroma format can be predefined or can be adaptively signaled. For example, the chroma format can be signaled based on at least one of chroma_format_idc or separate_colour_plane_flag as shown in Table 4.

[0192] [Table 4]

[0193]

[0194] In Table 4 above, chroma_format_idc is information indicating a format of a luma sample and chroma samples corresponding thereto, and separate_colour_plane_flag is information indicating three color components Y, cb, and cr coded respectively in a 4:4:4 chroma format. In Table 4 above, when chroma_format_idc is 0, the chroma format corresponds to monochrome, and the current block does not include chroma component blocks but includes only luma component blocks.

[0195] In Table 4 above, when chroma_format_idc is 1, the chroma format corresponds to a 4:2:0 chroma format, and the width and height of the chroma component blocks correspond to half of the width and half of the height of the luma component block, respectively. Figure 11a A positional relationship between a luma sample and chroma samples is shown in a 4:2:0 chroma format.

[0196] In Table 4 above, when chroma_format_idc is 2, the chroma format corresponds to a 4:2:2 chroma format, the width of the chroma component blocks corresponds to half of the width of the luma component block, respectively, and the height of the chroma component blocks is equal to the height of the luma component block. Figure 11b A positional relationship between a luma sample and chroma samples is shown in a 4:2:2 chroma format.

[0197] In Table 4 above, when chroma_format_idc is 3, the chroma format corresponds to a 4:4:4 chroma format, and the width and height of the chroma component blocks correspond to the width and height of the luma component block, respectively. Figure 11c A positional relationship between a luma sample and chroma samples is shown in a 4:4:4 chroma format.

[0198] In Table 4 above, SubWidthC and SubHeightC represent the ratio of luminance samples to chrominance samples. For example, when the width and height of the luminance component block are CbWidth and CbHeight respectively, the width and height of the corresponding chrominance component block can be derived as (CbWidth / SubwidthC) and (CbHeight / SubHeightC) respectively.

[0199] As described with reference to Figure 11, the size of the chroma component block corresponding to the luminance component block of the current block can vary according to the chroma format.

[0200] Figure 12 This is a view illustrating the size of the chroma component blocks according to the chroma format when the luminance component blocks are 64x128 blocks. For example... Figure 12 As shown, the chromaticity component block corresponding to the 64x128 luminance component block can be a 32x64 block in 4:2:0 chromaticity format, a 32x128 block in 4:2:2 chromaticity format, and a 64x128 block in 4:4:4 chromaticity format.

[0201] As mentioned above, according to the reference Figure 9 The conventional method described, in a single-tree structure where the luma and chroma component blocks are segmented into the same tree structure, when the ISP is applied to the current block, the luma component block is segmented into multiple sub-partitions, but the chroma component block is not segmented. For example, when the luma component block to which the ISP is applied is a 64x128 block, since the segmentation direction is determined to be horizontal, the luma component block is segmented into four 64x32 sub-partitions, and the width (64) and height (32) of each sub-partition are equal to or less than the maximum transform size (64). Therefore, a transform or inverse transform can be performed on each sub-partition of the luma component block. However, as Figure 12 As shown, the chroma component blocks are 32x64 blocks in 4:2:0 chroma format, 32x128 blocks in 4:2:2 chroma format, or 64x128 blocks in 4:4:4 chroma format. For example, in the 4:2:2 chroma format and the 4:4:4 chroma format, the transformation or inverse transformation of the chroma component blocks is not possible because the height (128) of the chroma component blocks is greater than the maximum transform size (64).

[0202] Various embodiments of this disclosure for solving the above-described problems will be described in detail below.

[0203] Embodiment #1

[0204] In Embodiment #1 of the disclosure, to solve the conventional problem, when the ISP is applied to the current block, the ISP is adaptively applied to the chroma component block based on a chroma format and / or a size of the chroma component block. According to Embodiment #1 of the disclosure, for example, when the ISP is applied to the chroma component block, the split direction and the split number determined with respect to the luma component block are identically applied to the chroma component block.

[0205] Figure 13 is a flowchart illustrating an ISP application method according to an embodiment of the disclosure.

[0206] Figure 14 is a view illustrating an example of splitting a chroma component block according to the ISP application method of Figure 13

[0207] When the current block to be encoded / decoded is input (S1310), it can be determined whether the ISP is applied to the current block (S1320). The determination of step S1320 can include determining whether the ISP is available for the current block and / or whether the ISP is applied to the current block. The determination as to whether the ISP is available for the current block can be performed based on the ISP availability conditions described above. When the ISP is available, the image encoding apparatus can determine whether the ISP is applied to the current block based on various methods as described above, and can encode the result of the determination as intra prediction technique information in a bitstream. When the ISP is available, the image decoding apparatus can determine whether the ISP is applied to the current block based on the signaled intra prediction technique information.

[0208] When the ISP is not applied to the current block, TU tiling can be performed with respect to the current block (S1330). Thereafter, encoding / decoding can be performed with respect to the current block or each block split by the TU tiling step (S1390). Step S1330 and S1390 are equal to step S930 and S960 of Figure 9 and thus a detailed description thereof will be omitted.

[0209] In step S1320, when the ISP is applied to the current block, a split direction and a split number (a sub-partition number) can be determined (S1340). The description of determining the split direction and the split number is equal to Figure 9 and thus will be omitted.

[0210] When the split direction and the split number are determined in step S1340, the luma component block of the current block can be split based on this (S1350).

[0211] Thereafter, it can be determined whether the ISP is applied to the chroma component block corresponding to the luma component block (S1360). The determination of step S1360 can be performed by a chroma format and / or by a comparison between a width and a height of the chroma component block and a maximum transform size.​

[0212] In the case of the 4:2:0 chroma format, the width and height of the chroma component block are equal to or smaller than the maximum transform size as described below. Therefore, in the case of the 4:2:0 chroma format, it can be determined that ISP is not applicable to the chroma component block without comparing the width and height of the chroma component block with the maximum transform size.

[0213] In the case of the 4:2:2 chroma format or the 4:4:4 chroma format, the width or height of the chroma component block can be larger than the maximum transform size as described below. Therefore, in the case of the 4:2:2 chroma format or the 4:4:4 chroma format, the width and height of the chroma component block can be compared with the maximum transform size, and when the width or height of the chroma component block is larger than the maximum transform size, it can be determined that ISP is applicable to the chroma component block.

[0214] As shown in Figure 14 When the luma component block is a 64x128 block, the chroma component block corresponding thereto is a 32x64 block of the 4:2:0 chroma format, a 32x128 block of the 4:2:2 chroma format, or a 64x128 block of the 4:4:4 chroma format.

[0215] In the example shown in Figure 14 It can be seen that both the width (32) and the height (64) of the chroma component block of the 4:2:0 chroma format are equal to or smaller than the maximum transform size (64). Therefore, in the case of the 4:2:0 chroma format, it can be determined that ISP is not applicable to the 32x64 chroma component block without additional size comparison.

[0216] In contrast, in the case of the 4:2:2 chroma format or the 4:4:4 chroma format, additionally, the width and height of the chroma component block can be compared with the maximum transform size. In the example shown in Figure 14 Since the width (128) of the 32x128 chroma component block of the 4:2:2 chroma format and the width (128) of the 64x128 chroma component block of the 4:4:4 chroma format are larger than the maximum transform size (64), it can be determined that ISP is applicable to the chroma component block.

[0217] As a variation of Embodiment #1, regardless of the chroma format, it can be determined whether to apply ISP to the chroma component block based on whether both the width and the height of the chroma component block are equal to or smaller than the maximum transform size. For example, when both the width and the height of the chroma component block are equal to or smaller than the maximum transform size, it can be determined that ISP is not applicable to the chroma component block. Also, when the width or the height of the chroma component block is larger than the maximum transform size, it can be determined that ISP is applicable to the chroma component block.

[0218] When it is determined that the ISP is applicable to the chroma component block, the chroma component block can be split (S1370). The splitting of the chroma component block can be performed based on the split direction and the split number determined in step S1340. Specifically, the split direction and the split number of the chroma component block can be determined identically to those of the luma component block. According to Embodiment #1 of the present disclosure, in the example shown in FIG. 13B, the luma component block is split into four sub-partitions in the horizontal direction. Accordingly, the 32x128 chroma component block of the 4:2:2 chroma format and the 64x128 chroma component block of the 4:4:4 chroma format can be split into four sub-partitions in the horizontal direction identically to the luma component block. Figure 14

[0219] Thereafter, each of the split sub-partitions can be encoded / decoded (S1390). In this case, the encoding / decoding of step S1390 can be performed for each of the sub-partitions of the luma component block and each of the sub-partitions of the chroma component block. Step S1390 is equal to step S960 and thus a detailed description thereof will be omitted.

[0220] In step S1360, when it is determined that the ISP is not applicable to the chroma component, the chroma component block is not split (S1380), and each of the sub-partitions of the luma component block and the unsplit chroma component block can be encoded / decoded (S1390). Step S1390 is equal to step S960 and thus a detailed description thereof will be omitted. According to Embodiment #1 of the present disclosure, in the example shown in FIG. 13B, the 32x64 chroma component block of the 4:2:0 chroma format can not be split. Figure 14

[0221] According to Embodiment #1 of the present disclosure, when the width or the height of the chroma component block of the current block to which the ISP is applicable is greater than the maximum transform size, it can be possible to solve the problem that the transform or the inverse transform of the chroma component block is not possible by applying the ISP to the chroma component block. In addition, according to Embodiment #1 of the present disclosure, when the chroma format of the current block to which the ISP is applicable is 4:2:0 or the width and the height of the chroma component block of the current block to which the ISP is applicable are equal to or less than the maximum transform size, the ISP is not applicable to the chroma component block. Accordingly, since it is possible to avoid the case where the ISP is unnecessarily applied to the chroma component block, it is possible to reduce the amount of calculation of encoding / decoding. In addition, according to Embodiment #1 of the present disclosure, since the split direction and the split number of the ISP for the chroma component block are determined identically to those of the ISP for the luma component block, it is not necessary to separately signal or derive the split direction and the split number of the ISP for the chroma component block.

[0222] Embodiment #2

[0223] ​​In Embodiment #2 of the disclosure, to solve the conventional problem, when the ISP is applied to the current block, the ISP is also applied to the chroma component block. According to Embodiment #2 of the disclosure, the split direction and the split number of the chroma component block can be determined based on the split direction and the split number determined for the luma component block.

[0224] Figure 15 is a flowchart illustrating an ISP application method according to another embodiment of the disclosure.

[0225] Figure 16 is a view illustrating an example of splitting a chroma component block according to the ISP application method of Figure 15

[0226] When the current block to be encoded / decoded is input (S1510), it can be determined whether the ISP is applied to the current block (S1520). The determination of step S1520 can include determining whether the ISP is available for the current block and / or whether the ISP is applied to the current block. The determination as to whether the ISP is available for the current block can be performed based on the ISP availability conditions described above. When the ISP is available, the image encoding apparatus can determine whether the ISP is applied to the current block based on various methods as described above, and can encode the result of the determination as intra prediction technique information in a bitstream. When the ISP is available, the image decoding apparatus can determine whether the ISP is applied to the current block based on the signaled intra prediction technique information.

[0227] When the ISP is not applied to the current block, TU tiling can be performed with respect to the current block (S1530). Thereafter, encoding / decoding can be performed with respect to the current block or each block split by the TU tiling step (S1570). Steps S1530 and S1570 are equal to steps S930 and S960 of Figure 9 and a detailed description thereof will be omitted.

[0228] In step S1520, when the ISP is applied to the current block, the split direction and the split number (the number of sub-partitions) can be determined (S1540). The description of determining the split direction and the split number is equal to that of Figure 9 and thus will be omitted.

[0229] The luma component block of the current block can be split based on the split direction and the split number determined in step S1540 (S1550).

[0230] ​Thereafter, the chroma component block of the current block can be split based on the split direction and the split number determined in step S1540 (S1560). Specifically, the split direction of the chroma component block can be determined identically as the split direction of the luma component block. In addition, the split number of the chroma component block can be determined based on the split number of the luma component block. For example, when the split number of the luma component block is N, the split number of the chroma component block can be determined as N / n. In Embodiment #2 of the disclosure, for example, n can be 2, but is not limited thereto and n can be any integer. According to Embodiment #2, in Figure 16 the example shown at the top of FIG. 16, a 16x16 luma component block can be split into four sub-partitions in the horizontal direction. In this case, the split direction of the corresponding chroma component block is the horizontal direction, and the split number can be determined as 2. That is, the corresponding chroma component block can be split into two sub-partitions in the horizontal direction regardless of the chroma format and / or size of the chroma component block. In addition, in the example shown at the bottom of FIG. 16, a 4x8 luma component block is split into two sub-partitions in the vertical direction. In this case, the split direction of the corresponding chroma component block is the vertical direction, and the split number can be determined as 1. In this case, since the chroma component block is not substantially split, it can be seen that ISP is not applicable to the chroma component block. Figure 16

[0231] Thereafter, each sub-partition or the non-split chroma component block can be encoded / decoded (S1570). In this case, the encoding / decoding of step S1570 can be performed for each sub-partition of the luma component block and each sub-partition or the chroma component block of the chroma component block. Step S1570 is equal to step S960 and thus a detailed description thereof will be omitted.

[0232] ​In Embodiment #2 of the disclosure, when the size of the maximum coding unit is 128x128, the maximum transform size is half of the width and height of the maximum coding unit, and the width and height of the current block are both greater than the maximum transform size, the ISP is not applicable to the current block. According to Embodiment #2 of the disclosure, since the width and height of the chroma component block of the current block or the sub-partition of the chroma component block to which the ISP is applicable are always equal to or smaller than the maximum transform size, the problem that the transform or inverse transform of the chroma component block is not possible can be solved. In addition, according to Embodiment #2 of the disclosure, since there is no need to perform the determination (e.g., the determination of S1360) on whether to apply the ISP to the chroma component block of the current block, the amount of calculation of encoding / decoding can be reduced. In addition, according to Embodiment #2 of the disclosure, since the number of sub-partitions of the chroma component block is determined to be 2 or 1, the encoding / decoding process of the chroma component block can be simplified. In addition, according to Embodiment #2 of the disclosure, since the split direction and the number of splits of the ISP for the chroma component block are determined in the same way as those for the ISP for the luma component block, there is no need to separately signal or derive the split direction and the number of splits of the ISP for the chroma component block.

[0233] Embodiment #3

[0234] In Embodiment #3 of the disclosure, in order to solve the conventional problem, the width and height of the chroma component block are equal to or smaller than the maximum transform size by changing the ISP applicability condition.

[0235] According to the reference Figure 9 The conventional ISP method described above determines that the ISP is not applicable to the current block when the width and height of the current block are both greater than the maximum transform size. That is, when one of the width or height of the current block is greater than the maximum transform size and the other is equal to or smaller than the maximum transform size, it is determined that the ISP is applicable to the current block.

[0236] In Embodiment #3 of the disclosure, the above-mentioned ISP applicability condition is modified to Table 5 below.

[0237] [Table 5]

[0238]

[0239] Among the modified ISP applicability conditions, the condition that intra_luma_ref_idx[x0][y0] is 0 and cbWidth cbHeight is greater than MinTbSizeY The condition of MinTbSizeY is equivalent to the existing ISP availability condition. According to the modified ISP availability condition, ISP can be determined to be available for the current block only when both cbWidth and cbHeight of the current block are equal to or smaller than MaxTbSizeY.

[0240] According to Embodiment #3 of the present disclosure to which the modified ISP availability condition is applied, ISP is determined to be available for the current block only when both width and height of the current block are equal to or smaller than the maximum transform size, and intra_subpartitions_mode_flag indicating whether to apply ISP or not can be transmitted.

[0241] According to Embodiment #3 of the present disclosure, since both width and height of the current block are equal to or smaller than the maximum transform size, width and / or height of the chroma sample block corresponding to the luma sample block of the current block is always equal to or smaller than the maximum transform size regardless of the chroma format. Therefore, the problem of referring to Figure 9 the conventional ISP method can be solved by applying the modified ISP availability condition.

[0242] The method according to Embodiment #3 of the present disclosure can be implemented by modifying the ISP availability condition of the conventional method. Therefore, the method according to Embodiment #3 of the present disclosure can be executed identically to the conventional method except that the condition for determining whether ISP is available in step S920 is different. Figure 9 The method according to Embodiment #3 of the present disclosure can be executed identically to the conventional method as shown in the flowchart.

[0243] Figure 17 is a view illustrating an example of the structure of a bitstream in which the modified ISP availability condition is reflected.

[0244] Figure 17 The rectangular box of Embodiment #3 of the present disclosure is related to signaling of information of ISP with respect to intra prediction technology information of the current coding unit. As Figure 17 indicated in Embodiment #3 of the present disclosure, when the above-mentioned modified ISP availability condition is satisfied, intra_subpartitions_mode_flag related to whether to apply ISP or not can be transmitted. intra_subpartitions_mode_flag is information indicating whether ISP is applied to the current coding unit. When the ISP availability condition is not satisfied, intra_subpartitions_mode_flag is not transmitted, and it can be determined that ISP is not available for the current coding unit.

[0245] The modified ISP availability condition can include one or more conditions, and the one or more conditions included in the modified ISP availability condition are not limited to the above-described examples. That is, within the scope of the technical idea according to the present disclosure, some conditions can be omitted or other conditions can be additionally included.

[0246] In addition, as Figure 17 indicated, when the ISP is applicable to the current block, that is, when the intra_subpartitions_mode_flag is 1, information indicating a split direction (intra_subpartitions_split_flag) can be transmitted.

[0247] In Figure 17 the example illustrated, the intra_subpartitions_mode_flag can be transmitted only when the condition of cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY is satisfied. That is, when the transmitted intra_subpartitions_mode_flag is 1, it can be seen that the condition of cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY I has been satisfied. Accordingly, the condition of Figure 18 whether to transmit the intra_sinpartitions_split_flag can be changed as illustrated. Figure 17

[0248] Figure 18 is a view illustrating another example of a structure of a bitstream in which the modified ISP availability condition is reflected.

[0249] Figure 18 The rectangular box of Figure 18 indicates signaling of information of the ISP regarding the intra prediction technology information of the current coding unit. As indicated in

[0250] According to Embodiment #3 of the disclosure, the width and the height of the current block to which the ISP is applied are both equal to or smaller than the maximum transform size. Accordingly, because the width and the height of the chroma component block of the current block are both equal to or smaller than the maximum transform size regardless of the chroma format, the problem that the chroma component block cannot be transformed or inverse-transformed can be solved. In addition, according to Embodiment #3 of the disclosure, because the determination of whether the chroma component block of the current block is applied with the ISP (e.g., the determination of S1360) and the ISP of the chroma component block do not need to be performed, the amount of calculation for encoding / decoding can be reduced. In addition, according to Embodiment #3 of the disclosure, because the conventional encoding / decoding process is not significantly changed, the conventional problem can be solved without significantly increasing the encoding / decoding complexity.

[0251] Although the exemplary methods of the disclosure described above are represented as a series of operations for clarity of description, the order of performing the steps is not intended to be limited, and the steps can be performed simultaneously or in a different order if necessary. To implement the methods according to the disclosure, the described steps can further include other steps, can include the remaining steps in addition to some steps, or can include other additional steps in addition to some steps.

[0252] In the disclosure, the image encoding apparatus or the image decoding apparatus that performs a predetermined operation (step) can perform an operation (step) that confirms an execution condition or a situation of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding apparatus or the image decoding apparatus can perform the predetermined operation after determining whether the predetermined condition is satisfied.

[0253] The various embodiments of the disclosure are not a list of all possible combinations and are intended to describe representative aspects of the disclosure, and matters described in the various embodiments can be applied independently or in combination of two or more.

[0254] The various embodiments of the disclosure can be implemented in hardware, firmware, software, or a combination thereof. In the case of implementing the disclosure through hardware, the disclosure can be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general purpose processor, a controller, a microcontroller, a microprocessor, etc.

[0255] Further, the image decoding apparatus and the image encoding apparatus to which embodiments of the disclosure are applied can be included in a multimedia broadcast transmitting and receiving device, a mobile communication terminal, a home theater video device, a digital theater video device, a surveillance camera, a video chat device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video on demand (VoD) service providing device, an over the top video (OTT video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a video phone video device, a medical video device, etc., and can be used to process a video signal or a data signal. For example, the OTT video device can include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0256] Figure 19 is a view showing a content streaming system to which embodiments of the disclosure can be applied.

[0257] As Figure 19 As shown in the middle, the content streaming system to which embodiments of the disclosure are applied can mainly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0258] The encoding server compresses content input from a multimedia input device such as a smart phone, a camera, a camcorder, etc., into digital data to generate a bitstream and transmits the bitstream to a streaming server. As another example, when a multimedia input device such as a smart phone, a camera, a camcorder, etc., directly generates a bitstream, the encoding server can be omitted.

[0259] The bitstream can be generated by an image encoding method or an image encoding apparatus to which embodiments of the disclosure are applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0260] The streaming server transmits multimedia data to a user device based on a request of a user through a web server, and the web server serves as a medium to inform a service to the user. When the user requests a desired service from the web server, the web server can deliver it to the streaming server, and the streaming server can transmit multimedia data to the user. In this case, the content streaming system can include a separate control server. In this case, the control server serves to control commands / responses between devices in the content streaming system.

[0261] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store a bitstream for a predetermined time.

[0262] Examples of the user equipment can include a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, a tablet computer, an ultrabook, a wearable device (e.g., a smart watch, smart glasses, a head-mounted display), a digital TV, a desktop computer, a digital signage, etc.

[0263] Each server in the content streaming system can operate as a distributed server, in which case the data received from each server can be distributed.

[0264] The scope of the present disclosure includes software or executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling operations of the methods according to various embodiments to be performed on a device or computer, or a non-transitory computer readable medium having such software or commands stored thereon and executable on a device or computer.

[0265] Industrial applicability

[0266] Embodiments of the present disclosure can be used for encoding or decoding an image.

Claims

1. An image decoding method performed by an image decoding apparatus, the image decoding method comprising: determining a prediction mode of a current block based on prediction mode information of the current block; determining whether intra-sub-partition (ISP) is available for the current block based on the prediction mode of the current block being an intra prediction mode; decoding ISP mode information for the current block based on ISP being available for the current block; and generating a prediction block for the current block by applying ISP to the current block when it is determined that the ISP is applied to the current block based on the ISP mode information, wherein the ISP is determined to be available for the current block based on a width and a height of the current block being equal to or smaller than a maximum transform block size of a luma component block and a product of the width and the height of the current block being greater than a square of a minimum transform block size of the luma component block. generating a prediction block for the current block by applying ISP to the current block includes:

2. The image decoding method of claim 1, wherein, generating a prediction block for a luma component block of the current block by applying ISP to the luma component block; determining whether to apply ISP to a chroma component block of the current block; and generating a prediction block for the chroma component block by applying ISP to the chroma component block when it is determined that the ISP is applied to the chroma component block of the current block. The determination of whether to apply ISP to the chroma component block of the current block is performed based on at least one of a size of the chroma component block or a chroma format of the current block.

3. The image decoding method of claim 2, wherein, The ISP is determined to be applied to the chroma component block of the current block and the chroma component block is partitioned into a plurality of transform blocks for a transform process based on the chroma format of the current block being 4:2:2 or 4:4:4 and at least one of a width or a height of the chroma component block being greater than a maximum transform size.

4. The image decoding method according to claim 3, wherein When the ISP is applied to the chroma component block, a partition direction and a number of sub-partitions for the chroma component block are determined based on a partition direction and a number of sub-partitions for the luma component block.

5. The image decoding method of claim 2, wherein, The partition direction and the number of sub-partitions for the chroma component block are equal to the partition direction and the number of sub-partitions for the luma component block, respectively.

6. The image decoding method of claim 5, wherein, generating a prediction block for the current block by applying ISP to the current block includes:

7. The image decoding method according to claim 1, wherein generating a prediction block for a luma component block of the current block by applying ISP to the luma component block; and generating a prediction block for a chroma component block of the current block by applying ISP to the chroma component block. The determination of a partition direction and a number of sub-partitions for the chroma component block is based on a partition direction and a number of sub-partitions for the luma component block.

8. The image decoding method according to claim 7, wherein The partition direction and the number of sub-partitions for the chroma component block are equal to the partition direction and the number of sub-partitions for the luma component block, respectively.

9. The image decoding method of claim 8, wherein, 10. An image decoding apparatus comprising: a memory; and at least one processor, wherein the at least one processor is configured to: determine a prediction mode of a current block based on prediction mode information of the current block; ​ ​ determining whether intra-sub-partition (ISP) is available for the current block based on the prediction mode of the current block being an intra prediction mode; based on ISP being available for the current block, decoding ISP mode information for the current block; and generating a prediction block for the current block by applying ISP to the current block based on determining that ISP is applied to the current block based on the ISP mode information, wherein ISP is determined to be available for the current block based on a width and a height of the current block being equal to or smaller than a maximum transform block size of a luma component block and a product of the width and the height of the current block being greater than a square of a minimum transform block size of the luma component block.

11. An image encoding method performed by an image encoding apparatus, the image encoding method comprising: determining a prediction mode of a current block; determining whether intra-sub-partition (ISP) is available for the current block based on the prediction mode of the current block being an intra prediction mode; based on ISP being available for the current block, determining whether to apply ISP to the current block; generating a prediction block for the current block by applying ISP to the current block based on determining that ISP is applied to the current block; and encoding prediction mode information for the current block and ISP mode information for the current block, wherein ISP is determined to be available for the current block based on a width and a height of the current block being equal to or smaller than a maximum transform block size of a luma component block and a product of the width and the height of the current block being greater than a square of a minimum transform block size of the luma component block.

12. A method of transmitting a bitstream generated by the image encoding method of claim 11.

Images