IMAGE DECODING METHODS AND EQUIPMENT, IMAGE ENCODING METHODS AND EQUIPMENT, AND COMPUTER-READABLE MEDIA

Abstract

The invention relates to a method and apparatus for decoding images, a method and apparatus for encoding images, and a means of reading them by a non-forwarding computer. The method for decoding images under the invention comprises the following steps: determining the predictive mode of the current block based on predictive mode information of the current block; determining whether the intra-subpartitions (ISPs) in the image are available to the current block based on the predictive mode of the current block being the predictive mode in the image; decoding the ISP mode information for the current block, based on the ISP available to the current block; and generating a predictive pattern for the current block based on the ISP mode information, wherein the step of determining whether the ISP is available to the current block is performed based on a comparison between the size of the luminance component block of the current block and a predetermined threshold.

Description

Video encoding / decoding method, device, and method for transmitting bitstream

[0001] The present disclosure relates to a video encoding / decoding method, a device, and a method for transmitting a bitstream, and more particularly, to a method and a device for encoding / decoding a video by applying ISP (Intra Subblock-Partitions) to videos of various chroma formats, and a method for transmitting a bitstream generated by the video encoding method / device of the present disclosure.

[0002] Recently, demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, has been increasing across various fields. As image data becomes higher resolution and higher quality, the amount of information transmitted, or bits, increases relative to conventional image data. This increase in information or bits transmitted leads to increased transmission and storage costs.

[0003] Accordingly, a highly efficient image compression technology is required to effectively transmit, store, and play high-resolution, high-quality image information.

[0004] The present disclosure aims to provide a video encoding / decoding method and device with improved encoding / decoding efficiency.

[0005] In addition, the present disclosure aims to provide a method and device for encoding / decoding an image by applying an ISP to an image of various chroma formats.

[0006] In addition, the present disclosure aims to provide a method for transmitting a bitstream generated by an image encoding method or device according to the present disclosure.

[0007] In addition, the present disclosure aims to provide a recording medium storing a bitstream generated by an image encoding method or device according to the present disclosure.

[0008] In addition, the present disclosure aims to provide a recording medium storing a bitstream received and decoded by an image decoding device according to the present disclosure and used for image restoration.

[0009] The technical problems to be achieved in the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure belongs from the description below.

[0010] A video decoding method according to one aspect of the present disclosure may include a step of determining whether a prediction mode of a current block is an intra prediction mode based on information about a prediction mode of the current block, a step of determining whether Intra Sub-Partitions (ISP) are available for the current block if the prediction mode of the current block is an intra prediction mode, a step of decoding an ISP application indicator indicating whether ISP is applied to the current block if ISP is available for the current block, and a step of applying an ISP to the current block to generate a prediction block for the current block if it is determined based on the ISP application indicator that ISP is applied to the current block.

[0011] In the video decoding method of the present disclosure, the step of determining whether the ISP is available for the current block can be performed based on a comparison of the size of the luma component block of the current block and a predetermined threshold value.

[0012] In the image decoding method of the present disclosure, the predetermined threshold value may be a maximum transformation size.

[0013] In the video decoding method of the present disclosure, if both the width and height of the luma component block of the current block are below a predetermined threshold value, the ISP can be determined to be available for the current block.

[0014] In the video decoding method of the present disclosure, the step of applying ISP to the current block to generate a prediction block for the current block may include the step of applying ISP to a luma component block of the current block to generate a prediction block for the luma component block, the step of determining whether ISP is applied to a chroma component block of the current block, and the step of applying ISP to the chroma component block to generate a prediction block for the chroma component block when it is determined that ISP is applied to the chroma component block of the current block.

[0015] In the video decoding method of the present disclosure, the step of determining whether ISP is applied to the chroma component block of the current block may be performed based on at least one of the size of the chroma component block or the chroma format of the current block.

[0016] In the video decoding method of the present disclosure, if the chroma format of the current block is 4:2:2 or 4:4:4, and at least one of the width or height of the chroma component block is greater than the maximum transformation size, it can be determined that ISP is applied to the chroma component block of the current block.

[0017] In the video decoding method of the present disclosure, when applying ISP to the chroma component block, the division direction and the number of subpartitions of the chroma component block can be determined based on the division direction and the number of subpartitions of the luma component block.

[0018] In the video decoding method of the present disclosure, the division direction and the number of subpartitions of the chroma component block may be the same as the division direction and the number of subpartitions of the luma component block, respectively.

[0019] In the video decoding method of the present disclosure, the step of applying ISP to the current block to generate a prediction block for the current block may include the step of applying ISP to a luma component block of the current block to generate a prediction block for the luma component block, and the step of applying ISP to a chroma component block of the current block to generate a prediction block for the chroma component block.

[0020] In the video decoding method of the present disclosure, the division direction and the number of subpartitions of the chroma component block can be determined based on the division direction and the number of subpartitions of the luma component block.

[0021] In the video decoding method of the present disclosure, the division direction and the number of subpartitions of the chroma component block may be the same as the division direction and (number of subpartitions / 2) of the luma component block, respectively.

[0022] According to another aspect of the present disclosure, a video decoding device includes a memory and at least one processor, wherein the at least one processor determines, based on information about a prediction mode of a current block, whether a prediction mode of the current block is an intra prediction mode, and if the prediction mode of the current block is an intra prediction mode, determines whether an ISP is available for the current block, and if an ISP is available for the current block, decodes an ISP application indicator indicating whether an ISP is applied to the current block, and if it is determined, based on the ISP application indicator, that an ISP is applied to the current block, the ISP can be applied to the current block to generate a prediction block for the current block.

[0023] According to another aspect of the present disclosure, a video encoding method may include a step of determining whether a prediction mode of a current block is an intra prediction mode; a step of determining whether an ISP is available for the current block if the prediction mode of the current block is an intra prediction mode; a step of determining whether an ISP is applied to the current block if an ISP is available for the current block; a step of applying an ISP to the current block if an ISP is applied to the current block to generate a prediction block for the current block; and a step of encoding information about the prediction mode of the current block and information about whether an ISP is applied to the current block.

[0024] A transmission method according to another aspect of the present disclosure can transmit a bitstream generated by an image encoding device or an image encoding method of the present disclosure.

[0025] A computer-readable recording medium according to another aspect of the present disclosure can store a bitstream generated by an image encoding method or an image encoding device of the present disclosure.

[0026] According to another aspect of the present disclosure, a computer-readable recording medium stores a bitstream that is received and decoded by an image decoding device and used to restore an image, wherein the bitstream includes information about a prediction mode of a current block and an ISP application indicator that indicates whether ISP is applied to the current block, and the information about the prediction mode of the current block is used to determine whether the prediction mode of the current block is an intra prediction mode, and when the prediction mode of the current block is an intra prediction mode and ISP is determined to be available for the current block, the ISP application indicator is used to determine whether ISP is applied to the current block, and when it is determined that ISP is applied to the current block, ISP is performed on the current block so that a prediction block of the current block can be generated.

[0027] The features briefly summarized above regarding the present disclosure are merely exemplary aspects of the detailed description of the present disclosure that follows and do not limit the scope of the present disclosure.

[0028] According to the present disclosure, a video encoding / decoding method and device with improved encoding / decoding efficiency can be provided.

[0029] In addition, according to the present disclosure, a method and device for encoding / decoding an image by applying an ISP to an image of various chroma formats can be provided.

[0030] Additionally, according to the present disclosure, a method for transmitting a bitstream generated by an image encoding method or device according to the present disclosure can be provided.

[0031] In addition, according to the present disclosure, a recording medium storing a bitstream generated by an image encoding method or device according to the present disclosure can be provided.

[0032] In addition, according to the present disclosure, a recording medium storing a bitstream received and decoded by an image decoding device according to the present disclosure and used for image restoration can be provided.

[0033] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.

[0034] FIG. 1 is a schematic diagram illustrating a video coding system to which an embodiment according to the present disclosure can be applied.

[0035] FIG. 2 is a schematic diagram illustrating an image encoding device to which an embodiment according to the present disclosure can be applied.

[0036] FIG. 3 is a schematic diagram illustrating an image decoding device to which an embodiment according to the present disclosure can be applied.

[0037] Figure 4 is a flowchart illustrating an intra prediction-based video / image encoding method.

[0038] FIG. 5 is a diagram exemplarily illustrating the configuration of an intra prediction unit (185) according to the present disclosure.

[0039] Fig. 6 is a flowchart illustrating an intra prediction-based video / image decoding method.

[0040] FIG. 7 is a diagram exemplarily illustrating the configuration of an intra prediction unit (265) according to the present disclosure.

[0041] FIG. 8A and FIG. 8B are diagrams illustrating intra prediction directions according to an embodiment of the present disclosure.

[0042] FIG. 9 is a diagram for explaining a conventional method for performing encoding / decoding of a current block according to an ISP mode.

[0043] Figure 10 is a diagram showing an example of ISP segmentation for current blocks of various sizes.

[0044] Figure 11 is a diagram for explaining the relationship between a luma component block (luma component array) and a chroma component block (chroma component array) according to a chroma format.

[0045] Figure 12 is a diagram showing the size of a chroma component block according to a chroma format when the luma component block is a 64x128 block.

[0046] FIG. 13 is a flowchart illustrating an ISP application method according to one embodiment of the present disclosure.

[0047] Fig. 14 is a diagram illustrating an example of dividing a chroma component block according to the ISP application method of Fig. 13.

[0048] FIG. 15 is a flowchart for explaining an ISP application method according to another embodiment of the present disclosure.

[0049] Fig. 16 is a diagram illustrating an example of dividing a chroma component block according to the ISP application method of Fig. 15.

[0050] Figure 17 is a diagram illustrating the structure of a bitstream reflecting modified ISP availability conditions.

[0051] Figure 18 is a diagram illustrating another example of the structure of a bitstream reflecting modified ISP availability conditions.

[0052] FIG. 19 is a diagram illustrating a content streaming system to which an embodiment of the present disclosure can be applied.

[0053] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0054] In describing embodiments of the present disclosure, detailed descriptions of known configurations or functions will be omitted if they are deemed to obscure the gist of the present disclosure. Furthermore, portions unrelated to the description of the present disclosure in the drawings have been omitted, and similar portions have been designated with similar reference numerals.

[0055] In the present disclosure, when a component is said to be "connected," "coupled," or "connected" to another component, this may include not only a direct connection, but also an indirect connection in which another component exists in between. Furthermore, when a component is said to "include" or "have" another component, unless otherwise specifically stated, this does not exclude the other component, but rather implies that the other component may be included.

[0056] In this disclosure, terms such as first, second, etc. are used solely to distinguish one component from another, and do not limit the order or importance of components unless specifically stated otherwise. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0057] In this disclosure, distinct components are used to clearly illustrate their respective characteristics, and do not necessarily imply that the components are separated. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed into multiple hardware or software units. Therefore, even if not specifically mentioned, such integrated or distributed embodiments are also included within the scope of this disclosure.

[0058] In the present disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments comprising a subset of the components described in one embodiment are also within the scope of the present disclosure. Furthermore, embodiments including other components in addition to the components described in various embodiments are also within the scope of the present disclosure.

[0059] The present disclosure relates to encoding and decoding of images, and terms used in the present disclosure may have their usual meanings commonly used in the technical field to which the present disclosure belongs, unless newly defined in the present disclosure.

[0060] In the present disclosure, a "picture" generally refers to a unit representing one image of a specific time period, and a slice / tile is a coding unit that constitutes a part of a picture, and a single picture may be composed of one or more slices / tiles. In addition, a slice / tile may include one or more coding tree units (CTUs).

[0061] In the present disclosure, "pixel" or "pel" may refer to the smallest unit that constitutes a picture (or image). Additionally, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luma component or only a pixel / pixel value of a chroma component.

[0062] In the present disclosure, a "unit" may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to the region. In some cases, the term "unit" may be used interchangeably with terms such as "sample array," "block," or "area." In general, an MxN block may include a set (or array) of samples (or sample array) or transform coefficients consisting of M columns and N rows.

[0063] In the present disclosure, the "current block" may mean one of the following: a "current coding block," a "current coding unit," a "block to be encoded," a "block to be decoded," or a "block to be processed." When prediction is performed, the "current block" may mean a "current prediction block" or a "block to be predicted." When transformation (inverse transformation) / quantization (inverse quantization) is performed, the "current block" may mean a "current transformation block" or a "block to be transformed." When filtering is performed, the "current block" may mean a "block to be filtered."

[0064] In this disclosure, " / " and "," can be interpreted as "and / or". For example, "A / B" and "A, B" can be interpreted as "A and / or B". Additionally, "A / B / C" and "A, B, C" can mean "at least one of A, B, and / or C."

[0065] In this disclosure, "or" may be interpreted as "and / or." For example, "A or B" may mean 1) "A" only, 2) "B" only, or 3) "A and B." Alternatively, "or" in this disclosure may mean "additionally or alternatively."

[0066] Overview of Video Coding Systems

[0067] FIG. 1 illustrates a video coding system according to the present disclosure.

[0068] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) in the form of a file or streaming through a digital storage medium or a network.

[0069] An encoding device (10) according to an embodiment may include a video source generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to an embodiment may include a reception unit (21), a decoding unit (22), and a rendering unit (23). The encoding unit (12) may be referred to as a video / image encoding unit, and the decoding unit (22) may be referred to as a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The reception unit (21) may be included in the decoding unit (22). The rendering unit (23) may include a display unit, and the display unit may be configured as a separate device or an external component.

[0070] The video source generation unit (11) can obtain video / images through a process of capturing, synthesizing, or generating video / images. The video source generation unit (11) can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device can include, for example, a computer, a tablet, a smartphone, etc., and can (electronically) generate video / images. For example, a virtual video / image can be generated through a computer, etc., in which case the video / image capture process can be replaced with a process of generating related data.

[0071] The encoding unit (12) can encode input video / images. The encoding unit (12) can perform a series of procedures such as prediction, transformation, and quantization to improve compression and encoding efficiency. The encoding unit (12) can output encoded data (encoded video / image information) in the form of a bitstream.

[0072] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the reception unit (21) of the decoding device (20) via a digital storage medium or a network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) can include an element for generating a media file through a predetermined file format and can include an element for transmission via a broadcasting / communication network. The reception unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0073] The decoding unit (22) can decode video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12).

[0074] The rendering unit (23) can render the decrypted video / image. The rendered video / image can be displayed through the display unit.

[0075] Overview of the video encoding device

[0076] FIG. 2 is a schematic diagram illustrating an image encoding device to which an embodiment according to the present disclosure can be applied.

[0077] As illustrated in FIG. 2, the image encoding device (100) may include an image segmentation unit (110), a subtraction unit (115), a transformation unit (120), a quantization unit (130), an inverse quantization unit (140), an inverse transformation unit (150), an addition unit (155), a filtering unit (160), a memory (170), an inter prediction unit (180), an intra prediction unit (185), and an entropy encoding unit (190). The inter prediction unit (180) and the intra prediction unit (185) may be collectively referred to as a “prediction unit.” The transformation unit (120), the quantization unit (130), the inverse quantization unit (140), and the inverse transformation unit (150) may be included in a residual processing unit. The residual processing unit may further include a subtraction unit (115).

[0078] All or at least some of the plurality of components constituting the video encoding device (100) may be implemented as a single hardware component (e.g., an encoder or a processor) according to an embodiment. In addition, the memory (170) may include a decoded picture buffer (DPB) and may be implemented by a digital storage medium.

[0079] The image segmentation unit (110) can segment an input image (or picture, frame) input to the image encoding device (100) into one or more processing units. For example, the processing unit may be called a coding unit (CU). The coding unit may be obtained by recursively segmenting a coding tree unit (CTU) or a largest coding unit (LCU) according to a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, one coding unit may be segmented into a plurality of coding units of deeper depth based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. For segmenting the coding unit, the quad-tree structure may be applied first, and the binary-tree structure and / or the ternary-tree structure may be applied later. The coding procedure according to the present disclosure may be performed based on the final coding unit that is no longer segmented. The maximum coding unit can be used directly as the final coding unit, and the coding unit of the lower depth obtained by dividing the maximum coding unit can be used as the final concatenated unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or restoration described below. As another example, the processing unit of the coding procedure may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may each be divided or partitioned from the final coding unit. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives a transform coefficient and / or a unit that derives a residual signal from a transform coefficient.

[0080] The prediction unit (inter-prediction unit (180) or intra-prediction unit (185)) can perform prediction on a block to be processed (current block) and generate a predicted block including prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or CU unit. The prediction unit can generate various information regarding the prediction of the current block and transmit the information to the entropy encoding unit (190). The information regarding the prediction can be encoded by the entropy encoding unit (190) and output in the form of a bitstream.

[0081] The intra prediction unit (185) can predict the current block by referring to samples within the current picture. The referenced samples may be located in the neighborhood of the current block or may be located away from it, depending on the intra prediction mode and / or intra prediction technique. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of detail in the prediction direction. However, this is merely an example, and a greater or lesser number of directional prediction modes may be used depending on the settings. The intra prediction unit (185) may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0082] The inter prediction unit (180) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring block can include a spatial neighboring block existing in the current picture and a temporal neighboring block existing in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different from each other. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc. A reference picture including the above temporal neighboring blocks may be called a collocated picture (colPic). For example, the inter prediction unit (180) may construct a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, in the case of skip mode and merge mode, the inter prediction unit (180) may use the motion information of neighboring blocks as the motion information of the current block. In the case of skip mode, unlike the merge mode, a residual signal may not be transmitted.In the motion vector prediction (MVP) mode, the motion vector of the current block can be signaled by using the motion vector of the surrounding blocks as the motion vector predictor and encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can mean the difference between the motion vector of the current block and the motion vector predictor.

[0083] The prediction unit can generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the prediction unit can apply intra prediction or inter prediction to predict the current block, and can also apply intra prediction and inter prediction simultaneously. A prediction method that simultaneously applies intra prediction and inter prediction to predict the current block may be called combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copy (IBC) to predict the current block. Intra block copy can be used for video / image coding of content such as games, such as screen content coding (SCC). IBC is a method of predicting the current block using a previously restored reference block within the current picture located at a predetermined distance from the current block. When IBC is applied, the location of the reference block within the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance.

[0084] The prediction signal generated through the prediction unit can be used to generate a restoration signal or a residual signal. The subtraction unit (115) can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (predicted block, predicted sample array) output from the prediction unit from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit (120).

[0085] The transform unit (120) can apply a transform technique to the residual signal to generate transform coefficients. For example, the transform technique can include at least one of a Discrete Cosine Transform (DCT), a Discrete Sine Transform (DST), a Karhunen-Loeve Transform (KLT), a Graph-Based Transform (GBT), or a Conditionally Non-linear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously reconstructed pixels. The transform process can be applied to a pixel block having a square equal size, or can be applied to a block of a non-square variable size.

[0086] The quantization unit (130) can quantize the transform coefficients and transmit them to the entropy encoding unit (190). The entropy encoding unit (190) can encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients can be called residual information. The quantization unit (130) can rearrange the quantized transform coefficients in a block form into a one-dimensional vector form based on a coefficient scan order, and can also generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0087] The entropy encoding unit (190) can perform various encoding methods, such as, for example, exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoding unit (190) can also encode, together or separately, information necessary for video / image restoration (e.g., values ​​of syntax elements) in addition to quantized transform coefficients. The encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in the form of a network abstraction layer (NAL) unit. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The signaling information, transmitted information and / or syntax elements mentioned in the present disclosure may be encoded through the encoding procedure described above and included in the bitstream.

[0088] The above bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmitting the signal output from the entropy encoding unit (190) and / or a storage unit (not shown) for storing the signal may be provided as an internal / external element of the video encoding device (100), or the transmission unit may be provided as a component of the entropy encoding unit (190).

[0089] The quantized transform coefficients output from the quantization unit (130) can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized transform coefficients through the inverse quantization unit (140) and inverse transformation unit (150), a residual signal (residual block or residual samples) can be restored.

[0090] The addition unit (155) can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit (180) or the intra prediction unit (185). When there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The addition unit (155) can be called a reconstructor or a reconstructed block generation unit. The generated reconstructed signal can be used for intra prediction of the next block to be processed within the current picture, and can also be used for inter prediction of the next picture after filtering as described below.

[0091] Meanwhile, as described later, LMCS (luma mapping with chroma scaling) may be applied during the picture encoding process.

[0092] The filtering unit (160) can improve subjective / objective picture quality by applying filtering to the restoration signal. For example, the filtering unit (160) can apply various filtering methods to the restoration picture to generate a modified restoration picture, and store the modified restoration picture in the memory (170), specifically, in the DPB of the memory (170). The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filtering unit (160) can generate various information regarding filtering and transmit the information to the entropy encoding unit (190), as described later in the description of each filtering method. The information regarding filtering may be encoded by the entropy encoding unit (190) and output in the form of a bitstream.

[0093] The modified restored picture transmitted to the memory (170) can be used as a reference picture in the inter prediction unit (180). Through this, when inter prediction is applied, the image encoding device (100) can avoid prediction mismatch between the image encoding device (100) and the image decoding device, and can also improve encoding efficiency.

[0094] The DPB in the memory (170) can store a modified reconstructed picture to be used as a reference picture in the inter prediction unit (180). The memory (170) can store motion information of a block from which motion information in the current picture is derived (or encoded) and / or motion information of blocks in a picture that has already been reconstructed. The stored motion information can be transferred to the inter prediction unit (180) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (170) can store reconstructed samples of reconstructed blocks in the current picture and transfer them to the intra prediction unit (185).

[0095] Video Decoding Device Overview

[0096] FIG. 3 is a schematic diagram illustrating an image decoding device to which an embodiment according to the present disclosure can be applied.

[0097] As illustrated in FIG. 3, the image decoding device (200) may be configured to include an entropy decoding unit (210), an inverse quantization unit (220), an inverse transformation unit (230), an addition unit (235), a filtering unit (240), a memory (250), an inter prediction unit (260), and an intra prediction unit (265). The inter prediction unit (260) and the intra prediction unit (265) may be collectively referred to as a “prediction unit.” The inverse quantization unit (220) and the inverse transformation unit (230) may be included in a residual processing unit.

[0098] All or at least some of the plurality of components constituting the video decoding device (200) may be implemented as a single hardware component (e.g., a decoder or processor) depending on the embodiment. In addition, the memory (170) may include a DPB and may be implemented by a digital storage medium.

[0099] The video decoding device (200) that receives a bitstream including video / image information can restore the image by performing a process corresponding to the process performed in the video encoding device (100) of FIG. 2. For example, the video decoding device (200) can perform decoding using a processing unit applied in the video encoding device. Therefore, the processing unit for decoding may be, for example, a coding unit. The coding unit may be a coding tree unit or may be obtained by dividing a maximum coding unit. In addition, the restored image signal decoded and output by the video decoding device (200) can be reproduced through a reproduction device (not shown).

[0100] The video decoding device (200) can receive a signal output from the video encoding device of FIG. 2 in the form of a bitstream. The received signal can be decoded through the entropy decoding unit (210). For example, the entropy decoding unit (210) can parse the bitstream to derive information (e.g., video / image information) necessary for image restoration (or picture restoration). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The video decoding device may additionally use information on the parameter set and / or the general constraint information to decode the image. The signaling information, received information, and / or syntax elements mentioned in the present disclosure can be obtained from the bitstream by being decoded through the decoding procedure. For example, the entropy decoding unit (210) can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements required for image restoration and the quantized values ​​of transform coefficients for residuals. More specifically, the CABAC entropy decoding method receives a bin corresponding to each syntax element in the bitstream, determines a context model using information of the syntax element to be decoded and decoding information of the surrounding block and the decoding target block or information of the symbol / bin decoded in the previous step, and predicts the occurrence probability of the bin according to the determined context model to perform arithmetic decoding of the bin to generate a symbol corresponding to the value of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the information of the decoded symbol / bin for the context model of the next symbol / bin after determining the context model. Information regarding prediction among the information decoded by the entropy decoding unit (210) is provided to the prediction unit (inter prediction unit (260) and intra prediction unit (265)), and the residual value on which entropy decoding is performed by the entropy decoding unit (210), i.e., quantized transform coefficients and related parameter information, can be input to the inverse quantization unit (220). In addition, information regarding filtering among the information decoded by the entropy decoding unit (210) can be provided to the filtering unit (240). Meanwhile, a receiving unit (not shown) that receives a signal output from an image encoding device may be additionally provided as an internal / external element of the image decoding device (200), or the receiving unit may be provided as a component of an entropy decoding unit (210).

[0101] Meanwhile, the video decoding device according to the present disclosure may be referred to as a video / video / picture decoding device. The video decoding device may include an information decoder (video / video / picture information decoder) and / or a sample decoder (video / video / picture sample decoder). The information decoder may include an entropy decoding unit (210), and the sample decoder may include at least one of an inverse quantization unit (220), an inverse transformation unit (230), an addition unit (235), a filtering unit (240), a memory (250), an inter prediction unit (260), and an intra prediction unit (265).

[0102] The inverse quantization unit (220) can inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit (220) can rearrange the quantized transform coefficients into a two-dimensional block form. In this case, the rearrangement can be performed based on the coefficient scanning order performed in the image encoding device. The inverse quantization unit (220) can perform inverse quantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain transform coefficients.

[0103] In the inverse transform unit (230), the transform coefficients can be inversely transformed to obtain a residual signal (residual block, residual sample array).

[0104] The prediction unit can perform a prediction on the current block and generate a predicted block containing prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block based on the prediction information output from the entropy decoding unit (210), and can determine a specific intra / inter-prediction mode (prediction technique).

[0105] The fact that the prediction unit can generate a prediction signal based on various prediction methods (techniques) described later is the same as that mentioned in the description of the prediction unit of the image encoding device (100).

[0106] The intra prediction unit (265) can predict the current block by referring to samples within the current picture. The description of the intra prediction unit (185) can be equally applied to the intra prediction unit (265).

[0107] The inter prediction unit (260) can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. At this time, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can further include information on the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit (260) can construct a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Inter prediction can be performed based on various prediction modes (techniques), and the information about the prediction can include information indicating the mode (technique) of inter prediction for the current block.

[0108] The addition unit (235) can generate a restoration signal (restored picture, restoration block, restoration sample array) by adding the acquired residual signal to the prediction signal (predicted block, prediction sample array) output from the prediction unit (including the inter-prediction unit (260) and / or intra-prediction unit (265)). The description of the addition unit (155) can be equally applied to the addition unit (235).

[0109] Meanwhile, as described later, LMCS (luma mapping with chroma scaling) may be applied during the picture decoding process.

[0110] The filtering unit (240) can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit (240) can apply various filtering methods to the restored picture to generate a modified restored picture, and store the modified restored picture in the memory (250), specifically, in the DPB of the memory (250). The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0111] The (modified) reconstructed picture stored in the DPB of the memory (250) can be used as a reference picture in the inter prediction unit (260). The memory (250) can store motion information of a block from which motion information is derived (or decoded) within the current picture and / or motion information of blocks within a picture that has already been reconstructed. The stored motion information can be transferred to the inter prediction unit (260) to be used as motion information of a spatial neighboring block or motion information of a temporal neighboring block. The memory (250) can store reconstructed samples of reconstructed blocks within the current picture and transfer them to the intra prediction unit (265).

[0112] In this specification, the embodiments described in the filtering unit (160), the inter prediction unit (180), and the intra prediction unit (185) of the image encoding device (100) can be applied to the filtering unit (240), the inter prediction unit (260), and the intra prediction unit (265) of the image decoding device (200) in the same or corresponding manner, respectively.

[0113] Overview of Intra Prediction

[0114] Hereinafter, intra prediction according to the present disclosure is described.

[0115] Intra prediction may refer to a prediction that generates prediction samples for a current block based on reference samples within a picture to which the current block belongs (hereinafter, referred to as the current picture). When intra prediction is applied to a current block, peripheral reference samples to be used for intra prediction of the current block may be derived. The peripheral reference samples of the current block may include a total of 2 x nH samples adjacent to a left boundary and a bottom-left neighboring sample of a current block of a size nW x nH, a total of 2 x nW samples adjacent to a top boundary and a top-right neighboring sample of the current block, and one sample adjacent to the top-left neighboring sample of the current block. Alternatively, the peripheral reference samples of the current block may include upper peripheral samples of multiple columns and left peripheral samples of multiple rows. Additionally, the surrounding reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nWxnH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0116] However, some of the surrounding reference samples of the current block may not yet be decoded or available. In this case, the decoder can construct the surrounding reference samples to be used for prediction by substituting the unavailable samples with available samples. Alternatively, the decoder can construct the surrounding reference samples to be used for prediction by interpolating the available samples.

[0117] When neighboring reference samples are derived, (i) a prediction sample can be derived based on an average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can be derived based on a reference sample existing in a specific (prediction) direction with respect to the prediction sample among the neighboring reference samples of the current block. The case of (i) can be called a non-directional mode or a non-angular mode, and the case of (ii) can be called a directional mode or an angular mode.

[0118] In addition, the prediction sample may be generated through interpolation between a first surrounding sample located in the prediction direction of the intra prediction mode of the current block and a second surrounding sample located in the opposite direction based on the prediction target sample of the current block among the surrounding reference samples. The above-described case may be referred to as linear interpolation intra prediction (LIP).

[0119] Additionally, chroma prediction samples can be generated based on luma samples using a linear model. This case can be called LM (Linear Model) mode.

[0120] In addition, a temporary prediction sample of the current block may be derived based on filtered peripheral reference samples, and a prediction sample of the current block may be derived by weighting at least one reference sample derived according to the intra prediction mode among the existing peripheral reference samples, i.e., unfiltered peripheral reference samples, with the temporary prediction sample. This case may be called PDPC (Position dependent intra prediction).

[0121] Additionally, a reference sample line with the highest prediction accuracy among the surrounding multiple reference sample lines of the current block can be selected, and a prediction sample can be derived using the reference sample located in the prediction direction of the selected line. At this time, information regarding the used reference sample line (e.g., intra_luma_ref_idx) can be encoded and signaled in the bitstream. This case can be called multi-reference line intra prediction (MRL) or MRL-based intra prediction.

[0122] In addition, the current block can be divided into vertical or horizontal subpartitions, and intra prediction can be performed based on the same intra prediction mode for each subpartition. At this time, surrounding reference samples for intra prediction can be derived for each subpartition unit. That is, the reconstructed sample of the previous subpartition in the encoding / decoding order can be used as the surrounding reference sample of the current subpartition. In this case, the intra prediction mode for the current block is applied equally to the subpartitions, and by deriving and using surrounding reference samples for each subpartition unit, the intra prediction performance can be improved in some cases. This prediction method can be called intra sub-partitions (ISP) or ISP-based intra prediction.

[0123] The intra prediction techniques described above may be referred to by various terms, such as intra prediction type or supplementary intra prediction mode, to distinguish them from directional or non-directional intra prediction modes. For example, the intra prediction technique (intra prediction type or supplementary intra prediction mode, etc.) may include at least one of the above-described LIP, LM, PDPC, MRL, and ISP. Meanwhile, post-processing filtering may be performed on the derived prediction samples as needed.

[0124] Specifically, the intra prediction procedure may include an intra prediction mode / type determination step, a surrounding reference sample derivation step, and an intra prediction mode / type-based prediction sample derivation step. Additionally, a post-processing filtering step may be performed on the derived prediction samples, if necessary.

[0125] Figure 4 is a flowchart illustrating an intra prediction-based video / image encoding method.

[0126] The encoding method of FIG. 4 can be performed by the video encoding device of FIG. 2. Specifically, step S410 can be performed by the intra prediction unit (185), and step S420 can be performed by the residual processing unit. Specifically, step S420 can be performed by the subtraction unit (115). Step S430 can be performed by the entropy encoding unit (190). The prediction information of step S430 is derived by the intra prediction unit (185), and the residual information of step S430 can be derived by the residual processing unit. The residual information is information about the residual samples. The residual information can include information about quantized transform coefficients for the residual samples. As described above, the residual samples are derived as transform coefficients through the transform unit (120) of the image encoding device, and the transform coefficients can be derived as quantized transform coefficients through the quantization unit (130). Information about the quantized transform coefficients can be encoded in the entropy encoding unit (190) through a residual coding procedure.

[0127] An image encoding device can perform intra prediction on a current block (S410). The image encoding device can determine an intra prediction mode / type for the current block, derive peripheral reference samples of the current block, and then generate prediction samples within the current block based on the intra prediction mode / type and the peripheral reference samples. Here, the procedures of determining the intra prediction mode / type, deriving peripheral reference samples, and generating prediction samples may be performed simultaneously, or one procedure may be performed before the other.

[0128] FIG. 5 is a diagram exemplarily illustrating the configuration of an intra prediction unit (185) according to the present disclosure.

[0129] As illustrated in FIG. 5, the intra prediction unit (185) of the video encoding device may include an intra prediction mode / type determination unit (186), a reference sample derivation unit (187), and / or a prediction sample derivation unit (188). The intra prediction mode / type determination unit (186) may determine an intra prediction mode / type for the current block. The reference sample derivation unit (187) may derive surrounding reference samples of the current block. The prediction sample derivation unit (188) may derive prediction samples of the current block. Meanwhile, although not illustrated, when a prediction sample filtering procedure described below is performed, the intra prediction unit (185) may further include a prediction sample filtering unit (not illustrated).

[0130] The video encoding device can determine a mode / type to be applied to the current block among a plurality of intra prediction modes / types. The video encoding device can compare rate distortion costs (RD costs) for the intra prediction modes / types and determine an optimal intra prediction mode / type for the current block.

[0131] Meanwhile, the video encoding device may also perform a predictive sample filtering procedure. Predictive sample filtering may be referred to as post-filtering. Some or all of the predictive samples may be filtered through the predictive sample filtering procedure. In some cases, the predictive sample filtering procedure may be omitted.

[0132] Referring again to FIG. 4, the video encoding device can generate residual samples for the current block based on the predicted samples or the filtered predicted samples (S420). The video encoding device can derive the residual samples by subtracting the predicted samples from the original samples of the current block. In other words, the video encoding device can derive the residual sample value by subtracting the corresponding predicted sample value from the original sample value.

[0133] The video encoding device can encode video information including information about the intra prediction (prediction information) and residual information about the residual samples (S430). The prediction information can include the intra prediction mode information and / or the intra prediction technique information. The video encoding device can output the encoded video information in the form of a bitstream. The output bitstream can be transmitted to the video decoding device via a storage medium or a network.

[0134] The residual information may include the residual coding syntax described below. The video encoding device may transform / quantize the residual samples to derive quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

[0135] Meanwhile, as described above, the video encoding device can generate a restored picture (including restored samples and restored blocks). To this end, the video encoding device can inversely quantize / inversely transform the quantized transform coefficients to derive (corrected) residual samples. The reason for performing inverse quantization / inverse transformation on the residual samples after transforming / quantizing them in this way is to derive residual samples that are identical to the residual samples derived from the video decoding device. The video encoding device can generate a restored block including restored samples for the current block based on the predicted samples and the (corrected) residual samples. A restored picture for the current picture can be generated based on the restored block. As described above, an in-loop filtering procedure, etc. can be further applied to the restored picture.

[0136] Fig. 6 is a flowchart illustrating an intra prediction-based video / image decoding method.

[0137] The image decoding device can perform an operation corresponding to an operation performed by the image encoding device.

[0138] The decoding method of FIG. 6 can be performed by the image decoding device of FIG. 3. Steps S610 to S630 can be performed by the intra prediction unit (265), and the prediction information of step S610 and the residual information of step S640 can be obtained from the bitstream by the entropy decoding unit (210). The residual processing unit of the image decoding device can derive residual samples for the current block based on the residual information (S640). Specifically, the inverse quantization unit (220) of the residual processing unit can derive transform coefficients by performing inverse quantization based on the quantized transform coefficients derived based on the residual information, and the inverse transform unit (230) of the residual processing unit can derive residual samples for the current block by performing inverse transformation on the transform coefficients. Step S650 can be performed by the addition unit (235) or the restoration unit.

[0139] Specifically, the video decoding device can derive an intra prediction mode / type for the current block based on the received prediction information (intra prediction mode / type information) (S610). In addition, the video decoding device can derive surrounding reference samples of the current block (S620). The video decoding device can generate prediction samples within the current block based on the intra prediction mode / type and the surrounding reference samples (S630). In this case, the video decoding device can perform a prediction sample filtering procedure. The prediction sample filtering may be referred to as post-filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

[0140] The video decoding device can generate residual samples for the current block based on the received residual information (S640). The video decoding device can generate reconstructed samples for the current block based on the prediction samples and the residual samples, and derive a reconstructed block including the reconstructed samples (S650). A reconstructed picture for the current picture can be generated based on the reconstructed block. As described above, an in-loop filtering procedure, etc. can be further applied to the reconstructed picture.

[0141] FIG. 7 is a diagram exemplarily illustrating the configuration of an intra prediction unit (265) according to the present disclosure.

[0142] As illustrated in FIG. 7, the intra prediction unit (265) of the video decoding apparatus may include an intra prediction mode / type determination unit (266), a reference sample derivation unit (267), and a prediction sample derivation unit (268). The intra prediction mode / type determination unit (266) determines the intra prediction mode / type for the current block based on intra prediction mode / type information generated and signaled by the intra prediction mode / type determination unit (186) of the video encoding apparatus, and the reference sample derivation unit (266) may derive surrounding reference samples of the current block from a reconstructed reference region within the current picture. The prediction sample derivation unit (268) may derive prediction samples of the current block. Meanwhile, although not illustrated, when the above-described prediction sample filtering procedure is performed, the intra prediction unit (265) may further include a prediction sample filtering unit (not illustrated).

[0143] The intra prediction mode information may include flag information (e.g., intra_luma_mpm_flag) indicating, for example, whether the most probable mode (MPM) or the remaining mode is applied to the current block, and when the MPM is applied to the current block, the intra prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be configured as an MPM candidate list or an MPM list. In addition, when the MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the intra prediction mode candidates (MPM candidates). The image decoding device may determine the intra prediction mode of the current block based on the intra prediction mode information. The MPM candidate modes may include intra prediction modes of neighboring blocks of the current block (e.g., left neighboring block and upper neighboring block) and additional candidate modes.

[0144] FIG. 8A is a diagram illustrating an intra prediction direction according to an embodiment of the present disclosure. In order to capture an arbitrary edge direction presented in a natural video, the intra prediction mode as illustrated in FIG. 8A may include two non-directional intra prediction modes and 65 directional intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode (planar mode) and a DC intra prediction mode (DC mode), and the directional intra prediction modes may include intra prediction modes 2 to 66.

[0145] Meanwhile, the intra prediction mode may further include a CCLM (cross-component linear model) mode for chroma samples in addition to the aforementioned intra prediction modes. The CCLM mode may be divided into L_CCLM, T_CCLM, and LT_CCLM depending on whether left samples, upper samples, or both are considered for deriving LM parameters, and may be applied only to chroma components.

[0146] Intra prediction modes can be indexed, for example, as shown in the following table.

[0147] Intra prediction modeAssociated name0INTRA_PLANAR1INTRA_DC2..66INTRA_ANGULAR2..INTRA_ANGULAR6681..83INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[0148] FIG. 8B is a diagram illustrating an intra prediction direction according to another embodiment of the present disclosure. In FIG. 8B, the dotted line direction indicates a wide angle mode that is applied only to non-square blocks. As illustrated in FIG. 8B, in order to capture arbitrary edge directions presented in natural video, an intra prediction mode according to an embodiment may include 93 directional intra prediction modes along with two non-directional intra prediction modes. The non-directional intra prediction modes may include a planar mode and a DC mode. The directional intra prediction mode may include intra prediction modes composed of numbers 2 to 80 and numbers -1 to -14, as indicated by the arrows in FIG. 8B. The planar mode may be denoted as INTRA_PLANAR, and the DC mode may be denoted as INTRA_DC. And the directional intra prediction mode can be expressed as INTRA_ANGULAR-14 to INTRA_ANGULAR-1 and INTRA_ANGULAR2 to INTRA_ANGULAR80. In addition, the intra prediction technique information can be implemented in various forms. For example, the intra prediction technique information can include intra prediction technique index information indicating one of the intra prediction techniques. As another example, the intra prediction technique information can include reference sample line information (e.g. intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if so, which reference sample line is used, ISP flag information (e.g. intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block, ISP type information (e.g.intra_subpartitions_split_flag), may include at least one of flag information indicating whether PDPC is applied or flag information indicating whether LIP is applied. In the present disclosure, the ISP flag information may be referred to as an ISP application indicator.

[0149] The intra prediction mode information and / or the intra prediction technique information may be encoded / decoded using the coding method described in the present disclosure. For example, the intra prediction mode information and / or the intra prediction technique information may be encoded / decoded using entropy coding (e.g., CABAC, CAVLC) based on a truncated (rice) binary code.

[0150] Before determining whether a given intra prediction technique is applicable to the current block, it may first be determined whether the intra prediction technique is available for the current block. For example, whether the intra prediction technique is available for the current block may be determined based on encoding parameters for the current block. These encoding parameters may include the size (width and / or height) of the current block, the position of the current block, the color components of the current block, or whether another intra prediction technique is applicable.

[0151] Additionally, the determination of whether the corresponding intra prediction technique is available for the current block may be performed based on information signaled at a higher level of the current block, such as a sequence, picture, slice, or CTU. For example, when information transmitted at the sequence level indicates that a given intra prediction technique is not available, the corresponding intra prediction technique may be determined to be unavailable for blocks belonging to the corresponding sequence.

[0152] If a given intra prediction technique is determined to be available for the current block, the video encoding device can determine whether the corresponding intra prediction technique is applicable to the current block using various methods. For example, the video encoding device can determine whether the corresponding intra prediction technique is applicable based on RDO.

[0153] A video encoding device can encode intra prediction technique information into a bitstream and signal it. The intra prediction technique information can be encoded in various forms depending on the intra prediction technique. For example, when the intra prediction technique is MRL, the intra prediction technique information can be an index (e.g., intra_luma_ref_idx) indicating a reference line among multiple reference lines used for predicting the current block. When the intra prediction technique is ISP, the intra prediction technique information can be flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block. In addition, when ISP is applied, the intra prediction technique information can additionally include information regarding a splitting direction (e.g., intra_subpartitions_split_flag). In addition, the intra prediction technique information can include flag information indicating whether PDPC is applied, flag information indicating whether LIP is applied, flag information indicating whether LM mode is applied, etc.

[0154] If it is determined that a given intra prediction technique is available for the current block, the video decoding device can determine whether to apply the intra prediction technique to the current block based on the signaled intra prediction technique information.

[0155] As another example, whether to apply a given intra prediction technique to the current block may be implicitly derived by the video encoding device and the video decoding device based on encoding parameters for the current block, in addition to the intra prediction technique information that is explicitly signaled. In this case, the encoding parameters may include the size (width and / or height) of the current block, the position of the current block, the color components of the current block, or whether to apply another intra prediction technique.

[0156] Hereinafter, the ISP mode according to the present disclosure will be described in detail.

[0157] A current block may include a luma component block (array) and a corresponding chroma component block (array). In the present disclosure, "current block" or "luma component block" may mean "luma component block of the current block", and "chroma component block" or "corresponding chroma component block" may mean "chroma component block of the current block". In addition, a "luma component block" may be referred to as a "luma block", a "luminance component block", a "luminance block", etc., and a "chroma component block" may be referred to as a "chroma block", a "chroma difference component block", a "chroma difference block", etc.

[0158] Conventional intra prediction treats the current encoding / decoding target block (the current block) as a single unit and performs encoding / decoding without segmentation. However, when the ISP mode is applied, the current block is segmented horizontally or vertically to perform intra prediction encoding / decoding. At this time, encoding / decoding is performed on each segmented subpartition to generate a reconstructed subpartition, which is then used as a reference block for the next segmented subpartition.

[0159] Whether an ISP is available for the current block can be determined based on the conditions in Table 2 below. The conditions below can be determined based on the luma component block of the current block. That is, the width, height, and position of the current block in the conditions below can refer to the width, height, and position of the luma component block of the current block, respectively.

[0160] <ISP 가용 조건> - intra_luma_ref_idx[ x0 ][ y0 ] = = 0 - cbWidth <= MaxTbSizeY || cbHeight <= MaxTbSizeY - cbWidth * cbHeight > MinTbSizeY * MinTbSizeY

[0161] For example, when all of the above conditions for the current block are satisfied, ISP may be determined to be available for the current block. After it is determined that ISP is available for the current block, it may be determined whether to apply ISP to the current block. In the ISP availability condition, (x0, y0) is a coordinate indicating the position of the upper left sample of the current block. In addition, intra_luma_ref_idx[x0][y0] is information indicating a reference line used for prediction of the current block. According to the ISP availability condition, when intra_luma_ref_idx is 0, that is, when the reference line used for prediction of the current block is line 0 (a reference line immediately adjacent to the current block), ISP may be determined to be available for the current block. When the reference line used for prediction of the current block is a line other than line 0, ISP may be determined to be not available for the current block.

[0162] Under the above ISP availability conditions, cbWidth and cbHeight correspond to the width and height of the current block, respectively. In addition, MaxTbSizeY and MinTbSizeY represent the maximum and minimum transformation sizes for the current block, respectively. As described above, residual processing may include transformation or inverse transformation. In this case, the size of a transform block for which transformation or inverse transformation is available may be predefined or signaled through the bitstream. That is, the maximum transformation size refers to the maximum size of a transform block for which transformation or inverse transformation can be performed. In addition, the minimum transformation size refers to the minimum size of a transform block for which transformation or inverse transformation can be performed. For example, if the size of the current block is larger than the maximum transformation size, the current block may be split into two or more transform blocks. In addition, the current block cannot be split into transform blocks with a size smaller than the minimum transformation size. The maximum transformation size and / or the minimum transformation size may be predefined in the video encoding device and the video decoding device, or may be derived based on information signaled at a higher level of the block.

[0163] According to the above ISP availability condition, ISP may be determined to be available for the current block only if at least one of cbWidth or cbHeight is less than or equal to MaxTbSizeY. That is, if both cbWidth and cbHeight are greater than MaxTbSizeY, ISP may be determined to be unavailable for the current block. If cbWidth is greater than MaxTbSizeY and cbHeight is less than or equal to MaxTbSizeY and ISP is applied to the current block, the ISP splitting direction may be determined to be vertical as described below. If cbHeight is greater than MaxTbSizeY and cbWidth is less than or equal to MaxTbSizeY and ISP is applied to the current block, the ISP splitting direction may be determined to be horizontal as described below.

[0164] According to the above ISP availability conditions, when cbWidth * cbHeight is greater than MinTbSizeY * MinTbSizeY, ISP may be determined to be available for the current block. cbWidth * cbHeight may mean the area of ​​the current block or the number of samples included in the current block. For example, when MinTbSizeY is 4, ISP may be determined to be available for the current block only when the number of samples included in the current block is greater than 16 (4 * 4).

[0165] FIG. 9 is a diagram for explaining a conventional method for performing encoding / decoding of a current block according to an ISP mode.

[0166] When a current block to be encoded / decoded is input (S910), whether ISP is to be applied to the current block can be determined (S920). The determination in step S920 may include determining whether ISP is available for the current block and / or determining whether ISP is to be applied to the current block. The determination of whether ISP is available for the current block can be performed based on the ISP availability condition described above. If ISP is available, the video encoding device can determine whether ISP is to be applied to the current block based on various methods as described above, and the determination result can be encoded in the bitstream as intra prediction technique information. If ISP is available, the video decoding device can determine whether ISP is to be applied to the current block based on signaled intra prediction technique information.

[0167] If ISP is not applied to the current block, TU tiling may be performed on the current block (S930). TU tiling refers to a process of dividing the current block into multiple transform blocks such that the width and height of the current block are less than or equal to the maximum transform size, which is a size in which transformation is possible. As described above, processing of the residual signal includes transformation, and the maximum transform size refers to the maximum size of the transform block in which the transformation process can be performed. Therefore, if the width or height of the current block is greater than the maximum transform size, the current block may be divided through TU tiling so that both the width and height of the divided block are less than or equal to the maximum transform size. For example, if the maximum transform size is 64 sample sizes and the current block has a size of 128x128, the current block may be divided into four 64x64 blocks. Alternatively, if the maximum transform size is 64 sample size and the current block has a size of 64x128 or 128x64, the current block may be split into two 64x64 blocks. For example, if the maximum transform size is 64 sample size and the current block is less than or equal to 64x64, TU tiling may not be performed.

[0168] Thereafter, encoding / decoding may be performed on each of the blocks divided through the current block or TU tiling step (S960). The encoding in step S960 may include intra prediction, residual processing, and / or encoding of prediction information and residual information. The decoding in step S960 may include intra prediction, derivation of residual samples, and / or generation of a reconstructed block.

[0169] In step S920, if ISP is applied to the current block, the division direction and the number of divisions (the number of subpartitions) can be determined (S940).

[0170] In step S920, the splitting direction may be derived based on information signaled through the bitstream (e.g., intra_subpartitions_split_flag). Alternatively, the splitting direction may be implicitly derived based on the size of the current block. For example, as described above, if the width of the current block is greater than the maximum transform size, the splitting direction of the ISP may be derived in the vertical direction. Additionally, if the height of the current block is greater than the maximum transform size, the splitting direction of the ISP may be derived in the horizontal direction. Additionally, if both the width and the height of the current block are greater than the maximum transform size, the ISP may be restricted from being available for the current block. For example, if the maximum transform size is a 64 sample size and the current block to which the ISP is applied is a 128x64 block, the ISP splitting direction is derived in the vertical direction, and both the width (128 / 4) and the height (64) of the subpartition are determined to be less than or equal to the maximum transform size (64). Similarly, if the maximum transformation size is 64 sample size and the current block to which ISP is applied is a 64x128 block, the ISP partitioning direction is induced in the horizontal direction, and the width (64) and height (128 / 4) of the subpartitions are both determined to be less than or equal to the maximum transformation size (64). As described above, when ISP is applied to the current block, the current block is partitioned so that the width and height of the subpartitions are both less than or equal to the maximum transformation size. For this reason, the TU tiling of step S930 does not need to be performed for the current block to which ISP is applied.

[0171] In step S940, the number of divisions can be implicitly derived based on the size of the current block. Specifically, the current block can be divided as shown in Table 3 based on its size.

[0172] Block size division number 4x4 No division 4x8, 8x42 All other cases 4

[0173] Fig. 10a shows an example of partitioning an ISP for a 4x8 block or an 8x4 block. As illustrated in Fig. 10a, a 4x8 block or an 8x4 block can be partitioned into two subpartitions. If the current block is a 4x8 block and is partitioned horizontally, the current block can be partitioned into two 4x4 subpartitions. If the current block is a 4x8 block and is partitioned vertically, the current block can be partitioned into two 2x8 subpartitions. If the current block is an 8x4 block and is partitioned horizontally, the current block can be partitioned into two 8x2 subpartitions. If the current block is an 8x4 block and is partitioned vertically, the current block can be partitioned into two 4x4 subpartitions.

[0174] Figure 10b shows an example of ISP segmentation for blocks having a size of 8x8 or larger.

[0175] As illustrated in Fig. 10b, a block with a size of 8x8 or larger can be divided into four subpartitions. If the current block is a WxH block, where both W and H are 8 or larger, and is divided horizontally, the current block can be divided into four Wx(H / 4) subpartitions. If the current block is a WxH block, where both W and H are 8 or larger, and is divided vertically, the current block can be divided into four (W / 4)xH subpartitions.

[0176] Referring again to FIG. 9, when the division direction and the number of divisions are determined in step S940, division of the luma component block of the current block can be performed based on this (S950).

[0177] Thereafter, encoding / decoding may be performed for each of the divided subpartitions (S960). As described above, the encoding of step S960 may include intra prediction, residual processing, and / or encoding of prediction information and residual information. In addition, the decoding of step S960 may include intra prediction, derivation of residual samples, and / or generation of a reconstruction block. Specifically, when ISP is applied, the intra prediction mode for the current block is applied equally to the subpartitions, and the intra prediction performance can be improved by deriving and using surrounding reference samples for each subpartition. That is, when ISP is applied, the residual sample processing procedure is performed for each subpartition. In other words, intra prediction samples are derived for each subpartition, and the residual signal (residual samples) for the corresponding subpartition is added thereto to obtain reconstruction samples. The above residual signal (residual samples) can be derived through an inverse quantization / inverse transformation procedure, etc., based on the residual information (quantized transform coefficient information or residual coding syntax) in the above-described bitstream. That is, prediction samples for the first subpartition and residual samples are derived, and reconstructed samples for the first subpartition can be derived based on the same. In this case, when deriving prediction samples for the second subpartition, some of the reconstructed samples in the first subpartition (e.g., reconstructed samples adjacent to the left or upper side of the second subpartition) can be used as surrounding reference samples for the second subpartition. Similarly, prediction samples for the second subpartition and residual samples are derived, and reconstructed samples for the second subpartition can be derived based on the same.In this case, when deriving prediction samples for the third subpartition, some of the reconstructed samples within the second subpartition (e.g., reconstructed samples adjacent to the left or upper side of the third subpartition) may be used as surrounding reference samples for the third subpartition. Similarly, some of the reconstructed samples within the third subpartition may be used as surrounding reference samples for the fourth subpartition.

[0178] The encoding / decoding order between multiple subpartitions is from top to bottom when the splitting direction is horizontal, and from left to right when the splitting direction is vertical. For example, in Fig. 10b, when the splitting direction is horizontal, each subpartition can be sequentially encoded / decoded from the topmost subpartition to the bottommost subpartition. In addition, when the splitting direction is vertical, each subpartition can be sequentially encoded / decoded from the leftmost subpartition to the rightmost subpartition.

[0179] When ISP is applied to the current block, in order to reduce encoding complexity, an MPM list is generated according to each partitioning method (horizontal partitioning and vertical partitioning), and an appropriate prediction mode among the prediction modes in the generated MPM list is compared in terms of rate distortion optimization (RDO) to generate an optimal mode. In addition, when multiple reference line (MRL) intra prediction is used, the aforementioned ISP may be restricted from being used. That is, ISP may be applied only when the 0th reference line is used (e.g., intra_luma_ref_idx==0). In addition, when ISP is applied, the aforementioned PDPC may be restricted from being used. That is, when ISP is applied, PDPC may not be used.

[0180] When ISP is applied, as intra prediction technique information, information indicating whether ISP is applied (intra_subpartitions_mode_flag) is transmitted in units of current blocks, and if the current block uses ISP (e.g., when intra_subpartitions_mode_flag is 1), information on the splitting method (horizontal splitting or vertical splitting) (intra_subpartitions_split_flag) may be transmitted.

[0181] According to the conventional method described with reference to FIG. 9, in a single tree structure in which the luma component block of the current block and the chroma component block of the current block are divided into the same tree structure, when ISP is applied to the current block, the luma component block is divided into multiple subpartitions, but the ISP is not applied to the chroma component block. In this case, there is a problem in that, depending on the chroma format, the width or height of the chroma component block becomes larger than the maximum conversion size, making conversion or inverse conversion of the chroma component block impossible.

[0182] Below, the relationship between the size of the luma component block and the size of the chroma component block according to the chroma format is explained.

[0183] Figure 11 is a diagram for explaining the relationship between a luma component block (luma component array) and a chroma component block (chroma component array) according to a chroma format.

[0184] A source or coded picture / video may include a luma component (Y) block and two chroma component (cb, cr) blocks. That is, one pixel of the picture / video may include a luma sample and two chroma samples (cb, cr). A color format may indicate a composition format of the luma sample and the chroma sample (cb, cr), and may be called a chroma format. The chroma format may be predetermined or may be signaled adaptively. For example, the chroma format may be signaled based on at least one of chroma_format_idc and separate_color_plane_flag, as shown in Table 4.

[0185] chroma_format_idcseparate_colour_plane_flagChromaArrayTypeChroma formatSubWidthCSubHeightC000Monochrome111014:2:0222024:2:2213034:4:4113104:4:411

[0186] In the above Table 4, chroma_format_idc is information indicating the format of a luma sample and its corresponding chroma sample, and separate_colour_plane_flag is information indicating whether three color components (Y, cb, cr) are encoded separately in a 4:4:4 chroma format. In the above Table 4, when chroma_format_idc is 0, the chroma format corresponds to monochrome, and the current block does not include a chroma component block but only a luma component block.

[0187] In Table 4 above, when chroma_format_idc is 1, the chroma format corresponds to the 4:2:0 chroma format, and the width and height of the chroma component block correspond to half of the width and height of the luma component block, respectively. Fig. 11a is a diagram illustrating the positional relationship between luma samples and chroma samples in the 4:2:0 chroma format.

[0188] In the above Table 4, when chroma_format_idc is 2, the chroma format corresponds to the 4:2:2 chroma format, the width of the chroma component block corresponds to half the width of the luma component block, and the height of the chroma component block is the same as the height of the luma component block. Fig. 11b is a diagram illustrating the positional relationship between luma samples and chroma samples in the 4:2:2 chroma format.

[0189] In Table 4 above, when chroma_format_idc is 3, the chroma format corresponds to the 4:4:4 chroma format, and the width and height of the chroma component block are respectively equal to the width and height of the luma component block. Fig. 11c is a diagram illustrating the positional relationship between luma samples and chroma samples in the 4:4:4 chroma format.

[0190] In Table 4 above, SubWidthC and SubHeightC represent the ratio between luma samples and chroma samples. For example, when the width and height of a luma component block are CbWidth and CbHeight, respectively, the width and height of the corresponding chroma component block can be derived as (CbWidth / SubwidthC) and (CbHeight / SubHeightC), respectively.

[0191] As described with reference to FIG. 11, depending on the chroma format, the size of the chroma component block corresponding to the luma component block of the current block may be different.

[0192] Fig. 12 is a diagram illustrating the sizes of chroma component blocks according to chroma formats when the luma component block is a 64x128 block. As illustrated in Fig. 12, a chroma component block corresponding to a 64x128 luma component block may be a 32x64 block in a 4:2:0 chroma format, a 32x128 block in a 4:2:2 chroma format, and a 64x128 block in a 4:4:4 chroma format.

[0193] As described above, according to the conventional method described with reference to FIG. 9, in a single tree structure in which a luma component block and a chroma component block are divided into the same tree structure, when ISP is applied to a current block, the luma component block is divided into a plurality of subpartitions, but the chroma component block is not divided. For example, when the luma component block of the current block to which ISP is applied is a 64x128 block, the division direction is determined in the horizontal direction, so the luma component block is divided into four 64x32 subpartitions, and the width (64) and the height (32) of each subpartition are both less than or equal to the maximum transformation size (64). Therefore, transformation or inverse transformation can be performed on each of the subpartitions of the luma component block. However, as illustrated in FIG. 12, the chroma component block becomes a 32x64 block in the 4:2:0 chroma format, a 32x128 block in the 4:2:2 chroma format, or a 64x128 block in the 4:4:4 chroma format. For example, in the 4:2:2 chroma format and the 4:4:4 chroma format, the height (128) of the chroma component block becomes larger than the maximum transformation size (64), so transformation or inverse transformation for the chroma component block becomes impossible.

[0194] Hereinafter, various embodiments according to the present disclosure for solving the above problems are specifically described.

[0195] Example #1

[0196] Embodiment #1 of the present disclosure, in order to solve the above-described conventional problem, adaptively applies ISP to chroma component blocks based on the chroma format and / or the size of the chroma component blocks when ISP is applied to the current block. According to Embodiment #1 of the present disclosure, for example, when applying ISP to a chroma component block, the splitting direction and number of splits determined for a luma component block are also applied to the chroma component block in the same manner.

[0197] FIG. 13 is a flowchart illustrating an ISP application method according to one embodiment of the present disclosure.

[0198] Fig. 14 is a diagram illustrating an example of dividing a chroma component block according to the ISP application method of Fig. 13.

[0199] When a current block to be encoded / decoded is input (S1310), it can be determined whether ISP is to be applied to the current block (S1320). The determination in step S1320 may include determining whether ISP is available for the current block and / or determining whether ISP is to be applied to the current block. The determination of whether ISP is available for the current block may be performed based on the ISP availability condition described above. If ISP is available, the video encoding device may determine whether ISP is to be applied to the current block based on various methods as described above, and the determination result may be encoded in the bitstream as intra prediction technique information. If ISP is available, the video decoding device may determine whether ISP is to be applied to the current block based on signaled intra prediction technique information.

[0200] If ISP is not applied to the current block, TU tiling may be performed (S1330). Thereafter, encoding / decoding may be performed on each of the blocks divided through the current block or the TU tiling step (S1390). Steps S1330 and S1390 are identical to steps S930 and S960 of FIG. 9, and thus, a detailed description thereof will be omitted.

[0201] In step S1320, if ISP is applied to the current block, the split direction and the number of splits (the number of subpartitions) may be determined (S1340). The description of the determination of the split direction and the number of splits is the same as that described with reference to FIG. 9, and therefore is omitted below.

[0202] When the division direction and the number of divisions are determined in step S1340, division of the luma component block of the current block can be performed based on this (S1350).

[0203] Thereafter, it may be determined whether to apply ISP to a chroma component block corresponding to a luma component block (S1360). The determination in step S1360 may be performed by comparing the width and height of the chroma format and / or the corresponding chroma component block with the maximum transformation size.

[0204] For the 4:2:0 chroma format, as described below, the width and height of the chroma component block are less than or equal to the maximum transformation size. Therefore, for the 4:2:0 chroma format, it is possible to determine that ISP is not applied to the chroma component block without having to compare the width and height of the chroma component block with the maximum transformation size.

[0205] For 4:2:2 chroma format or 4:4:4 chroma format, as described below, the width or height of a chroma component block may be greater than the maximum transformation size. Therefore, for 4:2:2 chroma format or 4:4:4 chroma format, the width and height of the chroma component block may be compared with the maximum transformation size, and if the width or height of the chroma component block is greater than the maximum transformation size, it may be determined that ISP is applied to the chroma component block.

[0206] As illustrated in FIG. 14, when a luma component block is a 64x128 block, the corresponding chroma component block is a 32x64 block in a 4:2:0 chroma format, a 32x128 block in a 4:2:2 chroma format, or a 64x128 block in a 4:4:4 chroma format.

[0207] In the example illustrated in Fig. 14, it can be seen that both the width (32) and the height (64) of the chroma component block of the 4:2:0 chroma format are less than or equal to the maximum conversion size (64). Therefore, in the case of the 4:2:0 chroma format, it can be determined that ISP is not applied to the 32x64 chroma component block without additional size comparison.

[0208] On the other hand, in the case of 4:2:2 chroma format or 4:4:4 chroma format, the width and height of the chroma component block and the maximum transformation size can be additionally compared. In the example shown in Fig. 14, the width (128) of the 32x128 chroma component block of the 4:2:2 chroma format and the width (128) of the 64x128 chroma component block of the 4:4:4 chroma format are larger than the maximum transformation size (64), so it can be determined that the ISP is applied to the corresponding chroma component block.

[0209] As a variation of Example #1, regardless of the chroma format, it may be determined whether to apply ISP to a chroma component block based on whether both the width and height of the chroma component block are less than or equal to the maximum transformation size. For example, if both the width and height of the chroma component block are less than or equal to the maximum transformation size, it may be determined not to apply ISP to the chroma component block. Additionally, if the width or height of the chroma component block is greater than the maximum transformation size, it may be determined to apply ISP to the chroma component block.

[0210] If it is determined that the ISP is applied to the chroma component block, segmentation of the chroma component block may be performed (S1370). The segmentation of the chroma component block may be performed based on the segmentation direction and the number of segments determined in step S1340. Specifically, the segmentation direction and the number of segments for the chroma component block may be determined to be the same as the segmentation direction and the number of segments for the luma component block. According to Embodiment #1 of the present disclosure, in the example illustrated in FIG. 14, the luma component block is horizontally segmented into four subpartitions. Accordingly, a 32x128 chroma component block of a 4:2:2 chroma format and a 64x128 chroma component block of a 4:4:4 chroma format may also be horizontally segmented into four subpartitions in the same manner as the luma component block.

[0211] Thereafter, encoding / decoding may be performed for each of the divided subpartitions (S1390). In this case, the encoding / decoding of step S1390 may be performed for each of the subpartitions of the luma component block and each of the subpartitions of the chroma component block. Since step S1390 is the same as step S960, a detailed description thereof will be omitted.

[0212] If it is determined in step S1360 that the ISP is not applied to the chroma component block, no segmentation is performed on the chroma component block (S1380), and encoding / decoding may be performed on each subpartition of the luma component block and the unsplit chroma component block (S1390). Since step S1390 is the same as step S960, a detailed description thereof is omitted. According to embodiment #1 of the present disclosure, in the example illustrated in FIG. 14, a 32x64 chroma component block of a 4:2:0 chroma format may not be segmented.

[0213] According to Embodiment #1 of the present disclosure, if the width or height of the chroma component block of the current block to which ISP is applied is larger than the maximum transformation size, the problem of making transformation or inverse transformation impossible for the chroma component block can be solved by applying ISP to the chroma component block as well. In addition, according to Embodiment #1 of the present disclosure, if the chroma format of the current block to which ISP is applied is 4:2:0 or both the width and the height of the chroma component block of the current block to which ISP is applied are smaller than the maximum transformation size, the ISP is not applied to the chroma component block, thereby avoiding cases where ISP is unnecessarily applied to the chroma component block, and thus the amount of computation for encoding / decoding can be reduced. In addition, according to Embodiment #1 of the present disclosure, since the splitting direction and the number of splits of the ISP for the chroma component block are determined to be the same as the splitting direction and the number of splits of the ISP for the luma component block, there is no need to separately signal or derive the splitting direction and the number of splits of the ISP for the chroma component block.

[0214] Example #2

[0215] Embodiment #2 of the present disclosure, in order to solve the above-described conventional problem, applies ISP to chroma component blocks as well when ISP is applied to the current block. According to Embodiment #2 of the present disclosure, the division direction and number of divisions of the chroma component block can be determined based on the division direction and number of divisions determined for the luma component block.

[0216] FIG. 15 is a flowchart for explaining an ISP application method according to another embodiment of the present disclosure.

[0217] Fig. 16 is a diagram illustrating an example of dividing a chroma component block according to the ISP application method of Fig. 15.

[0218] When a current block to be encoded / decoded is input (S1510), it can be determined whether ISP is to be applied to the current block (S1520). The determination in step S1520 may include determining whether ISP is available for the current block and / or determining whether ISP is to be applied to the current block. The determination of whether ISP is available for the current block may be performed based on the ISP availability condition described above. If ISP is available, the video encoding device may determine whether ISP is to be applied to the current block based on various methods as described above, and the determination result may be encoded in the bitstream as intra prediction technique information. If ISP is available, the video decoding device may determine whether ISP is to be applied to the current block based on signaled intra prediction technique information.

[0219] If ISP is not applied to the current block, TU tiling may be performed (S1530). Thereafter, encoding / decoding may be performed on each of the current block or blocks segmented through the TU tiling step (S1570). Steps S1530 and S1570 are identical to steps S930 and S960 of FIG. 9, and thus a detailed description thereof will be omitted.

[0220] In step S1520, if ISP is applied to the current block, the split direction and the number of splits (the number of subpartitions) may be determined (S1540). The description of the determination of the split direction and the number of splits is the same as that described with reference to FIG. 9, and therefore is omitted below.

[0221] Based on the division direction and division number determined in step S1540, division of the luma component block of the current block can be performed (S1550).

[0222] Thereafter, segmentation of the chroma component block of the current block may be performed based on the segmentation direction and the number of segments determined in step S1540 (S1560). Specifically, the segmentation direction for the chroma component block may be determined to be the same as the segmentation direction for the luma component block. In addition, the number of segments for the chroma component block may be determined based on the number of segments for the luma component block. For example, when the number of segments for the luma component block is N, the number of segments for the chroma component block may be determined as N / n. In Embodiment #2 of the present disclosure, n may be 2, for example, but is not limited thereto, and n may be any integer. According to Embodiment #2 of the present disclosure, in the example illustrated in the upper part of FIG. 16, a 16x16 luma component block is horizontally segmented into four subpartitions. At this time, the segmentation direction of the corresponding chroma component block is horizontal, and the number of segments may be determined to be 2. That is, the corresponding chroma component block can be divided into two subpartitions in the horizontal direction regardless of the chroma format and / or the size of the chroma component block. In addition, in the example shown at the bottom of Fig. 16, the 4x8 luma component block is divided into two subpartitions in the vertical direction. At this time, the division direction of the corresponding chroma component block is the vertical direction, and the number of divisions can be determined as 1. In this case, since the chroma component block is not actually divided, it can be seen that the ISP is not applied to the chroma component block.

[0223] Thereafter, encoding / decoding may be performed on each subpartition or on the undivided chroma component block (S1570). In this case, the encoding / decoding of step S1570 may be performed on each subpartition of the luma component block and each subpartition of the chroma component block or the chroma component block. Since step S1570 is the same as step S960, a detailed description thereof will be omitted.

[0224] Embodiment #2 of the present disclosure is an embodiment that considers that when the size of the maximum coding unit is 128x128, the maximum transform size is 64, which is half of the width and height of the maximum coding unit, and the width and height of the current block are both larger than the maximum transform size, ISP is not available for the current block. According to Embodiment #2 of the present disclosure, the width and height of the chroma component block of the current block to which ISP is applied or the subpartition of the chroma component block are always smaller than the maximum transform size, so the problem of transformation or inverse transformation for the chroma component block becoming impossible can be solved. In addition, according to Embodiment #2 of the present disclosure, since there is no need to perform a judgment on whether to apply ISP to the chroma component block of the current block (e.g., the judgment in S1360), the amount of computation for encoding / decoding can be reduced. In addition, according to embodiment #2 of the present disclosure, since the number of subpartitions of a chroma component block is determined as 2 or 1, the encoding / decoding process of the chroma component block can be simplified. In addition, according to embodiment #2 of the present disclosure, since the splitting direction and the number of splits of the ISP for the chroma component block are determined to be the same as the splitting direction and (number of splits / 2) of the ISP for the luma component block, there is no need to separately signal or derive the splitting direction and the number of splits of the ISP for the chroma component block.

[0225] Example #3

[0226] Embodiment #3 of the present disclosure solves the above conventional problem by changing the ISP availability conditions so that the width and height of the chroma component block are less than or equal to the maximum transformation size.

[0227] According to the conventional ISP method described with reference to FIG. 9, if both the width and height of the current block are greater than the maximum transformation size, the ISP for the current block is determined to be unavailable. That is, if either the width or the height of the current block is greater than the maximum transformation size and the other is less than or equal to the maximum transformation size, the ISP for the current block is determined to be available.

[0228] Embodiment #3 of the present disclosure modifies the ISP availability conditions described above as shown in Table 5 below.

[0229] <Modified ISP availability conditions> - intra_luma_ref_idx[ x0 ][ y0 ] = = 0 - cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY - cbWidth * cbHeight > MinTbSizeY * MinTbSizeY

[0230] Among the above modified ISP availability conditions, the condition that intra_luma_ref_idx[x0][y0] is 0 and the condition that cbWidth * cbHeight is greater than MinTbSizeY * MinTbSizeY are the same as the existing ISP availability conditions. According to the above modified ISP availability conditions, ISP can be determined to be available for the current block only when both cbWidth and cbHeight of the current block are less than or equal to MaxTbSizeY.

[0231] According to embodiment #3 of the present disclosure to which the above-mentioned modified ISP availability condition is applied, the ISP is determined to be available for the current block only when both the width and height of the current block are less than or equal to the maximum transformation size, and the intra_subpartitions_mode_flag indicating whether the ISP is applied can be transmitted.

[0232] According to embodiment #3 of the present disclosure, since both the width and height of the current block are less than or equal to the maximum transform size, the width and / or height of the chroma sample block corresponding to the luma sample block of the current block always falls within the maximum transform size, regardless of the chroma format. Therefore, by applying the above-mentioned modified ISP availability conditions, the problems of the conventional ISP method described with reference to FIG. 9 can be resolved.

[0233] The method according to Embodiment #3 of the present disclosure can be implemented by modifying only the ISP availability conditions from the conventional method. Accordingly, the method according to Embodiment #3 of the present disclosure can be performed in the same manner as the flowchart illustrated in FIG. 9, with the only difference being the conditions for determining ISP availability in step S920.

[0234] Figure 17 is a diagram illustrating the structure of a bitstream reflecting modified ISP availability conditions.

[0235] The rectangular box portion of Fig. 17 relates to signaling of information regarding ISP among the intra prediction technique information of the current coding unit. As illustrated in Fig. 17, if the above-described modified ISP availability condition is satisfied, the intra_subpartitions_mode_flag indicating whether ISP is applied may be transmitted. The intra_subpartitions_mode_flag is information indicating whether ISP is applied to the current coding unit. If the ISP availability condition is not satisfied, the intra_subpartitions_mode_flag is not transmitted, and it may be determined that ISP is not applied to the current coding unit.

[0236] The above-described modified ISP availability conditions may include one or more conditions, and the one or more conditions included in the above-described modified ISP availability conditions are not limited to the above-described examples. That is, within the scope of the technical concepts of the present disclosure, some conditions may be omitted or other conditions may be additionally included.

[0237] Additionally, as illustrated in FIG. 17, when ISP is applied to the current block, i.e., when intra_subpartitions_mode_flag is 1, information indicating the splitting direction (intra_subpartitions_split_flag) can be transmitted.

[0238] In the example illustrated in Fig. 17, intra_subpartitions_mode_flag can be transmitted only when the condition of cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY is satisfied. That is, if the transmitted intra_subpartitions_mode_flag is 1, it can be seen that the condition of cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY is already satisfied. Accordingly, the condition of Fig. 17 related to whether intra_sinpartitions_split_flag is transmitted can be changed as in Fig. 18.

[0239] Figure 18 is a diagram illustrating another example of the structure of a bitstream reflecting modified ISP availability conditions.

[0240] The rectangular box portion of Fig. 18 relates to signaling of information regarding ISP among the intra prediction technique information of the current coding unit. As illustrated in Fig. 18, intra_subpartitions_mode_flag is signaled when the aforementioned modified ISP availability condition is satisfied, and intra_sinpartitions_split_flag can be signaled when the condition that intra_subpartitions_mode_flag is 1 is satisfied.

[0241] According to Embodiment #3 of the present disclosure, the width and height of the current block to which ISP is applied are both less than or equal to the maximum transformation size. Therefore, regardless of the chroma format, the width and height of the chroma component block of the current block are also both less than or equal to the maximum transformation size, so that the problem of not being able to transform or inversely transform the chroma component block can be solved. In addition, according to Embodiment #3 of the present disclosure, since there is no need to perform a process of determining whether to apply ISP to the chroma component block of the current block (e.g., determination in S1360) and performing ISP on the chroma component block, the amount of computation for encoding / decoding can be reduced. In addition, according to Embodiment #3 of the present disclosure, since the conventional encoding / decoding process is not significantly changed, the conventional problem can be solved without significantly increasing the complexity of encoding / decoding.

[0242] While the exemplary methods of this disclosure are presented as a series of operations for clarity of description, this is not intended to limit the order in which the steps are performed, and individual steps may be performed simultaneously or in different orders, if desired. To implement a method according to this disclosure, additional steps may be included in addition to the steps illustrated, some steps may be excluded and the remaining steps included, or some steps may be excluded and additional steps included.

[0243] In the present disclosure, a video encoding device or video decoding device performing a predetermined operation (step) may perform an operation (step) of checking the conditions or circumstances under which the operation (step) is performed. For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the video encoding device or video decoding device may perform an operation of checking whether the predetermined condition is satisfied and then perform the predetermined operation.

[0244] The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0245] Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general processors, controllers, microcontrollers, microprocessors, etc.

[0246] In addition, the video decoding device and the video encoding device to which the embodiments of the present disclosure are applied may be included in a multimedia broadcasting transmitting and receiving device, a mobile communication terminal, a home cinema video device, a digital cinema video device, a surveillance camera, a video conversation device, a real-time communication device such as a video communication, a mobile streaming device, a storage medium, a camcorder, a video-on-demand (VoD) service providing device, an OTT (Over the top video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a video phone video device, and a medical video device, and may be used to process a video signal or a data signal. For example, the OTT (Over the top video) device may include a game console, a Blu-ray player, an Internet-connected TV, a home theater system, a smartphone, a tablet PC, a DVR (Digital Video Recorder), and the like.

[0247] FIG. 19 is a diagram illustrating a content streaming system to which an embodiment of the present disclosure can be applied.

[0248] As illustrated in FIG. 19, a content streaming system to which an embodiment of the present disclosure is applied may largely include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0249] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data, generates a bitstream, and transmits it to the streaming server. Alternatively, if multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server may be omitted.

[0250] The above bitstream can be generated by an image encoding method and / or an image encoding device to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0251] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, and the streaming server can transmit multimedia data to the user. At this time, the content streaming system may include a separate control server, and in this case, the control server may control commands / responses between each device within the content streaming system.

[0252] The streaming server can receive content from a media repository and / or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0253] Examples of the user devices may include mobile phones, smart phones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs), digital TVs, desktop computers, digital signage, etc.

[0254] Each server within the above content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0255] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

[0256] Embodiments according to the present disclosure can be used to encode / decode images.

Claims

1. An image decoding method performed by an image decoding device, A step of determining whether the prediction mode of the current block is an intra prediction mode based on information about the prediction mode of the current block; A step of determining whether ISP (Intra Sub-Partitions) are available for the current block when the prediction mode of the current block is an intra prediction mode; If an ISP is available for the current block, a step of decoding an ISP application indicator indicating whether an ISP is applied to the current block; and An image decoding method comprising a step of applying ISP to the current block to generate a prediction block for the current block, if it is determined that ISP is applied to the current block based on the ISP application indicator.

2. In paragraph 1, The step of determining whether the ISP is available for the current block above is: An image decoding method performed based on a comparison between the size of a luma component block of the current block and a predetermined threshold value.

3. In paragraph 2, A method for decoding an image, wherein the above predetermined threshold value is the maximum transformation size.

4. In paragraph 2, An image decoding method in which an ISP determines that the current block is available when both the width and height of the luma component block of the current block are below a predetermined threshold.

5. In paragraph 1, The step of generating a prediction block for the current block by applying ISP to the current block is as follows: A step of applying ISP to a luma component block of the current block to generate a prediction block for the luma component block; A step of determining whether ISP is applied to the chroma component block of the current block; and An image decoding method comprising a step of applying ISP to the chroma component block to generate a prediction block for the chroma component block when it is determined that ISP is applied to the chroma component block of the current block.

6. In paragraph 5, The step of determining whether ISP is applied to the chroma component block of the current block above is as follows: A video decoding method performed based on at least one of the size of the chroma component block or the chroma format of the current block.

7. In paragraph 6, A video decoding method for determining that ISP is applied to a chroma component block of the current block when the chroma format of the current block is 4:2:2 or 4:4:4 and at least one of the width or height of the chroma component block is greater than the maximum transformation size.

8. In paragraph 5, When applying ISP to the above chroma component block, A video decoding method in which the division direction and the number of subpartitions of the above chroma component block are determined based on the division direction and the number of subpartitions of the above luma component block.

9. In paragraph 8, A method for decoding an image, wherein the division direction and the number of subpartitions of the chroma component block are the same as the division direction and the number of subpartitions of the luma component block, respectively.

10. In paragraph 1, The step of generating a prediction block for the current block by applying ISP to the current block is as follows: A step of applying ISP to a luma component block of the current block to generate a prediction block for the luma component block; and A video decoding method comprising a step of applying ISP to a chroma component block of the current block to generate a prediction block for the chroma component block.

11. In paragraph 10, A video decoding method in which the division direction and the number of subpartitions of the above chroma component block are determined based on the division direction and the number of subpartitions of the above luma component block.

12. In paragraph 11, A video decoding method in which the division direction and the number of subpartitions of the above chroma component block are the same as the division direction and (number of subpartitions / 2) of the above luma component block, respectively.

13. An image decoding device comprising a memory and at least one processor, At least one processor above Based on information about the prediction mode of the current block, determine whether the prediction mode of the current block is an intra prediction mode, If the prediction mode of the current block is an intra prediction mode, determine whether the ISP is available for the current block, If an ISP is available for the current block, decrypt an ISP application indicator indicating whether an ISP is applied to the current block; An image decoding device that applies ISP to the current block to generate a prediction block for the current block when it is determined that ISP is applied to the current block based on the ISP application indicator.

14. An image encoding method performed by an image encoding device, wherein the image encoding method comprises: A step of determining whether the prediction mode of the current block is an intra prediction mode; A step of determining whether an ISP is available for the current block when the prediction mode of the current block is an intra prediction mode; A step of determining whether an ISP is applicable to the current block, if an ISP is available for the current block; When ISP is applied to the current block, a step of generating a prediction block for the current block by applying ISP to the current block; and An image encoding method comprising a step of encoding information about a prediction mode of the current block and information about whether an ISP is applied to the current block.

15. A method for transmitting a bitstream generated by the video encoding method of Article 14.

Images