Image encoding / decoding method and apparatus and method of transmitting bitstream

Abstract

An image encoding / decoding method and apparatus and a method of transmitting a bitstream are provided. An image decoding method according to the disclosure can include determining whether a prediction mode of a current block is an intra prediction mode based on information about the prediction mode of the current block, determining whether an intra sub-partition (ISP) is available for the current block when the prediction mode of the current block is the intra prediction mode, decoding an ISP application indicator indicating whether the ISP is applied to the current block when the ISP is available for the current block, and generating a prediction block of the current block by applying the ISP to the current block when the ISP is determined to be applied to the current block based on the ISP application indicator.

Description

Technical Field

[0001] This disclosure relates to an image encoding / decoding method and apparatus, and a method for transmitting a bitstream, and more specifically, to a method and apparatus for encoding / decoding an image by applying intra-frame sub-block partitioning (ISP) to an image having various chroma formats, and a method for transmitting a bitstream generated by the image encoding method / apparatus of this disclosure. Background Technology

[0002] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) and ultra-high-definition (UHD) images, has been increasing across various fields. With the improvement in image data resolution and quality, the amount of information or bits transmitted increases relative to existing image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0003] Therefore, efficient image compression techniques are needed to effectively transmit, store, and reproduce information about high-resolution and high-quality images. Summary of the Invention

[0004] Technical issues

[0005] The purpose of this disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] The purpose of this disclosure is to provide a method and apparatus for encoding / decoding images by applying an ISP to images having various chroma formats.

[0007] Another object of this disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0008] Another object of this disclosure is to provide a recording medium for storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0009] Another object of this disclosure is to provide a recording medium that stores a bitstream received, decoded and used to reconstruct an image by an image decoding apparatus according to this disclosure.

[0010] The technical problems solved by this disclosure are not limited to those described above. Other technical problems not described herein will become clear to those skilled in the art through the following description.

[0011] Technical solution

[0012] According to one aspect of this disclosure, an image decoding method performed by an image decoding apparatus may include: determining whether the prediction mode of the current block is an intra-prediction mode based on information about the prediction mode of the current block; when the prediction mode of the current block is an intra-prediction mode, determining whether an intra-segment sub-partition (ISP) is available for the current block; when the ISP is available for the current block, indicating whether to apply an ISP application indicator to the current block; and generating a prediction block for the current block by applying the ISP to the current block when it is determined based on the ISP application indicator.

[0013] In the image decoding method disclosed herein, it can be determined whether the ISP can be used for the current block based on a comparison between the size of the luminance component block of the current block and a predetermined threshold.

[0014] In the image decoding method disclosed herein, the predetermined threshold can be the maximum transform size.

[0015] In the image decoding method disclosed herein, when the width and height of the luminance component block of the current block are both equal to or less than a predetermined threshold, it can be determined that the ISP can be used for the current block.

[0016] In the image decoding method disclosed herein, generating a prediction block for the current block by applying ISP to the current block may include: generating a prediction block for the luminance component block by applying ISP to the luminance component block of the current block, determining whether to apply ISP to the chrominance component block of the current block, and when it is determined that ISP should be applied to the chrominance component block of the current block, generating a prediction block for the chrominance component block by applying ISP to the chrominance component block.

[0017] In the image decoding method disclosed herein, determining whether to apply ISP to the chroma component blocks of the current block can be performed based on at least one of the size of the chroma component blocks or the chroma format of the current block.

[0018] In the image decoding method disclosed herein, when the chroma format of the current block is 4:2:2 or 4:4:4 and at least one of the width or height of the chroma component block is greater than the maximum transform size, the ISP applied to the chroma component block of the current block can be determined.

[0019] In the image decoding method disclosed herein, when the ISP is applied to the chroma component block, the segmentation direction and the number of sub-partitions for the chroma component block can be determined based on the segmentation direction and the number of sub-partitions for the luminance component block.

[0020] In the image decoding method disclosed herein, the segmentation direction and the number of sub-partitions for the chroma component block can be equal to the segmentation direction and the number of sub-partitions for the luminance component block, respectively.

[0021] In the image decoding method disclosed herein, generating a prediction block for the current block by applying ISP to the current block may include: generating a prediction block for the luminance component block by applying ISP to the luminance component block of the current block and generating a prediction block for the chrominance component block by applying ISP to the chrominance component block of the current block.

[0022] In the image decoding method disclosed herein, the segmentation direction and the number of sub-partitions for the chroma component block can be determined based on the segmentation direction and the number of sub-partitions for the luminance component block.

[0023] In the image decoding method disclosed herein, the segmentation direction and the number of sub-partitions for the chroma component block can be equal to the segmentation direction of the luminance component block and (the number of sub-partitions / 2), respectively.

[0024] An image decoding apparatus according to another aspect of this disclosure may include a memory and at least one processor. The at least one processor may determine, based on information about the prediction mode of the current block, whether the prediction mode of the current block is an intra-prediction mode; if the prediction mode of the current block is an intra-prediction mode, determine whether an intra-segment sub-partition (ISP) is available for the current block; if the ISP is available for the current block, indicate whether to apply an ISP application indicator to the current block; and, if it is determined based on the ISP application indicator that the ISP is applied to the current block, generate a prediction block for the current block by applying the ISP to the current block.

[0025] An image coding method according to another aspect of this disclosure may include: determining whether the prediction mode of the current block is an intra-prediction mode; when the prediction mode of the current block is an intra-prediction mode, determining whether an intra-segmentation sub-partition (ISP) is available for the current block; when the ISP is available for the current block, determining whether to apply the ISP to the current block; when the ISP is available for the current block, generating a prediction block for the current block by applying the ISP to the current block; and encoding information about the prediction mode of the current block and information about whether to apply the ISP to the current block.

[0026] According to another aspect of the transmission method of this disclosure, a bit stream generated by the image encoding apparatus or image encoding method of this disclosure can be transmitted.

[0027] According to another aspect of this disclosure, a computer-readable recording medium may store a bitstream generated by the image encoding apparatus or image encoding method of this disclosure.

[0028] According to another aspect of this disclosure, a computer-readable recording medium can store a bitstream that is received and decoded by an image decoding apparatus and used to reconstruct an image. The bitstream may include information about the prediction mode of the current block and an ISP application indicator indicating whether ISP is applied to the current block. The information about the prediction mode of the current block can be used to determine whether the prediction mode of the current block is an intra-prediction mode. When the prediction mode of the current block is an intra-prediction mode and it is determined that ISP is available for the current block, the ISP application indicator can be used to determine whether ISP is applied to the current block, and a prediction block for the current block is generated by performing ISP on the current block when it is determined that ISP is applied to the current block.

[0029] The features described above in this brief overview are merely exemplary aspects of the following detailed description of this disclosure and do not limit the scope of this disclosure.

[0030] Beneficial effects

[0031] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0032] Furthermore, according to this disclosure, a method and apparatus for encoding / decoding images by applying an ISP to images having various chroma formats can be provided.

[0033] Furthermore, according to this disclosure, a method for transmitting a bitstream generated by an image encoding method or apparatus according to this disclosure can be provided.

[0034] Furthermore, according to this disclosure, it is possible to provide a recording medium for storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0035] Furthermore, according to this disclosure, it is possible to provide a recording medium that stores a bitstream that is received, decoded, and used to reconstruct an image by an image decoding apparatus according to this disclosure.

[0036] Those skilled in the art will understand that the effects achievable through this disclosure are not limited to those specifically described above, and that other advantages of this disclosure will become clearer from the detailed description. Attached Figure Description

[0037] Figure 1 This is a view schematically illustrating a video compilation system to which embodiments of this disclosure are applicable.

[0038] Figure 2 This is a schematic view illustrating an image encoding apparatus to which embodiments of the present disclosure are applicable.

[0039] Figure 3 This is a schematic view illustrating an image decoding apparatus to which embodiments of the present disclosure are applicable.

[0040] Figure 4 This is a flowchart illustrating a video / image coding method based on intra-frame prediction.

[0041] Figure 5 This is a view illustrating the configuration of the intra-predictor 185 according to this disclosure.

[0042] Figure 6 This is a flowchart illustrating a video / image decoding method based on intra-frame prediction.

[0043] Figure 7 This is a view illustrating the configuration of the intra-predictor 265 according to this disclosure.

[0044] Figure 8a and 8b This is a view illustrating the intra-frame prediction direction according to an embodiment of the present disclosure.

[0045] Figure 9 This is a view illustrating the traditional method of encoding / decoding the current block according to the ISP mode.

[0046] Figure 10 is a view illustrating examples of ISP segmentation for current blocks of various sizes.

[0047] Figure 11 is a view illustrating the relationship between the luminance component blocks (luminance component array) and the chrominance component blocks (chrominance component array) according to the chrominance format.

[0048] Figure 12 This is a view illustrating the size of the chroma component blocks according to the chroma format when the luminance component blocks are 64x128 blocks.

[0049] Figure 13 This is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[0050] Figure 14 The diagram is based on Figure 13 A view of an example of the ISP application method for segmenting chroma component blocks.

[0051] Figure 15 This is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[0052] Figure 16 The diagram is based on Figure 15 A view of an example of the ISP application method for segmenting chroma component blocks.

[0053] Figure 17 This is a view illustrating an example of the structure of a bitstream that reflects modified ISP availability conditions.

[0054] Figure 18This is a view illustrating another example of the structure of a bitstream that reflects modified ISP availability conditions.

[0055] Figure 19 This is a view illustrating a content streaming system to which embodiments of this disclosure are applicable. Detailed Implementation

[0056] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, this disclosure can be implemented in various different forms and is not limited to the embodiments described herein.

[0057] In describing this disclosure, detailed descriptions of relevant known functions or constructions will be omitted if they unnecessarily obscure the scope of this disclosure. In the accompanying drawings, portions irrelevant to the description of this disclosure are omitted, and similar reference numerals are assigned to similar portions.

[0058] In this disclosure, when a component is "connected," "coupled," or "linked" to another component, it can include not only direct connections but also indirect connections where intermediate components exist. Furthermore, when a component "comprises" or "has" other components, unless otherwise stated, it means that other components may be included, not excluded.

[0059] In this disclosure, the terms first, second, etc., are used only for the purpose of distinguishing one component from other components and do not limit the order or importance of the components unless otherwise stated. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0060] In this disclosure, the components are distinguished from each other to clearly describe each feature, but this does not mean that the components must be separate. That is, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed and implemented across multiple hardware or software units. Therefore, unless otherwise specified, embodiments of integrated or distributed components are included within the scope of this disclosure.

[0061] In this disclosure, the components described in the various embodiments are not necessarily essential components, and some components may be optional. Therefore, embodiments comprising a subset of the components described in the embodiments are also included within the scope of this disclosure. Furthermore, embodiments that include other components besides those described in the various embodiments are also included within the scope of this disclosure.

[0062] This disclosure relates to the encoding and decoding of images. Unless redefined in this disclosure, the terms used herein may have the general meaning commonly used in the art to which this disclosure pertains.

[0063] In this disclosure, "image" generally refers to a unit representing an image within a specific time period, while a slice / tile is a compilation unit that constitutes a part of an image. An image can be composed of one or more slices / tiles. Furthermore, a slice / tile may include one or more compilation tree units (CTUs).

[0064] In this disclosure, "pixel" or "cell" can refer to the smallest single element that constitutes a picture (or image). Furthermore, "sample" can be used as a term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, or it can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0065] In this disclosure, "unit" can refer to a basic unit of image processing. A unit may include at least one of a specific region of an image and information associated with that region. In some cases, the unit may be used interchangeably with terms such as "sample array," "block," or "region." In general, an M×N block may include a set (or array) or sample (or sample array) of transform coefficients with M columns and N rows.

[0066] In this disclosure, "current block" can mean one of "current compilation block," "current compilation unit," "compilation target block," "decoding target block," or "processing target block." When performing prediction, "current block" can mean "current prediction block" or "prediction target block." When performing transform (inverse transform) / quantization (dequantization), "current block" can mean "current transform block" or "transform target block." When performing filtering, "current block" can mean "filter target block."

[0067] In this disclosure, the forward slash " / " or "," can be interpreted as indicating "and / or". For example, "A / B" and "A, B" can mean "A and / or B". Furthermore, "A / B / C" and "A / B / C" can mean "at least one of A, B and / or C".

[0068] In this disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in this disclosure, the word "or" should be interpreted to indicate "additionally or alternatively".

[0069] Video coding system overview

[0070] Figure 1This is a schematic view of a video compilation system according to this disclosure.

[0071] The video compilation system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver encoded video and / or image information or data to the decoding device 20 in the form of a file or stream via a digital storage medium or network.

[0072] The encoding apparatus 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding apparatus 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0073] The video source generator 11 can acquire video / images through a process of capturing, compositing, or generating video / images. The video source generator 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device may include, for example, a computer, tablet computer, and smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc. In this case, the video / image capture process can be replaced by a process of generating related data.

[0074] The encoding unit 12 can encode the input video / image. For compression and compilation efficiency, the encoding unit 12 can perform a series of processes, such as prediction, transformation, and quantization. The encoding unit 12 can output encoded data (encoded video / image information) in the form of a bitstream.

[0075] The transmitter 13 can transmit encoded video / image information or data output as a bitstream to the receiver 21 of the decoding device 20 in the form of a file or stream via a digital storage medium or network. The digital storage medium can include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver 21 can extract / receive the bitstream from the storage medium or network and transmit the bitstream to the decoding unit 22.

[0076] The decoding unit 22 can decode video / images by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transform, and prediction.

[0077] Renderer 23 can render decoded video / images. The rendered video / images can be displayed on a monitor.

[0078] Image encoding device overview

[0079] Figure 2 This is a view schematically illustrating an image encoding apparatus to which embodiments of the present disclosure may be applied.

[0080] like Figure 2 As shown, the image encoding apparatus 100 may include an image partitioner 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as "predictors". The transformer 120, quantizer 130, dequantizer 140, and inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0081] In some embodiments, all or at least some of the components of the image encoding apparatus 100 may be configured by a single hardware component (e.g., an encoder or a processor). Furthermore, the memory 170 may include a decoded image buffer (DPB) and may be configured by a digital storage medium.

[0082] Image partitioner 110 can partition an input image (or picture or frame) input to image encoding device 100 into one or more processing units. For example, a processing unit may be called a compilation unit (CU). A compilation unit can be obtained by recursively partitioning a compilation tree unit (CTU) or a maximum compilation unit (LCU) according to a quadtree-binary-tritree (QT / BT / TT) structure. For example, a compilation unit can be partitioned into multiple compilation units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For partitioning a compilation unit, a quadtree structure can be applied first, followed by a binary tree structure and / or a ternary tree structure. The compilation process according to this disclosure can be performed based on the final compilation unit that is no longer partitioned. The maximum compilation unit can be used as the final compilation unit, or a deeper compilation unit obtained by partitioning the maximum compilation unit can be used as the final compilation unit. Here, the compilation process may include prediction, transformation, and reconstruction processes, which will be described later. As another example, the processing unit of the compilation process may be a prediction unit (PU) or a transformation unit (TU). Prediction and transform units can be split or partitioned from the final compilation unit. The prediction unit can be a sample prediction unit, and the transform unit can be a unit for deriving transform coefficients and / or a unit for deriving residual signals from transform coefficients.

[0083] The predictor (inter-frame predictor 180 or intra-frame predictor 185) can perform prediction on the block to be processed (the current block) and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The predictor can generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and output as a bitstream.

[0084] Intra-predictor 185 can predict the current block by referencing samples in the current image. Depending on the intra-prediction mode and / or intra-prediction technique, the reference samples may be located among the neighbors of the current block or may be placed separately. Intra-prediction modes may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC modes and planar modes. Depending on the level of detail in the prediction direction, directional modes may include, for example, 33 or 65 directional prediction modes. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-predictor 185 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0085] Inter-frame predictor 180 can derive the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc. The reference image including the temporally neighboring block may be referred to as a juxtaposed image (colPic). For example, inter-frame predictor 180 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to derive the motion vector and / or reference image index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame predictor 180 can use motion information from neighboring blocks as motion information for the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, motion vectors from neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can refer to the difference between the motion vector of the current block and the motion vector predictor.

[0086] The predictor can generate a prediction signal based on various prediction methods and techniques described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction, but also both intra-frame and inter-frame prediction simultaneously to predict the current block. The prediction method that applies both intra-frame and inter-frame prediction simultaneously to predict the current block can be called Combined Intra-Frame and Inter-Frame Prediction (CIIP). Furthermore, the predictor can perform Intra-Block Copy (IBC) to predict the current block. Intra-Block Copy can be used for content image / video compilation in games, such as Screen Content Compilation (SCC). IBC is a method of predicting the current image using a previously reconstructed reference block in the current image at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current image can be encoded as a vector (block vector) corresponding to the predetermined distance.

[0087] The predicted signal generated by the predictor can be used to generate a reconstructed signal or a residual signal. Subtractor 115 generates a residual signal (residual block or residual sample array) by subtracting the predicted signal (predicted block or predicted sample array) output from the predictor from the input image signal (original block or original sample array). The generated residual signal can be transmitted to converter 120.

[0088] Transformer 120 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented graphically. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size or to blocks of variable size instead of square.

[0089] Quantizer 130 quantizes the transform coefficients and transmits them to entropy encoder 190. Entropy encoder 190 encodes the quantized signal (information about the quantized transform coefficients) and outputs a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 130 can rearrange the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and generate information about the quantized transform coefficients based on the one-dimensional vector form of the quantized transform coefficients.

[0090] The entropy encoder 190 can perform various encoding methods, such as exponential Columbus coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 190 can encode, either together or separately, the information required for video / image reconstruction (e.g., values ​​of syntax elements, etc.), in addition to the quantization transform coefficients. The encoded information (e.g., encoded video / image information) can be transmitted or stored in units of network abstraction layers (NALs) in the form of a bitstream. The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). Furthermore, the video / image information may also include general constraint information. The signaled information, transmitted information, and / or syntax elements described in this disclosure can be encoded and included in the bitstream through the above encoding process.

[0091] The bitstream can be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external components of the image encoding apparatus 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0092] The quantized transform coefficients output from quantizer 130 can be used to generate residual signals. For example, the residual signals (residual blocks or residual samples) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 140 and inverse transformer 150.

[0093] Adder 155 adds the reconstructed residual signal to the prediction signal output from inter-frame predictor 180 or intra-frame predictor 185 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as in the case of applying skip mode, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current image, and can be used for inter-frame prediction of the next image by filtering as described below.

[0094] Additionally, as described below, Luminance Mapping with Chroma Scaling (LMCS) can be applied during image encoding.

[0095] Filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 170, specifically in the DPB of memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. Filter 160 can generate various filtering-related information and transmit the generated information to entropy encoder 190, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 190 and output as a bitstream.

[0096] The modified reconstructed image transferred to memory 170 can be used as a reference image in inter-frame predictor 180. When inter-frame prediction is applied by image coding device 100, prediction mismatch between image coding device 100 and image decoding device can be avoided and coding efficiency can be improved.

[0097] The DPB of memory 170 can store modified reconstructed images for use as reference images in inter-frame predictor 180. Memory 170 can store motion information of blocks from which motion information in the current image is derived (or encoded) and / or motion information of already reconstructed blocks in the image. The stored motion information can be transmitted to inter-frame predictor 180 and used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current image and can transmit the reconstructed samples to intra-frame predictor 185.

[0098] Image decoding device overview

[0099] Figure 3 This is a schematic view illustrating an image decoding apparatus to which embodiments of the present disclosure may be applied.

[0100] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as "predictors". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0101] According to an embodiment, all or at least some of the components of the image decoding device 200 may be configured by hardware components (e.g., a decoder or a processor). Furthermore, the memory 250 may include a decoded image buffer (DPB) or may be configured by a digital storage medium.

[0102] The image decoding device 200, having received a bitstream including video / image information, can perform operations related to... Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding apparatus 100. For example, the image decoding apparatus 200 can perform decoding using a processing unit applied in the image encoding apparatus. Therefore, the decoding processing unit can be, for example, a compilation unit. The compilation unit can be obtained by partitioning a compilation tree unit or a maximum compilation unit. The reconstructed image signal decoded and output by the image decoding apparatus 200 can be reproduced by a reproduction apparatus (not shown).

[0103] Image decoding device 200 can receive data in bitstream form from... Figure 2The signal output by the image encoding apparatus. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The image decoding apparatus can also decode the picture based on the parameter set information and / or general constraint information. The information and / or syntax elements notified / received by signals described in this disclosure can be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on a compilation method such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine the context model using information about the target syntax element, the decoding information of neighboring blocks and the target block, or information about symbols / bins decoded in the previous stage, and perform arithmetic decoding on the bins based on the determined context model by predicting the occurrence probability of the bins, generating symbols corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model after determining the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 210 can be provided to the predictors (inter-frame predictor 260 and intra-frame predictor 265), and the residual values ​​of the entropy decoding performed in the entropy decoder 210, i.e., the quantized transform coefficients and related parameter information, can be input to the dequantizer 220. In addition, the filtering information in the information decoded by the entropy decoder 210 can be provided to the filter 240. Meanwhile, the receiver (not shown) for receiving the signal output from the image encoding device can be further configured as an internal / external element of the image decoding device 200, or the receiver can be a component of the entropy decoder 210.

[0104] Meanwhile, the image decoding apparatus according to this disclosure can be referred to as a video / image / picture decoding apparatus. The image decoding apparatus can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 160, or an intra-frame predictor 265.

[0105] The dequantizer 220 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the image encoding device. The dequantizer 220 can obtain the transform coefficients by performing dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information).

[0106] The inverse transformer 230 can perform inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0107] The predictor can perform predictions on the current block and generate a prediction block that includes prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on information about the predictions output from the entropy decoder 210, and can determine a specific intra-frame / inter-frame prediction mode (prediction technique).

[0108] Similar to the predictor described in the image coding apparatus 100, the predictor can generate a prediction signal based on various prediction methods (techniques) described later.

[0109] Intra-predictor 265 can predict the current block by referring to samples in the current image. The description of intra-predictor 185 also applies to intra-predictor 265.

[0110] Inter-frame predictor 260 can derive the predicted block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference image. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current image and temporally neighboring blocks existing in the reference image. For example, inter-frame predictor 260 can configure a motion information candidate list based on neighboring blocks and derive the motion vector and / or reference image index of the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0111] Adder 235 can generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the predicted signal (predicted block, predicted sample array) output from the predictor (including inter-frame predictor 260 and / or intra-frame predictor 265). The description of adder 155 also applies to adder 235.

[0112] Additionally, as described below, Luminance Mapping with Chroma Scaling (LMCS) can be applied during image decoding.

[0113] Filter 240 can improve the quality of subjective / objective images by applying filtering to the reconstructed signal. For example, filter 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 250, specifically in the DPB of memory 250. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0114] The (modified) reconstructed image stored in the DPB of memory 250 can be used as a reference image in inter-frame predictor 260. Memory 250 can store motion information of blocks from which motion information in the current image is derived (or decoded) and / or motion information of already reconstructed blocks in the image. The stored motion information can be transmitted to inter-frame predictor 260 to be used as motion information for spatially or temporally neighboring blocks. Memory 250 can store reconstructed samples of reconstructed blocks in the current image and transmit the reconstructed samples to intra-frame predictor 265.

[0115] In this disclosure, the embodiments described in the filter 160, inter-frame predictor 180 and intra-frame predictor 185 of the image encoding apparatus 100 can be applied equally or correspondingly to the filter 240, inter-frame predictor 260 and intra-frame predictor 265 of the image decoding apparatus 200.

[0116] Intra prediction overview

[0117] The following describes an intra-frame prediction method according to an embodiment.

[0118] Intra-frame prediction can instruct the generation of prediction samples for the current block based on reference samples in the image to which the current block belongs (hereinafter referred to as the current image). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. The neighboring reference samples of the current block may include samples adjacent to the left boundary of the current block of size nW×nH and a total of 2×nH samples adjacent to the lower left, samples adjacent to the top boundary of the current block and a total of 2×nW samples adjacent to the upper right, and one sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of top neighbor samples and multiple rows of left neighbor samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the lower right of the current block.

[0119] Some neighboring reference samples of the current block have not yet been decoded or may be unavailable. In this case, the decoder can construct neighboring reference samples for prediction by replacing unavailable samples with available samples. Alternatively, interpolation of available samples can be used to construct neighboring reference samples for prediction.

[0120] When deriving neighboring reference samples, (i) the predicted sample can be derived based on the average or interpolation of the neighboring reference samples of the current block, and (ii) the predicted sample can be derived based on a reference sample present in a specific (predictive) direction relative to the predicted sample in the neighboring reference samples of the current block. Case (i) can be referred to as non-directional mode or non-angular mode, and case (ii) can be referred to as directional mode or angular mode.

[0121] Furthermore, prediction samples can be generated based on the predicted target sample of the current block in the neighboring reference samples by interpolating with the first neighboring sample in the prediction direction of the intra-prediction mode of the current block and the second neighboring sample in the opposite direction. This situation can be called linear interpolation intra-prediction (LIP).

[0122] In addition, linear models can be used to generate chromaticity prediction samples based on luminance samples. This can be called the linear model (LM) mode.

[0123] Additionally, a provisional prediction sample for the current block can be derived based on filtered neighboring reference samples, and the prediction sample for the current block can be derived by weighted summing the provisional prediction sample with at least one reference sample derived from the intra-prediction pattern in the existing neighboring reference samples (i.e., unfiltered neighboring reference samples). This process can be referred to as position-dependent intra-prediction (PDPC).

[0124] Alternatively, the reference sample line with the highest prediction accuracy can be selected from multiple neighboring reference sample lines in the current block to derive the prediction sample using reference samples located in the prediction direction of the corresponding line. In this case, information about the reference sample line used is encoded as a bitstream and transmitted as a signal (e.g., intra_luma_ref_idx). This can be referred to as multi-reference line (MRL) intra-prediction or MRL-based intra-prediction.

[0125] Alternatively, the current block can be divided into vertical or horizontal sub-partitions to perform intra-prediction relative to each sub-partition based on the same intra-prediction mode. In this case, neighboring reference samples for intra-prediction can be derived on a sub-partition basis. That is, reconstructed samples from previous sub-partitions in encoding / decoding order can be used as neighboring reference samples for the current sub-partition. In this scenario, the intra-prediction mode of the current block is applied equally to the sub-partitions, and neighboring reference samples are derived and used on a sub-partition basis, thereby improving intra-prediction performance. This prediction method can be called intra-partitioning (ISP) or ISP-based intra-prediction.

[0126] Intra-prediction techniques can be referred to by various terms, such as intra-prediction type or additional intra-prediction mode, to distinguish them from directional or non-directional intra-prediction modes. For example, intra-prediction techniques (intra-prediction type or additional intra-prediction mode) may include at least one of LIP, LM, PDPC, MRL, ISP, or MIP. Additionally, post-filtering may be performed on the derived prediction samples if necessary.

[0127] Specifically, the intra-frame prediction process may include an intra-frame prediction mode / type determination step, a neighboring reference sample derivation step, and a prediction sample derivation step based on the intra-frame prediction mode / type. Furthermore, if necessary, the derived prediction samples may be post-filtered.

[0128] Figure 4 This is a flowchart illustrating a video / image coding method based on intra-frame prediction.

[0129] Figure 4 Image encoding methods can be derived from Figure 2 The image encoding apparatus performs this operation. Specifically, step S410 can be performed by the intra-predictor 185, and step S420 can be performed by the residual processor. Specifically, step S420 can be performed by the subtractor 115. Step S430 can be performed by the entropy encoder 190. The prediction information of step S430 can be derived by the intra-predictor 185, and the residual information of step S430 can be processed by the residual processor. The residual information is information about the residual samples. The residual information may include information about the quantized transform coefficients of the residual samples. As described above, the residual samples can be derived into transform coefficients by the transformer 120 of the image encoding apparatus, and the transform coefficients can be derived into transform coefficients quantized by the quantizer 130. Information about the quantized transform coefficients can be encoded by the entropy encoder 190 through the residual compilation process.

[0130] The image coding apparatus can perform intra-prediction for the current block (S410). The image coding apparatus can determine the intra-prediction mode / type of the current block, derive the neighboring reference samples of the current block, and generate prediction samples in the current block based on the intra-prediction mode / type and the neighboring reference samples. Here, the processes of determining the intra-prediction mode / type, deriving the neighboring reference samples, and generating the prediction samples can be performed simultaneously, or any one of these processes can be performed before the other processes.

[0131] Figure 5 This is a view illustrating the configuration of the intra-predictor 185 according to this disclosure.

[0132] like Figure 5 As shown, the intra-predictor 185 of the image coding apparatus may include an intra-prediction mode / type determination unit 186, a reference sample derivation unit 187, and / or a prediction sample derivation unit 188. The intra-prediction mode / type determination unit 186 can determine the intra-prediction mode / type of the current block. The reference sample derivation unit 187 can derive neighboring reference samples of the current block. The prediction sample derivation unit 188 can derive prediction samples for the current block. Additionally, although not shown, the intra-predictor 185 may further include a prediction sample filter (not shown) when performing the prediction sample filtering process described below.

[0133] The image coding apparatus can determine the mode / type to apply to the current block from among multiple intra prediction modes / types. The image coding apparatus can compare the rate-distortion (RD) costs of the intra prediction modes / types and determine the optimal intra prediction mode / type for the current block.

[0134] Simultaneously, the image encoding device can perform a predictive sample filtering process. Predictive sample filtering can also be called post-filtering. Through this process, some or all of the predicted samples can be filtered. In some cases, the predictive sample filtering process can be omitted.

[0135] Refer again Figure 4 The image coding device can generate residual samples for the current block based on predicted samples or filtered predicted samples (S420). The image coding device can derive residual samples by subtracting predicted samples from the original samples of the current block. That is, the image coding device can derive residual sample values ​​by subtracting the corresponding predicted sample values ​​from the original sample values.

[0136] An image coding apparatus can encode image information, including information about intra-frame prediction (prediction information) and residual information of residual samples. The prediction information may include intra-frame prediction mode information and / or intra-frame prediction technique information. The image coding apparatus can output the encoded image information as a bitstream. The output bitstream can be transmitted to an image decoding apparatus via a storage medium or network.

[0137] Residual information may include residual compilation syntax, which will be described later. The image encoding device can transform / quantize the residual samples and derive the quantized transform coefficients. The residual information may include information about the quantized transform coefficients.

[0138] Simultaneously, as described above, the image encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks). To this end, the image encoding device can perform dequantization / inverse transform on the quantized transform coefficients and derive (modified) residual samples. The reason for transforming / quantizing the residual samples and then performing dequantization / inverse transform is to derive the same residual samples as those derived by the image decoding device. The image encoding device can generate a reconstructed block, including reconstructed samples for the current block, based on the predicted samples and the (modified) residual samples. Based on the reconstructed blocks, a reconstructed image for the current image can be generated. As described above, the in-loop filtering process is further applied to the reconstructed image.

[0139] Figure 6 This is a flowchart illustrating a video / image decoding method based on intra-frame prediction.

[0140] The image decoding device can perform operations corresponding to those performed by the image encoding device.

[0141] Figure 6 The image decoding device can be made by Figure 3 The image decoding apparatus executes steps S610 to S630. Steps S610 to S630 can be executed by the intra-frame predictor 265, and the prediction information of step S610 and the residual information of step S640 can be obtained from the bitstream by the entropy decoder 210. The residual processor of the image decoding apparatus can derive residual samples of the current block based on the residual information (S640). Specifically, the dequantizer 220 of the residual processor can perform dequantization based on the dequantized transform coefficients derived according to the residual information to derive transform coefficients, and the inverse transformer 230 of the residual processor can perform inverse transform on the transform coefficients to derive residual samples for the current block. Step S650 can be executed by the adder 235 or the reconstructor.

[0142] Specifically, the image decoding device can derive the intra-prediction mode / type for the current block based on the received prediction information (intra-prediction mode / type information) (S610). The image decoding device can derive the neighboring reference samples of the current block (S620). Based on the intra-prediction mode / type and the neighboring reference samples, the image decoding device can generate prediction samples in the current block (S630). In this case, the image decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Through the prediction sample filtering process, some or all prediction samples can be filtered. In some cases, the prediction sample filtering process can be omitted.

[0143] The image decoding device can generate residual samples for the current block based on the received residual information (S640). The image decoding device can generate reconstruction samples for the current block based on the predicted samples and residual samples, and derive a reconstruction block including the reconstruction samples (S650). Based on the reconstruction block, a reconstructed image for the current image can be generated. As described above, the in-loop filtering process is further applied to the reconstructed image.

[0144] Figure 7 This is a view illustrating the configuration of the intra-predictor 265 according to this disclosure.

[0145] like Figure 7 As shown, the intra-predictor 265 of the image decoding apparatus may include an intra-prediction mode / type determination unit 266, a reference sample derivation unit 267, and a prediction sample derivation unit 268. The intra-prediction mode / type determination unit 266 can determine the intra-prediction mode / type for the current block based on intra-prediction mode / type information generated by the intra-prediction mode / type determination unit 186 of the image encoding apparatus and transmitted by a signal. The reference sample derivation unit 267 can derive neighboring reference samples of the current block from the reconstructed reference region in the current image. The prediction sample derivation unit 268 can derive prediction samples for the current block. Meanwhile, although not shown, when performing the above-described prediction sample filtering process, the intra-predictor 265 may further include a prediction sample filter (not shown).

[0146] Intra-luma_mpm_flag may include, for example, flag information indicating whether the most probable mode (MPM) or remaining modes are applied to the current block. When the MPM is applied to the current block, the intra-luma_mpm_idx may further include index information indicating one of the intra-luma_mpm_candidates. The intra-luma_mpm_candidates may consist of a list of MPM candidates or a list of MPMs. Furthermore, when the MPM is not applied to the current block, the intra-luma_mpm_remainder may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra-luma_mpm_candidates besides the MPM candidates. The image decoding apparatus may determine the intra-luma_mpm_mode for the current block based on the intra-luma_mpm_remainder information. MPM candidate modes may include the intra-luma_mpm_modes of the current block's neighboring blocks (e.g., the left neighboring block and the top neighboring block) and additional candidate modes.

[0147] Figure 8aThis illustrates intra-frame prediction direction according to an embodiment of the present disclosure. This is to capture any edge directions presented in natural video, such as... Figure 8a As shown, the intra-prediction modes can include two non-directional intra-prediction modes and 65 directional intra-prediction modes. The non-directional intra-prediction modes can include a planar intra-prediction mode (planar mode) and a DC intra-prediction mode (DC mode), and the directional intra-prediction modes can include intra-prediction modes 2 to 66.

[0148] In addition to the intra-prediction modes mentioned above, intra-prediction modes can further include a Cross-Component Linear Model (CCLM) mode for chroma samples. The CCLM mode can be segmented into L_CCLM, T_CCLM, and LT_CCLM based on the LM parameters to determine whether left samples, top samples, or both are considered, and can be applied only to the chroma components.

[0149] For example, intra-frame prediction modes can be indexed as shown in the table below.

[0150] [Table 1]

[0151]

[0152] Figure 8b Intra-frame prediction direction is shown according to another embodiment of this disclosure. Figure 8b In the diagram, the dashed line indicates that the wide-angle mode applies only to non-square blocks. For example... Figure 8b As shown, in order to capture any edge orientations presented in natural video, the intra-prediction modes according to embodiments may include two non-directional intra-prediction modes and 93 directional intra-prediction modes. The non-directional intra-prediction modes may include a planar mode and a DC mode, and the directional intra-prediction modes may include intra-prediction modes 2 to 80 and -1 to -14, such as... Figure 8bThe arrow indicates the mode. Planar mode can be represented by INTRA_PLANAR, and DC mode can be represented by INTRA_DC. Furthermore, directional intra-prediction modes can be represented by INTRA_ANGULAR-14 to INTRA_ANGULAR-1 and INTRA_ANGULAR2 to INTRA_ANGULAR80. In addition, intra-prediction technique information can be implemented in various forms. For example, intra-prediction technique information may include intra-prediction type index information indicating one of the intra-prediction techniques. As another example, intra-prediction technique information may include at least one of the following: reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block and, if applied, which reference sample line is used; ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block; ISP type information (e.g., intra_subpartitions_split_flag) indicating the subpartition split type when ISP is applied; flag information indicating whether PDPC is applied, or flag information indicating whether LIP is applied. In this disclosure, ISP identification information may be referred to as ISP application indicator.

[0153] Intra-prediction mode information and / or intra-prediction technique information can be encoded / decoded using the compilation methods described in this disclosure. For example, intra-prediction mode information and / or intra-prediction technique information can be encoded / decoded using entropy compilation based on truncated (Rice) binary code (e.g., CABAC, CAVLC).

[0154] Before determining whether to apply a predetermined intra-prediction technique to the current block, it can first be determined whether the corresponding intra-prediction technique is available for the current block. For example, the availability of the corresponding intra-prediction technique can be determined based on the compiler parameters of the current block. In this case, the compiler parameters may include the size (width and / or height) of the current block, the position of the current block, the color components of the current block, or whether to apply another intra-prediction technique.

[0155] Additionally, the determination of whether a corresponding intra-prediction technique is available for the current block can be performed based on information signaled at a higher level than the current block, such as sequences, images, slices, and CTUs. For example, when information sent at the sequence level indicates that a pre-determined intra-prediction technique is unavailable, it can be determined that the corresponding intra-prediction technique is unavailable for blocks belonging to the corresponding sequence.

[0156] When it is determined that a predetermined intra-prediction technique is available for the current block, the image coding device can use various methods to determine whether to apply the corresponding intra-prediction technique to the current block. For example, the image coding device can determine whether to apply the corresponding intra-prediction technique based on RDO.

[0157] Image encoding devices can encode and signal intra-prediction technique information within the bitstream. Intra-prediction technique information can be encoded in various forms according to the corresponding intra-prediction technique. For example, when the intra-prediction technique is MRL, the intra-prediction technique information can be an index of a reference line among multiple reference lines used for prediction of the current block (e.g., intra_luma_ref_idx). When the intra-prediction technique is ISP, the intra-prediction technique information can be flag information indicating whether ISP is applied to the current block (e.g., intra_subpartitions_mode_flag). Additionally, when ISP is applied, the intra-prediction technique information can also include information about the segmentation direction (e.g., intra_subpartitions_split_flag). Furthermore, the intra-prediction technique information can include flag information indicating whether PDPC is applied, flag information indicating whether LIP is applied, or flag information indicating whether LM mode is applied.

[0158] If it is determined that a predetermined intra-prediction technique can be used for the current block, the image decoding device can determine whether to apply the corresponding intra-prediction technique to the current block based on the intra-prediction technique information signaled.

[0159] As another example, in addition to explicitly signaling intra-prediction technique information, the image encoding and decoding devices can also implicitly determine whether to apply a predetermined intra-prediction technique to the current block based on the compilation parameters of the current block. In this case, the compilation parameters may include the size (width and / or height) of the current block, the position of the current block, the color components of the current block, or whether to apply another intra-prediction technique.

[0160] The ISP model according to this disclosure will be described in detail below.

[0161] The current block may include a luminance component block (array) and a corresponding chrominance component block (array). In this disclosure, "current block" or "luminance component block" may mean "the luminance component block of the current block," while "chrominance component block" or "corresponding chrominance component block" may mean "the chrominance component block of the current block." Furthermore, "luminance component block" may be referred to as the term "luminance block," "luminance component block," or "luminance block," while "chrominance component block" may be referred to as the term "chrominance block," "color component block," or "color block."

[0162] In regular intra-frame prediction, the current compile / decode target block (the current block) is treated as a single unit and therefore compile / decode is performed without segmentation. However, when ISP mode is applied, the current block is segmented horizontally or vertically to perform intra-frame predictive encoding / decoding. In this case, encoding / decoding is performed on a sub-partition basis to generate a reconstructed sub-partition, which is then used as the reference block for the next sub-partition.

[0163] Whether an ISP is available for the current block can be determined based on the following conditions in Table 2. The following conditions can also be determined based on the luma component blocks of the current block. That is, the width, height, and position of the current block can refer to the width, height, and position of the luma component blocks of the current block, respectively, under the following conditions.

[0164] [Table 2]

[0165]

[0166] For example, when all the above conditions for the current block are met, it can be determined that the ISP is available for the current block. After determining that the ISP is available for the current block, it can be determined whether to apply the ISP to the current block. In the ISP availability condition, (x0, y0) are the coordinates indicating the position of the top left sample of the current block. Additionally, intra_luma_ref_idx[x0][y0] indicates information about the reference line used for the prediction of the current block. According to the ISP availability condition, when intra_luma_ref_idx is 0, that is, when the reference line used for the prediction of the current block is line 0 (the reference line immediately adjacent to the current block), it can be determined that the ISP is available for the current block. When the reference line used for the prediction of the current block is a line other than line 0, it can be determined that the ISP is not available for the current block.

[0167] In the ISP availability conditions, cbWidth and cbHeight correspond to the width and height of the current block, respectively. Additionally, MaxTbSizeY and MinTbSizeY can indicate the maximum and minimum transform sizes, respectively. As mentioned above, residual processing can include transform or inverse transform. In this case, the size of the transform block available for transform or inverse transform can be predefined or signaled via a bitstream. That is, the maximum transform size refers to the maximum size of the transform block that can perform the transform or inverse transform. Conversely, the minimum transform size refers to the minimum size of the transform block that can perform the transform or inverse transform. For example, if the current block size is greater than the maximum transform size, the current block can be divided into two or more transform blocks. Alternatively, the current block may not be divided into transform blocks with sizes smaller than the minimum transform size. The maximum and / or minimum transform sizes can be predefined in the image encoding and decoding apparatus, or they can be derived based on information signaled at a higher level of the block.

[0168] According to the ISP availability condition, an ISP can only be determined to be available for the current block if at least one of cbWidth or cbHeight is equal to or less than MaxTbSizeY. That is, if both cbWidth and cbHeight are greater than MaxTbSizeY, it can be determined that the ISP is not available for the current block. When cbWidth is greater than MaxTbSizeY, cbHeight is equal to or less than MaxTbSizeY, and the ISP is applicable to the current block, the ISP partitioning direction can be determined as vertical, as described below. When cbHeight is greater than MaxTbSizeY, cbWidth is equal to or less than MaxTbSizeY, and the ISP is applicable to the current block, the ISP partitioning direction can be determined as horizontal, as described below.

[0169] According to the ISP availability condition, when cbWidth * cbHeight is greater than MinTbSizeY * MinTbSizeY, it can be determined that the ISP is available for the current block. cbWidth * cbHeight can refer to the area of ​​the current block or the number of samples included in the current block. For example, if MinTbSizeY is 4, then the ISP can only be determined to be available for the current block if the number of samples included in the current block is greater than 16 (4 * 4).

[0170] Figure 9 This is a view illustrating the conventional method of encoding / decoding the current block according to the ISP mode.

[0171] When the current block to be encoded / decoded is input (S910), it can be determined whether to apply the ISP to the current block (S920). The determination in step S920 may include determining whether the ISP is available for the current block and / or determining whether to apply the ISP to the current block. The determination of whether the ISP is available for the current block can be made based on the aforementioned ISP availability conditions. When the ISP is available, the image encoding device can determine whether to apply the ISP to the current block based on the various methods described above, and can encode the determination result as intra-frame prediction technology information in the bitstream. When the ISP is available, the image decoding device can determine whether to apply the ISP to the current block based on the intra-frame prediction technology information signaled.

[0172] When the ISP is not applicable to the current block, TU tiling (S930) can be performed on the current block. TU tiling refers to the process of dividing the current block into multiple transform blocks such that the width and height of the current block are equal to or less than the maximum transform size, which is the transformable size. As mentioned above, the processing of the residual signal includes transformation, and the maximum transform size refers to the maximum size of the transform block on which the transformation process can be performed. Therefore, when the width or height of the current block is greater than the maximum transform size, by dividing the current block via TU tiling, the width and height of the divided blocks can be equal to or less than the maximum transform size. For example, when the maximum transform size is 64 samples and the current block has a size of 128x128, the current block can be divided into four 64x64 blocks. Alternatively, when the maximum transform size is 64 samples and the current block has a size of 64x128 or 128x64, the current block can be divided into two 64x64 blocks. For example, when the maximum transform size is 64 samples and the current block has a size of 64x64 or smaller, TU tiling may not be performed.

[0173] Subsequently, encoding / decoding (S960) can be performed on the current block or on each block segmented by the TU tiling step. The encoding in step S960 may include intra-frame prediction, residual processing, and / or encoding of prediction and residual information. The encoding in step S960 may include intra-frame prediction, residual sample derivation, and / or reconstructed block generation.

[0174] In step S920, when the ISP is applied to the current block, the partitioning direction and the number of partitions (number of sub-partitions) can be determined (S940).

[0175] In step S920, the segmentation direction can be derived based on information signaled via a bitstream (e.g., intra_subpartitions_split_flag). Alternatively, the segmentation direction can be derived implicitly based on the size of the current block. For example, as described above, when the width of the current block is greater than the maximum transform size, the segmentation direction of the ISP can be derived in the vertical direction. Additionally, when the height of the current block is greater than the maximum transform size, the segmentation direction of the ISP can be derived in the horizontal direction. Furthermore, when both the width and height of the current block are greater than the maximum transform size, restrictions can be imposed to prevent the ISP from being used on the current block. For example, when the maximum transform size is 64 samples and the current block to which the ISP applies is a 128x64 block, the segmentation direction of the ISP is derived in the vertical direction, and the width (128 / 4) and height (64) of the subpartition are determined to be the maximum transform size (64) or smaller. Similarly, when the maximum transform size is 64 samples and the current block to which the ISP applies is a 64x128 block, the segmentation direction of the ISP is derived in the horizontal direction, and the width (64) and height (128 / 4) of the sub-partitions are determined to be the maximum transform size (64) or smaller. As described above, when the ISP is applied to the current block, the current block is segmented such that the width and height of the sub-partitions are equal to or less than the maximum transform size. For this reason, it is not necessary to perform TU tiling in step S930 with respect to the current block to which the ISP applies.

[0176] In step S940, the number of splits can be implicitly derived based on the size of the current block. Specifically, the current block can be split according to its size, as shown in Table 3.

[0177] [Table 3]

[0178] Block size Number of partitions 4x4 No partition 4x8, 8x4 2 All other cases 4

[0179] Figure 10a This is a view illustrating an example of ISP segmentation for 4x8 or 8x4 blocks. (Example:) Figure 10a As shown in the diagram, a 4x8 block or an 8x4 block can be divided into two sub-partitions. When the current block is a 4x8 block and is divided horizontally, it can be divided into two 4x4 sub-partitions. When the current block is a 4x8 block and is divided vertically, it can be divided into two 2x8 sub-partitions. When the current block is an 8x4 block and is divided horizontally, it can be divided into two 8x2 sub-partitions. When the current block is an 8x4 block and is divided vertically, it can be divided into two 4x4 sub-partitions.

[0180] Figure 10b This is a view illustrating an example of ISP segmentation for blocks of 8x8 size or larger.

[0181] like Figure 10b As shown, a block of size 8x8 or larger can be divided into four sub-partitions. When the current block is a WxH block, where W and H are both 8 or larger, and the current block is divided horizontally, the current block can be divided into four Wx(H / 4) sub-partitions. When the current block is a WxH block, where W and H are both 8 or larger, and the current block is divided vertically, the current block can be divided into four (W / 4)xH sub-partitions.

[0182] Refer again Figure 9 When the segmentation direction and number of segmentations are determined in step S940, the luminance component blocks of the current block can be segmented based on this (S950).

[0183] Subsequently, encoding / decoding (S960) can be performed for each segmented sub-partition. As described above, the encoding in step S960 can include intra-frame prediction, residual processing, and / or encoding of prediction information and residual information. Additionally, the decoding in step S960 can include intra-frame prediction, residual sample derivation, and / or reconstructed block generation. Specifically, when ISP is applied, the intra-frame prediction mode for the current block is also applied to the sub-partitions, and neighboring reference samples are derived and used on a sub-partition basis to improve intra-frame prediction performance. That is, when ISP is applied, residual sample processing is performed on a sub-partition basis. In other words, intra-frame prediction samples are derived for each sub-partition, and the corresponding sub-partition's residual signal (residual sample) is added to it to obtain reconstructed samples. The residual signal (residual sample) can be derived based on the residual information (quantized transform coefficient information or residual compilation syntax) in the bitstream described above, through a dequantization / inverse transform process. That is, prediction sample derivation and residual sample derivation for the first sub-partition can be performed, and reconstructed samples for the first sub-partition can be derived based on this. In this scenario, when exporting the predicted samples for the second sub-partition, some of the reconstructed samples in the first sub-partition (e.g., reconstructed samples adjacent to the left or top of the second sub-partition) can be used as neighboring reference samples for the second sub-partition. Similarly, the predicted sample export and residual sample export for the second sub-partition can be exported, and the reconstructed samples for the second sub-partition can be derived based on these. In the same scenario, when exporting the predicted samples for the third sub-partition, some of the reconstructed samples in the second sub-partition (e.g., reconstructed samples adjacent to the left or top of the third sub-partition) can be used as neighboring reference samples for the third sub-partition. Similarly, some of the reconstructed samples in the third sub-partition can be used as neighboring reference samples for the fourth sub-partition.

[0184] The encoding / decoding order within multiple sub-partitions is top-down when the partitioning direction is horizontal, and left-to-right when the partitioning direction is vertical. For example, in Figure 10bIn this process, when the partitioning direction is horizontal, sub-partitions can be encoded / decoded sequentially from the topmost sub-partition to the bottommost sub-partition. Conversely, when the partitioning direction is vertical, sub-partitions can be encoded / decoded sequentially from the leftmost sub-partition to the rightmost sub-partition.

[0185] When applying ISP to the current block, to reduce compilation complexity, an MPM list is generated for each segmentation method (horizontal and vertical segmentation), and the appropriate prediction modes in the generated MPM list are compared in terms of Rate Distortion Optimization (RDO) to generate the optimal mode. Additionally, when using Multi-Reference Line (MRL) intra-frame prediction, restrictions can be imposed to prevent the use of the aforementioned ISP. That is, ISP is applicable when using the 0th reference line (e.g., intra_luma_ref_idx == 0). Furthermore, when applying ISP, restrictions can be imposed to prevent the use of the aforementioned PDPC. That is, when applying ISP, PDPC can be omitted.

[0186] When ISP is applied, as intra-prediction technology information, information indicating whether ISP is applied can be sent on a per-block basis (intra_subpartitions_mode_flag), and if ISP is used in the current block (e.g., when intra_subpartitions_mode_flag is 1), information about the segmentation method (horizontal segmentation or vertical segmentation) can be sent (intra_subpartitions_split_flag).

[0187] According to the reference Figure 9 The conventional method described involves a single-tree structure where the luma component block and the chroma component block of the current block are partitioned into the same tree structure. When ISP is applied to the current block, the luma component block is divided into multiple sub-partitions, but ISP is not applied to the chroma component block. In this case, the width or height of the chroma component block becomes larger than the maximum transform size according to the chroma format, and therefore, transforming or inverse transforming the chroma component block may become impossible.

[0188] The following text will describe the relationship between the size of the luminance component block and the size of the chroma component block according to the chroma format.

[0189] Figure 11 is a view illustrating the relationship between luminance component blocks (luminance component array) and chrominance component blocks (chrominance component array) according to the chrominance format.

[0190] A source or compiled image can include a luma component (Y) block and two chroma component (cb, cr) blocks. That is, a pixel in an image can include a luma sample and two chroma samples (cb, cr). A color format can represent the configuration format of the luma and chroma samples (cb, cr) and can be referred to as a chroma format. The chroma format can be predefined or can be adaptively signaled. For example, the chroma format can be signaled based on at least one of chroma_format_idc or separate_colour_plane_flag as shown in Table 4.

[0191] [Table 4]

[0192] chroma_format_idc separate_colour_plane_flag ChromaArrayType Chroma format SubWidthC SubHeightC 0 0 0 Monochrome 1 1 1 0 1 4:2:0 2 2 2 0 2 4:2:2 2 1 3 0 3 4:4:4 1 1 3 1 0 4:4:4 1 1

[0193] In Table 4 above, `chroma_format_idc` indicates the format of the luminance sample and its corresponding chrominance sample, and `separate_colour_plane_flag` indicates the information of the three color components Y, cb, and cr encoded in a 4:4:4 chrominance format. In Table 4, when `chroma_format_idc` is 0, the chrominance format corresponds to monochrome, and the current block does not include chrominance component blocks but only luminance component blocks.

[0194] In Table 4 above, when chroma_format_idc is 1, the chroma format corresponds to the 4:2:0 chroma format, and the width and height of the chroma component block correspond to half the width and half the height of the luma component block, respectively. Figure 11a The positional relationship between the luminance and chrominance samples is shown in a 4:2:0 chrominance format.

[0195] In Table 4 above, when chroma_format_idc is 2, the chroma format corresponds to the 4:2:2 chroma format, the width of the chroma component block corresponds to half the width of the luma component block, and the height of the chroma component block is equal to the height of the luma component block. Figure 11b The positional relationship between the luminance and chrominance samples is shown in a 4:2:2 chrominance format.

[0196] In Table 4 above, when chroma_format_idc is 3, the chroma format corresponds to the 4:4:4 chroma format, and the width and height of the chroma component block correspond to the width and height of the luma component block, respectively. Figure 11c The positional relationship between the luminance and chrominance samples is shown in a 4:4:4 chrominance format.

[0197] In Table 4 above, SubWidthC and SubHeightC represent the ratio of luminance samples to chrominance samples. For example, when the width and height of the luminance component block are CbWidth and CbHeight, respectively, the width and height of the corresponding chrominance component block can be derived as (CbWidth / SubwidthC) and (CbHeight / SubHeightC), respectively.

[0198] As described with reference to Figure 11, the size of the chroma component block corresponding to the luminance component block of the current block can vary according to the chroma format.

[0199] Figure 12 This is a view illustrating the size of the chroma component blocks according to the chroma format when the luminance component blocks are 64x128 blocks. For example... Figure 12 As shown, the chromaticity component block corresponding to the 64x128 luminance component block can be a 32x64 block in 4:2:0 chromaticity format, a 32x128 block in 4:2:2 chromaticity format, and a 64x128 block in 4:4:4 chromaticity format.

[0200] As mentioned above, according to the reference Figure 9 The conventional method described, in a single-tree structure where the luma and chroma component blocks are segmented into the same tree structure, when the ISP is applied to the current block, the luma component block is segmented into multiple sub-partitions, but the chroma component block is not segmented. For example, when the luma component block to which the ISP is applied is a 64x128 block, since the segmentation direction is determined to be horizontal, the luma component block is segmented into four 64x32 sub-partitions, and the width (64) and height (32) of each sub-partition are equal to or less than the maximum transform size (64). Therefore, a transform or inverse transform can be performed on each sub-partition of the luma component block. However, as Figure 12 As shown, the chroma component blocks are 32x64 blocks in 4:2:0 chroma format, 32x128 blocks in 4:2:2 chroma format, or 64x128 blocks in 4:4:4 chroma format. For example, in the 4:2:2 chroma format and the 4:4:4 chroma format, the transformation or inverse transformation of the chroma component blocks is not possible because the height (128) of the chroma component blocks is greater than the maximum transform size (64).

[0201] Various embodiments of this disclosure for solving the above-described problems will be described in detail below.

[0202] Embodiment #1

[0203] In Embodiment #1 of this disclosure, to address a common problem, when the ISP is applied to the current block, the ISP is adaptively adapted to the chroma component block based on its chroma format and / or size. According to Embodiment #1 of this disclosure, for example, when the ISP is applied to the chroma component block, the segmentation direction and number of segments determined with respect to the luma component block are similarly applied to the chroma component block.

[0204] Figure 13 This is a flowchart illustrating an ISP application method according to an embodiment of the present disclosure.

[0205] Figure 14 The diagram is based on Figure 13 A view of an example of the ISP application method for segmenting chroma component blocks.

[0206] When the current block to be encoded / decoded is input (S1310), it can be determined whether to apply the ISP to the current block (S1320). The determination in step S1320 may include determining whether the ISP is available for the current block and / or whether to apply the ISP to the current block. The determination of whether the ISP is available for the current block can be performed based on the aforementioned ISP availability conditions. When the ISP is available, the image encoding device can determine whether to apply the ISP to the current block based on the various methods described above, and can encode the determination result as intra-frame prediction technology information in the bitstream. When the ISP is available, the image decoding device can determine whether to apply the ISP to the current block based on the intra-frame prediction technology information signaled.

[0207] When ISP is not applicable to the current block, TU tiling can be performed on the current block (S1330). Thereafter, encoding / decoding can be performed on the current block or on each block segmented by the TU tiling step (S1390). Steps S1330 and S1390 are equal to... Figure 9 Steps S930 and S960 will therefore be omitted in their detailed description.

[0208] In step S1320, when the ISP is applied to the current block, the partitioning direction and the number of partitions (number of sub-partitions) can be determined (S1340). The description of determining the partitioning direction and the number of partitions is equal to... Figure 9 The description will therefore be omitted.

[0209] When the segmentation direction and number of segmentations are determined in step S1340, the luminance component blocks of the current block can be segmented based on this (S1350).

[0210] Subsequently, it can be determined whether to apply the ISP to the chroma component block corresponding to the luma component block (S1360). The determination in step S1360 can be performed by the chroma format and / or by comparing the width and height of the chroma component block with the maximum transform size.

[0211] In the 4:2:0 chroma format, as described below, the width and height of the chroma component blocks are equal to or less than the maximum transform size. Therefore, in the 4:2:0 chroma format, without comparing the width and height of the chroma component blocks with the maximum transform size, it can be determined that ISP is not applicable to the chroma component blocks.

[0212] In the case of 4:2:2 or 4:4:4 chroma formats, as described below, the width or height of the chroma component block can be greater than the maximum transform size. Therefore, in the case of 4:2:2 or 4:4:4 chroma formats, the width and height of the chroma component block can be compared with the maximum transform size, and when the width or height of the chroma component block is greater than the maximum transform size, it can be determined that the ISP is applicable to the chroma component block.

[0213] like Figure 14 As shown, when the luminance component block is 64x128 blocks, the corresponding chrominance component blocks are 32x64 blocks in 4:2:0 chrominance format, 32x128 blocks in 4:2:2 chrominance format, or 64x128 blocks in 4:4:4 chrominance format.

[0214] exist Figure 14 In the example shown, it can be seen that the width (32) and height (64) of the chroma component block in the 4:2:0 chroma format are both equal to or less than the maximum transform size (64). Therefore, in the case of the 4:2:0 chroma format, it can be determined that the ISP is not applicable to the 32x64 chroma component block without additional size comparison.

[0215] In contrast, in 4:2:2 or 4:4:4 chroma formats, additionally, the width and height of the chroma component blocks can be compared to the maximum transform size. Figure 14 In the example shown, since the width (128) of the 32x128 chroma component block in 4:2:2 chroma format and the width (128) of the 64x128 chroma component block in 4:4:4 chroma format are greater than the maximum transform size (64), it can be determined that the ISP is applicable to the chroma component blocks.

[0216] As a variation of Example #1, regardless of the chroma format, it can be determined whether to apply the ISP to the chroma component blocks based on whether the width and height of the chroma component blocks are both equal to or less than the maximum transform size. For example, when the width and height of the chroma component blocks are both equal to or less than the maximum transform size, it can be determined that the ISP is not applicable to the chroma component blocks. Conversely, when the width or height of the chroma component blocks is greater than the maximum transform size, it can be determined that the ISP is applicable to the chroma component blocks.

[0217] When it is determined that the ISP is applicable to the chroma component block, the chroma component block can be segmented (S1370). The segmentation of the chroma component block can be performed based on the segmentation direction and number of segments determined in step S1340. Specifically, the segmentation direction and number of segments of the chroma component block can be determined in the same way as the segmentation direction and number of segments of the luminance component block. According to Embodiment #1 of this disclosure, in Figure 14 In the example shown, the luma component block is divided into four sub-partitions horizontally. Therefore, the same method can be used to divide the 32x128 chroma component block in 4:2:2 chroma format and the 64x128 chroma component block in 4:4:4 chroma format into four sub-partitions horizontally.

[0218] Subsequently, encoding / decoding can be performed on each segmented sub-partition (S1390). In this case, the encoding / decoding of step S1390 can be performed for each sub-partition of the luma component block and each sub-partition of the chroma component block. Step S1390 is equivalent to step S960 and therefore its detailed description will be omitted.

[0219] In step S1360, when it is determined that the ISP is not applicable to the chroma component, the chroma component block is not segmented (S1380), and each sub-partition of the luma component block and the unsegmented chroma component block can be encoded / decoded (S1390). Step S1390 is equivalent to step S960 and therefore its detailed description will be omitted. According to embodiment #1 of this disclosure, in Figure 14 In the example shown, the 32x64 chroma component blocks in 4:2:0 chroma format can be left undivided.

[0220] According to Embodiment #1 of this disclosure, when the width or height of the chroma component block to which the ISP is applied in the current block is greater than the maximum transform size, the problem of the impossibility of transforming or inverse transforming the chroma component block can be solved by applying the ISP to the chroma component block. Furthermore, according to Embodiment #1 of this disclosure, when the chroma format of the current block to which the ISP is applied is 4:2:0, or when the width and height of the chroma component block to which the ISP is applied in the current block are equal to or less than the maximum transform size, the ISP is not applied to the chroma component block. Therefore, since the unnecessary application of the ISP to the chroma component block can be avoided, the computational load of encoding / decoding can be reduced. Additionally, according to Embodiment #1 of this disclosure, since the segmentation direction and number of segments for the ISP for the chroma component block are determined in the same way as the segmentation direction and number of segments for the ISP for the luma component block, it is not necessary to separately signal or derive the segmentation direction and number of segments for the ISP for the chroma component block.

[0221] Embodiment #2

[0222] In Embodiment #2 of this disclosure, to address a common problem, when the ISP is applied to the current block, the ISP is also applied to the chroma component block. According to Embodiment #2 of this disclosure, the segmentation direction and the number of segmentations of the chroma component block can be determined based on the segmentation direction and the number of segmentations determined for the luma component block.

[0223] Figure 15 This is a flowchart illustrating an ISP application method according to another embodiment of the present disclosure.

[0224] Figure 16 The diagram is based on Figure 15 A view of an example of the ISP application method for segmenting chroma component blocks.

[0225] When the current block to be encoded / decoded is input (S1510), it can be determined whether to apply the ISP to the current block (S1520). The determination in step S1520 may include determining whether the ISP is available for the current block and / or whether to apply the ISP to the current block. The determination of whether the ISP is available for the current block can be performed based on the aforementioned ISP availability conditions. When the ISP is available, the image encoding device can determine whether to apply the ISP to the current block based on the various methods described above, and can encode the determination result as intra-frame prediction technology information in the bitstream. When the ISP is available, the image decoding device can determine whether to apply the ISP to the current block based on the intra-frame prediction technology information signaled.

[0226] When ISP is not applicable to the current block, TU tiling can be performed on the current block (S1530). Afterwards, encoding / decoding can be performed on the current block or on each block segmented by the TU tiling step (S1570). Steps S1530 and S1570 are equal to... Figure 9 Steps S930 and S960 will be described in detail, and their specific descriptions will be omitted.

[0227] In step S1520, when the ISP is applied to the current block, the partitioning direction and the number of partitions (number of sub-partitions) can be determined (S1540). The description of determining the partitioning direction and the number of partitions is equal to... Figure 9 The description will therefore be omitted.

[0228] The luminance component blocks of the current block can be segmented based on the segmentation direction and number of segmentations determined in step S1540 (S1550).

[0229] Subsequently, the chroma component blocks of the current block can be segmented based on the segmentation direction and number of segmentations determined in step S1540 (S1560). Specifically, the segmentation direction of the chroma component blocks can be determined in the same way as the segmentation direction of the luminance component blocks. Furthermore, the number of segmentations of the chroma component blocks can be determined based on the number of segmentations of the luminance component blocks. For example, when the number of segmentations of the luminance component blocks is N, the number of segmentations of the chroma component blocks can be determined as N / n. In Embodiment #2 of this disclosure, for example, n can be 2, but is not limited thereto, and n can be any integer. According to Embodiment #2, in Figure 16 In the example shown at the top, a 16x16 luma component block can be horizontally divided into four sub-partitions. In this case, the corresponding chroma component block is divided horizontally, and the number of divisions can be determined to be 2. That is, the corresponding chroma component block can be horizontally divided into two sub-partitions, regardless of the chroma format and / or size of the chroma component block. Additionally, in Figure 16 In the example shown at the bottom, the 4x8 luma component block is divided into two sub-partitions in the vertical direction. In this case, the corresponding chroma component block is divided in the vertical direction, and the number of divisions can be determined to be 1. In this case, since the chroma component block is essentially not divided, it can be seen that ISP is not applicable to the chroma component block.

[0230] Subsequently, encoding / decoding can be performed on each sub-partition or undivided chroma component block (S1570). In this case, the encoding / decoding of step S1570 can be performed on each sub-partition of the luma component block and each sub-partition or chroma component block of the chroma component block. Step S1570 is equivalent to step S960 and therefore its detailed description will be omitted.

[0231] In Embodiment #2 of this disclosure, when the size of the maximum compilation unit is 128x128, the maximum transform size is half the width and height of the maximum compilation unit, and the width and height of the current block are both greater than the maximum transform size, the ISP is not applied to the current block. According to Embodiment #2 of this disclosure, since the width and height of the chroma component block or sub-partition of the chroma component block to which the ISP applies are always equal to or less than the maximum transform size, the problem of the impossibility of transforming or inverse transforming the chroma component block can be solved. Furthermore, according to Embodiment #2 of this disclosure, since it is not necessary to perform a determination regarding whether to apply the ISP to the chroma component block of the current block (e.g., the determination in S1360), the computational load for encoding / decoding can be reduced. Additionally, according to Embodiment #2 of this disclosure, since the number of sub-partitions of the chroma component block is determined to be 2 or 1, the encoding / decoding process of the chroma component block can be simplified. Furthermore, according to Embodiment #2 of this disclosure, since the segmentation direction and number of segments of the ISP for the chroma component block are determined in the same way as the segmentation direction and (segmentation number / 2) for the luminance component block, it is not necessary to separately signal or derive the segmentation direction and number of segments of the ISP for the chroma component block.

[0232] Embodiment #3

[0233] In embodiment #3 of this disclosure, in order to solve the conventional problem, the width and height of the chroma component block under the ISP availability condition are changed to be equal to or less than the maximum transform size.

[0234] According to the reference Figure 9 The described conventional ISP method determines that ISP is not applicable to the current block if both its width and height are greater than the maximum transform size. In other words, ISP is applicable to the current block if either its width or height is greater than the maximum transform size and the other is equal to or less than the maximum transform size.

[0235] In Embodiment #3 of this disclosure, the above-mentioned ISP availability conditions are modified as shown in Table 5 below.

[0236] [Table 5]

[0237]

[0238] In the revised ISP availability conditions, the conditions for intra_luma_ref_idx[x0][y0] to be 0 and cbWidth*cbHeight to be greater than MinTbSizeY*MinTbSizeY are equivalent to the existing ISP availability conditions. According to the revised ISP availability conditions, an ISP can only be determined to be available for the current block if both cbWidth and cbHeight are equal to or less than MaxTbSizeY.

[0239] According to embodiment #3 of this disclosure, which applies the modified ISP availability conditions, the ISP is determined to be applicable to the current block only when both the width and height of the current block are equal to or less than the maximum transformation size, and an intra_subpartitions_mode_flag indicating whether to apply the ISP can be sent.

[0240] According to embodiment #3 of this disclosure, since the width and height of the current block are both equal to or less than the maximum transform size, the width and / or height of the chroma sample block corresponding to the luminance sample block of the current block, regardless of the chroma format, are always equal to or less than the maximum transform size. Therefore, the reference availability condition can be solved by applying the modified ISP availability condition. Figure 9 The problems described in the conventional ISP method.

[0241] The method according to embodiment #3 of this disclosure can be implemented by modifying the ISP availability conditions of the conventional method. Therefore, except for the difference in the conditions used only to determine whether the ISP is available in step S920, it can be similar to... Figure 9 The flowchart shown also performs the method according to embodiment #3 of this disclosure.

[0242] Figure 17 This is a view illustrating an example of the structure of a bitstream that reflects the modified ISP availability conditions.

[0243] Figure 17 The bounding box is related to signaling information about the intra-prediction technique (ISP) of the current compilation unit. For example... Figure 17 As shown, when the modified ISP availability conditions described above are met, the `intra_subpartitions_mode_flag` indicating whether ISP should be applied can be sent. `intra_subpartitions_mode_flag` indicates whether ISP should be applied to the current compilation unit. When the ISP availability conditions are not met, `intra_subpartitions_mode_flag` is not sent, and it can be determined that ISP is not applicable to the current compilation unit.

[0244] The modified ISP availability conditions may include one or more conditions, and the conditions included in the modified ISP availability conditions are not limited to the examples above. That is, within the scope of the technical concept of this disclosure, some conditions may be omitted or other conditions may be additionally included.

[0245] In addition, such as Figure 17 As shown, when the ISP applies to the current block, that is, when intra_subpartitions_mode_flag is 1, information indicating the split direction (intra_subpartitions_split_flag) can be sent.

[0246] exist Figure 17 In the example shown, `intra_subpartitions_mode_flag` can only be sent if the condition `cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY` is met. That is, when the sent `intra_subpartitions_mode_flag` is 1, it can be seen that the condition `cbWidth <= MaxTbSizeY && cbHeight <= MaxTbSizeY` is already met. Therefore, it can be done as follows... Figure 18 The change shown is related to whether or not the intra_sinpartitions_split_flag is sent. Figure 17 conditions.

[0247] Figure 18 This is a view illustrating another example of the structure of a bitstream that reflects modified ISP availability conditions.

[0248] Figure 18 The bounding box contains signaling information about the intra-prediction technique (ISP) of the current compilation unit. For example... Figure 18 As shown, when the modified ISP availability conditions described above are met, the intra_subpartitions_mode_flag can be sent by signal, and when the condition where intra_subpartitions_mode_flag is 1 is met, the intra_sinpartitions_split_flag can be sent by signal.

[0249] According to Embodiment #3 of this disclosure, the width and height of the current block to which ISP is applied are both equal to or less than the maximum transform size. Correspondingly, since the width and height of the chroma component blocks of the current block are both equal to or less than the maximum transform size, and are independent of the chroma format, the problem that chroma component blocks cannot be transformed or inversely transformed can be solved. Furthermore, according to Embodiment #3 of this disclosure, since the determination of whether to apply ISP to the chroma component blocks of the current block (e.g., the determination of S1360) and the ISP of the chroma component blocks do not need to be performed, the computational load of encoding / decoding can be reduced. Additionally, according to Embodiment #3 of this disclosure, since the conventional encoding / decoding process is not significantly changed, conventional problems can be solved without significantly increasing encoding / decoding complexity.

[0250] Although the exemplary methods of this disclosure described above are represented as a series of operations for clarity of description, they are not intended to limit the order in which the steps are performed, and these steps may be performed simultaneously or in different orders if necessary. To implement the method according to the invention, the described steps may further include other steps, including steps in addition to some steps, or may include additional steps in addition to some steps.

[0251] In this disclosure, the image encoding device or image decoding device that performs a predetermined operation (step) can perform an operation (step) that confirms the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when predetermined conditions are met, the image encoding device or image decoding device can perform the predetermined operation after determining whether the predetermined conditions are met.

[0252] The various embodiments disclosed herein are not a list of all possible combinations and are intended to describe representative aspects of the disclosure; the matters described in the various embodiments may be applied independently or in combination of two or more.

[0253] Various embodiments of this disclosure can be implemented in hardware, firmware, software, or a combination thereof. When this disclosure is implemented in hardware, it can be implemented using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0254] Furthermore, the image decoding and image encoding apparatuses applying embodiments of this disclosure can be included in multimedia broadcasting transmission and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, OTT (over-the-top) video devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, medical video devices, etc., and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0255] Figure 19 This is a view illustrating a content streaming system to which embodiments of the present disclosure may be applied.

[0256] like Figure 19 As shown, the content streaming system using embodiments of this disclosure may mainly include an encoding server, a streaming server, a web server, media storage, user equipment, and a multimedia input device.

[0257] The encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream and then sends the bitstream to the streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0258] The bitstream can be generated by an image encoding method or image encoding apparatus applying embodiments of the present disclosure, and the streaming server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0259] A streaming server sends multimedia data to a user's device based on a user's request via a web server, and the web server acts as a medium for notifying the user of services. When a user requests a service from a web server, the web server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server acts as a command / response controller between devices in the content streaming system.

[0260] A streaming server can receive content from media storage and / or encoding servers. For example, when content is received from an encoding server, it can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0261] Examples of user equipment may include mobile phones, smartphones, laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[0262] In a content streaming system, each server can operate as a distributed server, in which case the data received from each server can be distributed.

[0263] The scope of this disclosure includes software or executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling the operation of methods according to various embodiments to be performed on a device or computer, and a non-transitory computer-readable medium having such software or commands stored thereon and executable on a device or computer.

[0264] Industrial applicability

[0265] The embodiments disclosed herein can be used to encode or decode images.

Claims

1. An image decoding method executed by an image decoding device, the image decoding method comprising: The prediction mode of the current block is determined based on the prediction mode information of the current block; Based on the fact that the prediction mode of the current block is the intra-prediction mode, determine whether the intra-sub-partition ISP can be used for the current block; Based on the fact that the ISP can be used for the current block, decode the ISP mode information for the current block; as well as When determining that the ISP is applied to the current block based on the ISP mode information, a predicted block for the current block is generated by applying the ISP to the current block. Specifically, if the index intra_luma_ref_idx of the reference line used for prediction of the current block is 0, the width of the current block is equal to or less than the maximum transform block size of the luma component block, and the height of the current block is equal to or less than the maximum transform block size of the luma component block, then the ISP is determined to be available for the current block.

2. The image decoding method according to claim 1, wherein, Generating a predicted block for the current block by applying the ISP to the current block includes: A prediction block for the luminance component block is generated by applying the ISP to the luminance component block of the current block; Determine whether to apply the ISP to the chroma component block of the current block; and When determining the chroma component block to which the ISP is applied to the current block, a prediction block for the chroma component block is generated by applying the ISP to the chroma component block.

3. The image decoding method according to claim 2, wherein, The determination of whether to apply ISP to the chroma component blocks of the current block is performed based on at least one of the size of the chroma component blocks or the chroma format of the current block.

4. The image decoding method according to claim 3, wherein, Based on the fact that the chroma format of the current block is 4:2:2 or 4:4:4 and at least one of the width or height of the chroma component block is greater than the maximum transform size, the ISP is determined to apply to the chroma component blocks of the current block and the chroma component blocks are partitioned into multiple transform blocks for the transform process.

5. The image decoding method according to claim 2, wherein, When the ISP is applied to the chroma component block, the segmentation direction and the number of sub-partitions for the chroma component block are determined based on the segmentation direction and the number of sub-partitions for the luminance component block.

6. The image decoding method according to claim 5, wherein, The number of division directions and sub-partitions used for the chroma component blocks are equal to the number of division directions and sub-partitions used for the luminance component blocks, respectively.

7. The image decoding method according to claim 1, wherein, Generating a predicted block for the current block by applying the ISP to the current block includes: A prediction block for the luminance component block is generated by applying the ISP to the luminance component block of the current block; and A prediction block for the chroma component block is generated by applying ISP to the chroma component block of the current block.

8. The image decoding method according to claim 7, wherein, The segmentation direction and the number of sub-partitions for the chroma component block are determined based on the segmentation direction and the number of sub-partitions used for the luminance component block.

9. The image decoding method according to claim 8, wherein, The division direction and the number of sub-partitions used for the chroma component block are respectively equal to the division direction and the number of sub-partitions of the luminance component block / 2.

10. An image decoding apparatus, comprising: Memory; as well as At least one processor, Wherein, the at least one processor is configured to: The prediction mode of the current block is determined based on the prediction mode information of the current block; Based on the fact that the prediction mode of the current block is the intra-prediction mode, determine whether the intra-sub-partition ISP can be used for the current block; Based on the availability of ISP for the current block, decode the ISP mode information for the current block; and When determining that an ISP is to be applied to the current block based on the ISP pattern information, a predicted block for the current block is generated by applying the ISP to the current block. Specifically, if the index intra_luma_ref_idx of the reference line used for prediction of the current block is 0, the width of the current block is equal to or less than the maximum transform block size of the luma component block, and the height of the current block is equal to or less than the maximum transform block size of the luma component block, then the ISP is determined to be available for the current block.

11. An image encoding method executed by an image encoding device, the image encoding method comprising: Determine the prediction mode for the current block; Based on the fact that the prediction mode of the current block is the intra-prediction mode, determine whether the intra-sub-partition ISP can be used for the current block; Based on whether the ISP is available for the current block, determine whether to apply the ISP to the current block; Based on the determination that the ISP is applied to the current block, a predicted block for the current block is generated by applying the ISP to the current block; as well as Encode the prediction mode information and the ISP mode information for the current block. Specifically, if the index intra_luma_ref_idx of the reference line used for prediction of the current block is 0, the width of the current block is equal to or less than the maximum transform block size of the luma component block, and the height of the current block is equal to or less than the maximum transform block size of the luma component block, then the ISP is determined to be available for the current block.

12. A method for transmitting a bit stream, characterized in that, include: The image encoding method according to claim 11 is used to generate a bitstream; And send the bit stream.

Images