Image encoding method and apparatus based on directional intra prediction

Abstract

The image decoding method according to embodiments of the present disclosure includes the following steps: obtaining prediction-related information through a bitstream; deriving an orientation plane pattern for the current block based on the prediction-related information; and generating a prediction sample for the current block based on the orientation plane pattern, wherein the orientation plane pattern includes at least one of a plane vertical pattern, a plane horizontal pattern, or a plane diagonal pattern.

Description

Technical Field

[0001] This disclosure relates to image / video coding methods and devices based on intra-frame prediction. Background Technology

[0002] Image / video encoding is used in a variety of applications, such as digital storage media, television broadcasting, video streaming services, and real-time communications. Recently, the demand for high-resolution and high-quality images / videos has been increasing across various fields.

[0003] As images / videos achieve higher resolution and quality, their data size increases, leading to a relative increase in the amount of information or bits transmitted. Consequently, the costs of transmission and storage rise when using conventional media such as wired and wireless broadband lines to transmit image data or when using existing storage media to store image / video data.

[0004] Recently, there has been an increasing interest in and demand for immersive media such as virtual reality (VR), augmented reality (AR), mixed reality (MR) content, and holograms. Furthermore, there is a growing number of attempts to use immersive media to deliver immersive experiences in gaming, education, healthcare, real estate, and marketing.

[0005] Therefore, efficient image / video compression technology is needed to efficiently compress, send, store, and play high-resolution and high-quality image / video information with various characteristics. Summary of the Invention

[0006] Technical solution

[0007] According to embodiments of this disclosure, a method and apparatus for improving video / image coding efficiency are provided.

[0008] According to embodiments of this disclosure, a video / image coding method and a video / image coding apparatus based on intra-frame prediction are provided.

[0009] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes the following steps: obtaining prediction-related information via a bitstream; deriving a directional plane pattern for a current block based on the prediction-related information; and generating a prediction sample for the current block based on the directional plane pattern, wherein the directional plane pattern includes at least one of a vertical plane pattern, a horizontal plane pattern, or a diagonal plane pattern.

[0010] According to embodiments of this disclosure, an image encoding method performed by an encoding device is provided. The method includes the following steps: determining an orientation plane pattern for a current block; generating a prediction sample for the current block based on the orientation plane pattern; generating prediction-related information based on the orientation plane pattern; and encoding image information including the prediction-related information, wherein the orientation plane pattern includes at least one of a vertical plane pattern, a horizontal plane pattern, or a diagonal plane pattern.

[0011] According to embodiments of the present disclosure, a decoding apparatus for image decoding is provided. The decoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: obtaining prediction-related information via a bitstream, deriving an orientation plane pattern for a current block based on the prediction-related information, and generating a prediction sample for the current block based on the orientation plane pattern, wherein the orientation plane pattern includes at least one of a plane vertical pattern, a plane horizontal pattern, or a plane diagonal pattern.

[0012] According to embodiments of the present disclosure, an encoding apparatus for image encoding is provided. The encoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: determining an orientation plane mode for a current block, generating a prediction sample for the current block based on the orientation plane mode, generating prediction-related information based on the orientation plane mode, and encoding image information including the prediction-related information, wherein the orientation plane mode includes at least one of a vertical plane mode, a horizontal plane mode, or a diagonal plane mode.

[0013] According to embodiments of the present disclosure, a method for transmitting video / image data is provided, the video / image data comprising a bitstream generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0014] According to embodiments of the present disclosure, an apparatus for transmitting video / image data is provided, the video / image data comprising a bitstream generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0015] According to embodiments of the present disclosure, a computer-readable storage medium is provided that stores a program for performing a method according to at least one embodiment of the present disclosure.

[0016] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded video / image information generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0017] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded information or encoded video / image information, the encoded information or encoded video / image information causing a decoding device to perform a video / image decoding method according to at least one embodiment of the present disclosure.

[0018] Beneficial effects

[0019] According to the embodiments of this disclosure, the overall video / image compression efficiency can be improved.

[0020] According to embodiments of this disclosure, prediction performance for the current block can be improved.

[0021] According to embodiments of this disclosure, directional planar patterns can be efficiently indicated.

[0022] According to embodiments of this disclosure, the problem of asymmetric memory buffer allocation for oriented plane mode can be solved.

[0023] According to embodiments of this disclosure, based on a stored intra-prediction mode, an intra-prediction mode stored for reference when deriving the intra-prediction mode of a subsequent block and an intra-prediction mode stored for transform kernel selection of the current block can be coordinated. Attached Figure Description

[0024] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0025] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied.

[0026] Figure 3 This diagram schematically illustrates the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0027] Figure 4 The intra-frame prediction process is illustrated.

[0028] Figure 5 An example of a video / image coding method based on intra-frame prediction is shown.

[0029] Figure 6 An example of a video / image decoding method based on intra-frame prediction is shown.

[0030] Figure 7 An example of a directional intra-frame prediction mode is shown.

[0031] Figure 8 An example of template-based HoG computation in DIMD is shown.

[0032] Figure 9 An example of adjacent blocks used to derive the MPM list is shown.

[0033] Figure 10 Intra-prediction modes derived from neighboring blocks and DIMD-based modes are shown.

[0034] Figure 11 The intra-prediction mode of the previous block within a specific region is shown.

[0035] Figure 12 Examples of predictions based on a horizontal plane pattern and examples of predictions based on a vertical plane pattern are shown.

[0036] Figure 13 An example of prediction based on a planar diagonal pattern is shown.

[0037] Figure 14 The derivation of the transformation kernel for oriented plane modes is shown.

[0038] Figure 15 A video / image encoding method according to an embodiment of the present disclosure is illustrated schematically.

[0039] Figure 16 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Detailed Implementation

[0040] Because this disclosure can have various variations and implementations, specific embodiments are illustrated in the accompanying drawings and will be described in detail. However, it should be understood that the embodiments of this disclosure are not intended to be limited to a particular embodiment. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the technical spirit of this disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise. As used herein, the term “and / or” includes any one and all combinations of two or more associated listed items. As used herein, the terms “comprising,” “including,” and “having” specify the presence of stated features, numbers, operations, elements, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, numbers, operations, elements, components, and / or combinations thereof. In this disclosure, the use of the term “may” in connection with examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature, but all examples are not limited thereto, and the corresponding feature or configuration may be omitted.

[0041] For ease of explanation of the different features, each component in the accompanying drawings described in this disclosure is shown independently, but this does not mean that each component is implemented as a separate piece of hardware or separate piece of software. For example, two or more of these components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which components are integrated and / or separated are also included within the scope of this disclosure without departing from its spirit.

[0042] In this disclosure, “A or B” may mean “A only”, “B only”, or “both A and B”. In other words, “A or B” in this disclosure may be interpreted as “A and / or B”. For example, “A, B or C” in this disclosure may mean “A only”, “B only”, “C only”, or “any one and all combinations of A, B and C”.

[0043] The forward slash ( / ) or comma used in this article can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0044] In this disclosure, "at least one of A and B" may mean "only A", "only B" or "both A and B". Furthermore, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted in the same way as "at least one of A and B".

[0045] In this disclosure, "at least one of A, B, and C" may mean "only A", "only B", "only C", or "any one and all combinations of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" may mean "at least one of A, B, and C".

[0046] The parentheses used in this disclosure can indicate "for example". Specifically, when indicated as "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" in this disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when indicated as "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0047] In this disclosure, the technical features explained individually in a single figure may be implemented independently or simultaneously.

[0048] This disclosure relates to video / image coding. For example, the methods / implementations described in this disclosure can be applied to methods disclosed in enhanced compression models or the H.267 standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the AOMedia Video2 (AV2) standard or next-generation video / image coding standards (e.g., H.268 and H.269).

[0049] In this disclosure, encoding may include encoding and / or decoding. In this disclosure, image encoding may be used interchangeably with video encoding.

[0050] In this disclosure, video can refer to a collection of images over time. An image typically refers to a unit representing a single image at a specific time, while a slice / tile refers to a unit that forms part of an image during encoding. A slice / tile may include one or more coding tree units (CTUs). A single image may include one or more slices / tiles. A tile may represent a rectangular area of ​​a CTU within a specific tile row and column of an image.

[0051] A single image can be divided into two or more sub-images. A sub-image can be a rectangular region of one or more slices of the image.

[0052] A pixel or cell can refer to the smallest unit that makes up a picture (or image). The term "sample" can be used in contrast to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the luminance component (pixel / pixel value) or only the chrominance component (pixel / pixel value).

[0053] A unit can represent a basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A single unit may include a luminance block and two chrominance (e.g., Cb and cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M ​​columns and N rows.

[0054] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, throughout the drawings, the same reference numerals may be used to denote similar elements, and redundant descriptions of similar elements may be omitted.

[0055] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0056] Reference Figure 1A video / image encoding system may include a first device (encoding device) and a second device (decoding device). The first device may deliver encoded video / image information or data to the second device in the form of a file or stream via a digital storage medium or network.

[0057] A video / image encoding system may also include a video / image acquisition device and a video / image renderer. The video / image acquisition device may be included in the encoding device, or it may be configured as a separate device or an external component. The video / image renderer may be included in the decoding device, or it may be configured as a separate device or an external component.

[0058] The first device may include a transmitter as an internal component, or a transmitter as a separate device or an external component.

[0059] The second device may include a receiver as an internal component, or as a receiver as a separate device or an external component.

[0060] An encoding device can be called an encoder, and a decoding device can be called a decoder. A transmitter can be included in the encoding device. A receiver can be included in the decoding device. A renderer can include a display, and the display can be configured as a separate device or an external component.

[0061] Decoding and encoding devices using embodiments of this disclosure can be included in multimedia broadcasting transmitters / receivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices such as video communication, mobile streaming devices, storage media, portable cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, Internet streaming service providers, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicle terminals), aircraft terminals, and ship terminals), and medical video devices, and can be used to process video signals or data signals. For example, over-the-top (OTT) video devices can include game consoles, Blu-ray players, Internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0062] A video / image acquisition device can acquire video / image sources. The video / image acquisition device can acquire video / images through processes of capturing, compositing, or generating video / images. The video / image acquisition device may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras and a video / image archive including previously captured video / images. The video / image generation device may include, for example, a portable camcorder, a computer, a tablet PC, and a smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, in which case the process of generating relevant data can replace the video / image capture process. The video / image source can perform a video / image preprocessing process to input the optimized video / image into the encoder.

[0063] Encoding devices can encode input video / images. For compression and encoding efficiency, encoding devices can perform a series of processes such as prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output as a bitstream.

[0064] A transmitter can send encoded images / image information or data, output as a bitstream, to a receiver in a receiving device via a digital storage medium or network, either as a file or a stream. The encoded images / image information or data output as a bitstream can be sent to the receiver via a streaming server. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0065] During the sending or receiving of bitstreams, the streaming server may temporarily store the bitstreams. The streaming server sends multimedia data to the user's device via a web server based on user requests, and the web server acts as an intermediary for notifying the user of available services. When a user requests a desired service from the web server, the web server forwards the request to the streaming server, and the streaming server sends the multimedia data to the user. The content streaming system may include a separate control server, in which case the control server controls the commands / responses between devices in the content streaming system.

[0066] A streaming server can receive content from media storage devices and / or encoding devices. For example, when receiving content from an encoding device, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.

[0067] Decoding devices can decode video / images by performing a series of processes corresponding to the operations of encoding devices, such as dequantization, inverse transform, and prediction.

[0068] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.

[0069] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied. In the following, the encoding apparatus may include image encoding apparatus and / or video encoding apparatus.

[0070] Reference Figure 2 The encoding device 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor and an intra-frame predictor. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image partitioner 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured as at least one hardware component (e.g., an encoder chipset or processor). The memory 270 may include a decoded picture buffer (DPB) or may be configured as a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0071] Image partitioner 210 can partition an input image (or picture or frame) input to encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit can be recursively partitioned from coding tree unit (CTU) or maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure. For example, a single coding unit can be partitioned into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, and a binary tree structure and / or a ternary structure can be applied later. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding unit that is no longer partitioned. In this case, based on the encoding efficiency according to the image characteristics, the maximum coding unit can be used as the final coding unit, or if necessary, the coding unit can be recursively partitioned into deeper coding units, thereby using the coding unit with the optimal size as the final coding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described below. In another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be divided or partitioned from the final encoding unit described above. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving the residual signal from the transform coefficients.

[0072] Depending on the context, the term "unit" can be used interchangeably with the terms "area" or "region". Generally, an M×N block can represent an array of samples or transform coefficients arranged in M ​​rows and N columns. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. "Sample" can be used as a term corresponding to a pixel or cell in a single picture (or image).

[0073] Encoding device 200 generates a residual signal (residual signal, residual block, or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the predictor from the input image signal (original block or original sample array), and sends the generated residual signal to converter 232. In this case, as shown, the component in encoder 200 used to subtract the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) can be referred to as subtractor 231. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The predictor can generate various information about the prediction, such as prediction mode information, and can send the generated information to entropy encoder 240, as described below in the description of each prediction mode. The information about the prediction can be encoded by entropy encoder 240 and output as a bitstream.

[0074] An intra-frame predictor can refer to samples within the current image to predict the current block. The referenced samples can be located as neighboring (adjacent to) the current block, or, depending on the prediction mode, as being located away from the current block. In intra-frame prediction, prediction modes can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC modes and planar modes. Depending on the granularity of the prediction direction, directional modes can include, for example, 33 or 65 directional prediction modes. However, this example is for illustration only, and more or fewer directional prediction modes can be used depending on the configuration. The intra-frame predictor can determine the prediction mode applied to the current block based on the prediction modes applied to neighboring blocks.

[0075] Inter-frame predictors can derive predicted blocks for the current block based on reference blocks (reference sample arrays) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted based on the correlation between motion information between neighboring blocks and the current block, at the block, sub-block, or sample level. Motion information can include motion vectors and reference image indices. Motion information can also include inter-frame prediction direction (L0 prediction, L1 prediction, and Bi prediction) information. In inter-frame prediction, neighboring blocks can include spatially adjacent blocks existing within the current image and temporally adjacent blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally adjacent block can be the same or different. A temporally adjacent block can be referred to as a co-located reference block or a co-located CU (colCU), and a reference image including a temporally adjacent block can also be referred to as a co-located image (colPic). For example, the inter-frame predictor can configure a motion information candidate list based on neighboring blocks and can generate information indicating candidates for deriving motion vectors and / or reference image indices for the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip and merge modes, the inter-frame predictor can use motion information about neighboring blocks as motion information about the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using motion vectors from neighboring blocks as motion vector predictors and signaling the motion vector difference.

[0076] Predictor 220 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also simultaneous intra-frame prediction and inter-frame prediction, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for predicting blocks. The IBC prediction mode or palette mode can be used for, for example, screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame based on the block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0077] The predicted signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graphical-Based Transform (GBT), or Conditional Nonlinear Transform (CNT).

[0078] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal as a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector based on the coefficient scan order, and can generate information about the transform coefficients based on the one-dimensional vector form of the quantized transform coefficients. Entropy encoder 240 can perform various encoding methods, such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). Entropy encoder 240 can encode information required for video / image reconstruction (e.g., values ​​of syntax elements) other than the quantized transform coefficients, either together or separately. The encoded information (e.g., encoded video / image information) can be sent or stored as a bitstream based on the Network Abstraction Layer (NAL). The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Furthermore, the video / image information may also include general constraint information. In this disclosure, information and / or syntax elements transmitted from the encoding device / signed to the decoding device may be included in the video / image information. The video / image information may be encoded by the aforementioned encoding process and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) and / or memory (not shown) for transmitting and / or storing signals output from the entropy encoder 240 may be configured as internal / external components of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0079] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 234 and inverse transform unit 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from the predictor to generate a reconstructed signal (reconstructed image, reconstructed block, or array of reconstructed samples). When no residual is available for processing the target block (e.g., when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current image, or for inter-frame prediction of the next image after filtering, as described below.

[0080] Luminance mapping and chroma scaling (LMCS) can be applied to image encoding and / or reconstruction processing.

[0081] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270, specifically in the DPB of memory 270. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, and bilateral filtering. Filter 260 can generate filtering-related information and can send the generated information to entropy encoder 240. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.

[0082] The modified reconstructed image sent to memory 270 can be used as a reference image in the inter-frame predictor. When inter-frame prediction is applied via the modified reconstructed image, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can improve encoding efficiency.

[0083] The DPB of memory 270 can store modified reconstructed images for use as reference images in the inter-frame predictor. Memory 270 can store motion information about blocks in the current image from which motion information is derived (or encoded) and / or about blocks in already reconstructed images. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and can send these reconstructed samples to the intra-frame predictor.

[0084] Figure 3This diagram schematically illustrates the configuration of a video / image decoding apparatus to which embodiments of the present disclosure may be applied. Hereinafter, the decoding apparatus may include an image decoding apparatus and / or a video decoding apparatus.

[0085] Reference Figure 3 The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor and an intra-frame predictor. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be configured as a single hardware component (e.g., a decoder chipset or processor). The memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0086] When the input includes a bitstream containing video / image information, the decoding device 300 can determine the appropriate response based on the input data. Figure 2 The decoding device 300 reconstructs the image by processing video / image information in the encoding device. For example, the decoding device 300 can derive units / blocks based on block partitioning information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied to the encoding device. Therefore, the processing unit used for decoding can be, for example, an encoding unit, and can be partitioned from encoding tree units or maximum encoding units according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding units. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced via a reproduction device.

[0087] Decoding device 300 can receive signals output from encoding device in the form of a bitstream, and the received signals can be decoded by entropy decoder 310. For example, entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). Furthermore, the video / image information may also include general constraint information. Decoding device can further decode the picture based on the information about the parameter sets and / or general constraint information. In this disclosure, the information and / or syntax elements transmitted / received by signals, as described below, can be decoded through a decoding process and can be obtained from the bitstream. For example, entropy decoder 310 can decode the information in the bitstream based on an encoding method (such as exponential Golomb coding, CAVLC, or CABAC) and can output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients for the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream. It can determine a context model using information about the target syntax element and decoding information about adjacent and target blocks, or information about symbols / bins decoded in previous stages. It can generate symbols corresponding to the values ​​of each syntax element by predicting the occurrence probability of bins based on the determined context model and performing arithmetic decoding on the bins. Here, after determining the context model, the CABAC entropy decoding method can update the context model using information about the decoded symbols / bins for the next symbol / bin. Prediction-related information from the information decoded by the entropy decoder 310 can be provided to the predictor 330, and the residual values ​​obtained through entropy decoding in the entropy decoder 310 (i.e., quantized transform coefficients and related parameter information) can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, or residual sample arrays). Filter-related information from the information decoded by the entropy decoder 310 can be provided to the filter 350. A receiver (not shown) for receiving signals output from the encoding device can also be configured as an internal / external element of the decoding device 300, or the receiver can be a component of the entropy decoder 310. The decoding device according to this disclosure can be referred to as a video / image / picture decoding device and can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, and a predictor 330.

[0088] Dequantizer 321 can dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the reordering can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0089] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block or residual sample array).

[0090] The predictor can perform predictions for the current block and generate prediction blocks that include prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on prediction-related information output from the entropy decoder 310, and determine a specific intra-frame / inter-frame prediction mode.

[0091] Predictor 330 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also simultaneous intra-frame prediction and inter-frame prediction, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for predicting blocks. The IBC prediction mode or palette mode can be used for, for example, screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame based on the block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0092] Intra-frame predictors can refer to samples within the current image to predict the current block. The referenced samples can be located as neighbors of the current block (adjacent to the current block) or, depending on the prediction mode, as distant from the current block. In intra-frame prediction, prediction modes can include multiple non-directional modes and multiple directional modes. The intra-frame predictor can determine the prediction mode applied to the current block based on the prediction modes applied to neighboring blocks.

[0093] An inter-frame predictor can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, and Bi prediction) information. In inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing within the current image and temporally adjacent blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and can derive the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode for the current block.

[0094] Adder 340 can add the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from predictor 330 to generate a reconstruction signal (reconstructed image, reconstruction block, or reconstruction sample array). When no residual exists for the processing target block, such as when a skip mode is applied, the prediction block can be used as a reconstruction block.

[0095] Adder 340 can be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current image, or it can be output after being filtered as described below, or used for inter-frame prediction of the next image.

[0096] Luminance mapping and chroma scaling (LMCS) can be applied to the image decoding process.

[0097] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be sent to memory 360, specifically to the DPB in memory 360. Various filtering methods can include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, and bilateral filtering.

[0098] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in the inter-frame predictor. Memory 360 can store motion information about blocks from which motion information in the current image is derived (or decoded) and / or about blocks in already reconstructed images. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and can send the reconstructed samples to the intra-frame predictor.

[0099] The embodiments described in this specification for the filter 260 and predictor 210 of the encoding device 200 can also be applied equivalently or accordingly to the filter 350 and predictor 330 of the decoding device 300.

[0100] As described above, prediction is performed during video encoding to improve compression efficiency. This prediction generates a prediction block that includes prediction samples for the current block, which is the target block for encoding. The prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived identically in both the encoding and decoding devices, and the encoding device can improve image encoding efficiency by signaling information about the residual between the original block and the prediction block (residual information) instead of the original sample values ​​of the original block to the decoding device. The decoding device can derive a residual block including residual samples based on the residual information, generate a reconstructed block including reconstructed samples by combining the residual block and the prediction block, and generate a reconstructed image including the reconstructed block.

[0101] Residual information can be generated through transformation and quantization processes. For example, an encoding device can derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and signal the relevant residual information to the decoding device (via a bitstream). The residual information may include information such as the values ​​and locations of the quantized transform coefficients, transform technique, transform kernel, and quantization parameters. The decoding device can perform dequantization / inverse transform processes based on the residual information to derive residual samples (or residual blocks). The decoding device can generate a reconstructed image based on the prediction block and the residual block. The encoding device can perform dequantization / inverse transform on the quantized transform coefficients to derive residual blocks for reference in inter-frame prediction of subsequent images, and can generate a reconstructed image based on the residual blocks.

[0102] In this disclosure, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficients may be referred to as transform coefficients. When transform / inverse transform is omitted, the transform coefficients may be referred to as coefficients or residual coefficients, or, for the sake of consistency in the expression, may still be referred to as transform coefficients.

[0103] Furthermore, in this disclosure, quantized transform coefficients and transform coefficients can be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, residual information can include information about the transform coefficients, and this information can be signaled via residual coding syntax. Transform coefficients can be derived based on residual information (or information about transform coefficients), and scaled transform coefficients can be derived by the inverse transform (scaling) of the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaled transform coefficients. These details can be equivalently applied to or described in other parts of this disclosure.

[0104] Intra-frame prediction can refer to generating a prediction sample for the current block based on reference samples within the image to which the current block belongs (hereinafter, the current image). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. Neighboring reference samples for the current block may include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, neighboring reference samples for the current block may include multiple rows of upper adjacent samples and multiple columns of left adjacent samples.

[0105] Some neighboring reference samples for the current block may not have been decoded yet, or may be unavailable. In this case, the decoder can configure the neighboring reference samples to be used for prediction by filling in or replacing unavailable samples with available samples.

[0106] When deriving neighboring reference samples, the predicted samples for the current block can be derived based on the neighboring reference samples and intra-prediction mode / type information. Here, the intra-prediction mode can indicate one of a non-directional prediction mode and a directional prediction mode representing the spatial correlation used for intra-prediction. The directional prediction mode can be referred to as the angular prediction mode, and the non-directional prediction mode can be referred to as the non-angular prediction mode. The intra-prediction type can indicate various prediction types used to perform intra-prediction. Intra-prediction types can include, for example, multiple reference lines (MRL), intra-segmentation sub-partitions (ISP), position-dependent intra-prediction (PDPC), matrix-weighted intra-prediction or matrix-based intra-prediction (MIP), cross-component linear model (CCLM), multi-model linear model (MMLM), decoder-side intra-mode derivation (DIMD), fusion of chroma intra-prediction modes, intra-template matching, fusion of template-based intra-mode derivation (TIMD), intra-prediction fusion, cross-component convolutional model (CCCM), cross-component prediction (CCP), and spatial geometric partitioning mode (SGPM). Depending on the situation, intra-prediction can be performed using intra-prediction mode and / or intra-prediction type.

[0107] Specifically, the intra-frame prediction process may include an intra-frame prediction mode / type determination operation, a reference sample derivation operation, and a prediction sample derivation operation based on the intra-frame prediction mode / type. Furthermore, post-filtering operations for the derived prediction samples may be performed as needed.

[0108] Figure 4 The intra-frame prediction process is illustrated.

[0109] Reference Figure 4 As described above, the intra-frame prediction process may include operations to determine the intra-frame prediction mode / type, operations to derive reference samples, and operations to perform intra-frame prediction (generate prediction samples). As described above, the intra-frame prediction process can be performed in both the encoding and decoding devices.

[0110] The encoding device determines the intra-frame prediction mode / type (S400). As described above, the encoding device may include an encoding device and / or a decoding device.

[0111] The encoding device can determine the intra prediction mode / type applicable to the current block from the various intra prediction modes / types described in this disclosure, and can generate prediction-related information. The prediction-related information may include intra prediction mode information indicating the intra prediction mode applied to the current block and / or intra prediction type information indicating the intra prediction type applied to the current block. The decoding device can determine the intra prediction mode / type applicable to the current block based on the prediction-related information.

[0112] For example, when applying intra-prediction, the intra-prediction mode to be applied to the current block can be determined based on the intra-prediction modes of neighboring blocks. For instance, the coding device can select one of the most probable mode (MPM) candidates from a list of most probable modes derived from the intra-prediction modes and / or additional candidate modes of the current block's neighboring blocks (e.g., the left and / or upper neighboring blocks) based on received index information, or it can select one of the remaining intra-prediction modes not included in the MPM candidates based on remaining MPM information (remaining intra-prediction mode information). The MPM list can be configured to include or exclude planar modes as candidates.

[0113] The encoding device can be configured with a list of most probable modes (MPMs) for the current block. The MPM list can also be referred to as the MPM candidate list. Here, MPM can refer to a mode used in intra-frame predictive mode coding to improve coding efficiency by considering the similarity between the current block and neighboring blocks.

[0114] The encoding device can perform prediction based on various intra-prediction modes and determine the optimal intra-prediction mode based on rate distortion optimization (RDO). In this case, the encoding device can determine the optimal intra-prediction mode using MPM candidates configured in the MPM list, or it can determine the optimal intra-prediction mode using any other intra-prediction modes besides those in the MPM list. Specifically, for example, when the intra-prediction type of the current block is not the normal intra-prediction type but a specific type (e.g., DIMD, TIMD, MRL, or ISP), the encoding device can determine the optimal intra-prediction mode by only considering the MPM candidates as intra-prediction mode candidates for the current block. That is, in this case, the intra-prediction mode for the current block can be determined only from the MPM candidates, and the MPM flag can be left unencoded / signaled. In this case, the decoding device can estimate that the MPM flag is 1 without receiving the MPM flag separately.

[0115] Typically, when the intra-prediction mode for the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device generates an MPM index (mpm idx) indicating one of the MPM candidates. When the intra-prediction mode for the current block is not even in the MPM list, the encoding device generates MPM remainder information (remaining intra-prediction mode information) indicating the same mode as the intra-prediction mode for the current block among the remaining intra-prediction modes (and planar modes) not included in the MPM list. MPM remainder information may include, for example, the intra_luma_mpm_remainder syntax element.

[0116] The decoding device obtains intra-prediction mode information from the bitstream. As described above, the intra-prediction mode information may include at least one of an MPM flag, an MPM index, and remaining MPM information (remaining intra-prediction mode information). The decoding device may configure an MPM list. The MPM list is configured to be the same as the MPM list configured by the encoding device. That is, the MPM list may include intra-prediction modes of adjacent blocks, or it may also include specific intra-prediction modes according to a predetermined method.

[0117] The decoding device can determine the intra-prediction mode for the current block based on the MPM list and intra-prediction mode information. For example, when the value of the MPM flag is 1, the decoding device can deduce the candidate indicated by the MPM index among the MPM candidates in the MPM list as the intra-prediction mode for the current block.

[0118] In another example, when the value of the MPM flag is 0, the decoding device can deduce the intra-prediction mode indicated by the remaining intra-prediction mode information (which may be called the MPM remaining information) in the remaining intra-prediction modes as the intra-prediction mode for the current block.

[0119] The encoding device derives reference samples for the current block (S410). The reference samples may include adjacent reference samples of the current block. The adjacent reference samples of the current block may include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, the adjacent reference samples of the current block may include multiple rows of adjacent samples above and multiple columns of adjacent samples to the left.

[0120] The encoding device performs intra-frame prediction on the current block to derive prediction samples (S430). The encoding device can derive prediction samples based on the intra-frame prediction mode / type and reference samples. The encoding device can derive reference samples from the reference samples of the current block according to the intra-frame prediction mode of the current block, and can derive prediction samples of the current block based on the reference samples.

[0121] Intra-prediction-based coding processes can schematically include, for example, the following.

[0122] Figure 5 An example of a video / image coding method based on intra-frame prediction is shown.

[0123] Reference Figure 5The process can be as follows: S500 can be executed by the predictor of the encoding device, S505 can be executed by the residual processor of the encoding device, and S510 or S515 can be executed by the entropy encoder of the encoding device. Specifically, prediction-related information can be derived by the predictor and encoded by the entropy encoder. Residual information can be derived by the residual processor and encoded by the entropy encoder. Residual information is information about residual samples. Residual information may include information about the quantized transform coefficients for the residual samples. As mentioned above, transform coefficients can be derived from residual samples by the transformer of the encoding device, and quantized transform coefficients can be derived from transform coefficients by the quantizer. The information about the quantized transform coefficients can be encoded in the entropy encoder through a residual encoding process.

[0124] The encoding device performs intra-prediction for the current block (S500). The encoding device derives the intra-prediction mode / type for the current block, derives reference samples for the current block, and generates prediction samples within the current block based on the intra-prediction mode / type and the reference samples. The processes for determining the intra-prediction mode / type, deriving adjacent reference samples, and generating prediction samples can be performed simultaneously, or one process can be performed before the other. The encoding device can determine the mode / type applicable to the current block from multiple intra-prediction modes / types. The encoding device can compare the RD costs for the intra-prediction mode / type and determine the optimal intra-prediction mode / type for the current block.

[0125] Encoding devices can perform a predictive sample filtering process. Predictive sample filtering can also be called post-filtering. Some or all of the predicted samples can be filtered through this process. In some cases, the predictive sample filtering process can be omitted.

[0126] The encoding device generates residual samples for the current block based on the predicted samples (S505). The encoding device can compare the predicted samples with the original samples of the current block based on the phase and derive the residual samples.

[0127] The encoding device can encode image / video information, including information about intra-frame prediction (prediction-related information) and / or information about residual samples (residual information) (S510 or S515). Prediction-related information may include information about the intra-frame prediction mode and information about the intra-frame prediction type. The encoding device can output the encoded image / video information as a bitstream. The output bitstream can be sent to the decoding device via a storage medium or network.

[0128] Residual information can include residual coding syntax elements. The coding device can derive quantized transform coefficients by transforming / quantizing the residual samples. The residual information can include information about the quantized transform coefficients.

[0129] As described above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks). To do this, the encoding device can derive (modified) residual samples by performing dequantization / inverse transform on the quantized transform coefficients. The reason for transforming / quantizing the residual samples and then performing dequantization / inverse transform on them is, as mentioned above, to derive the same residual samples as those derived in the decoding device. The encoding device can generate a reconstructed block that includes reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. A reconstructed image for the current image can be generated based on the reconstructed block. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0130] Decoding devices can perform operations corresponding to those performed by encoding devices. Intra-frame prediction-based video / image decoding processes may include, for example, the following.

[0131] Figure 6 An example of a video / image decoding method based on intra-frame prediction is shown.

[0132] Reference Figure 6 S600 can be executed by the entropy decoder of the decoding device, S610 can be executed by the predictor of the decoding device, S615 can be executed by the residual processor of the decoding device, and S620 can be executed by the adder or reconstructor of the decoding device.

[0133] Specifically, the decoding device obtains image / video information from the bitstream (S600). The image / video information may include prediction-related information and / or residual information.

[0134] The decoding device performs intra-frame prediction based on prediction-related information (S610). The decoding device can derive an intra-frame prediction mode / type for the current block based on the prediction-related information, derive a reference sample for the current block, and generate prediction samples within the current block based on the intra-frame prediction mode / type and the reference sample. In this case, the decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Some or all of the prediction samples can be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process can be omitted.

[0135] The decoding device performs residual processing based on the residual information (S615). The decoding device can derive residual samples for the current block based on the residual information. Specifically, the dequantizer of the residual processor can derive the transform coefficients by dequantizing the quantized transform coefficients derived based on the residual information, and the inverse transformer of the residual processor can derive residual samples for the current block by performing an inverse transform on the transform coefficients.

[0136] The decoding device generates a reconstructed block / image (S620). The decoding device can generate reconstructed samples for the current block based on predicted samples and / or residual samples, and can derive a reconstruction including the reconstructed samples. A reconstructed image for the current image can be generated based on the reconstructed block. As described above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0137] Information about the intra-prediction mode and / or the intra-prediction type can be encoded / decoded using the binarylization and encoding methods described in this disclosure. For example, information about the intra-prediction mode and / or the intra-prediction type can be binarylized using fixed-length binarylization, truncated Rice binarylization, or truncated unary binarylization. For example, information about the intra-prediction mode and / or the intra-prediction type can be encoded / decoded using entropy coding (e.g., CABAC or CAVLC).

[0138] Figure 7 An example of a directional intra-frame prediction mode is shown. Figure 7 An example can be shown that includes 65 directional intra-frame prediction modes.

[0139] Reference Figure 7 Directional intra-prediction modes can include modes #2 through #65. In this case, mode #50 could represent a vertical intra-prediction mode, and mode #18 could represent a horizontal intra-prediction mode. However, this example is for illustration, and the number and type of candidate intra-prediction modes can vary. One or more non-directional intra-prediction modes can be considered; for example, mode #0 could represent an intra-plane prediction mode (plane mode), and mode #1 could represent an intra-DC prediction mode (DC mode).

[0140] Intra-frame prediction modes can be assigned mode numbers, as shown in the table below, for example.

[0141] [Table 1]

[0142] According to this disclosure, various intra-prediction modes can be used for intra-prediction of the current block. As an example, intra-prediction modes can be derived based on decoder-side intra-mode derivation (DIMD) technology. DIMD can be referred to as DIMD type.

[0143] Figure 8 An example of template-based HoG computation in DIMD is shown.

[0144] Reference Figure 8In DIMD, a histogram of gradients (HoG) can be computed using a template that includes neighboring samples of the current block. For example, the HoG can be computed based on a template with n (e.g., 3) neighboring sample rows using horizontal / vertical Sobel filters. In this case, both horizontal and vertical Sobel filters can be applied to compute the HoG. When the templates are located in different CTUs, the Sobel filter may not be applied to the upper CTU boundary, or the DIMD technique may not be applied to the current block.

[0145] Different filters can be determined for HoG computation based on the block size. For example, when the block size is 4×4, 4×8, or 8×4, a 2×2 kernel filter can be used instead of a 3×3 Sobel filter for HoG computation.

[0146] An adaptive number of (neighboring) reference samples can be used for HoG calculations in DIMD. In this case, the number of sample rows in the DIMD template can be four or more.

[0147] Among candidate intra-prediction modes, the n intra-prediction modes with the highest histogram values ​​can be selected / extracted through HoG calculation. In this case, the n intra-prediction modes can be used to derive the prediction block for the current block. Furthermore, in this case, the n intra-prediction modes and non-directional modes (e.g., planar or DC modes) can be used to derive the prediction block. In this case, predictors can be derived for the corresponding intra-prediction modes, and the prediction block can be derived by performing a weighted sum / weighted average of the predictors. That is, through prediction fusion, the predictors of the n (e.g., 5) selected / extracted intra-prediction modes can be fused with the predictor of the planar mode.

[0148] For example, n can be 5. In this case, the 5 derived patterns with the highest HoG can be derived. The derived patterns can be called DIMD-based patterns or DIMD-derived patterns. For example, the number n can be determined based on different block sizes. For example, when the block's W When H is 128 or greater, n can be 7; otherwise, n can be 5.

[0149] For HoG calculations in DIMD, specific block vectors can be used. For example, block vectors can be derived based on the motion vectors or block vectors of neighboring blocks. A reference region at the shifted position can be derived using specific block vectors based on the current block's position, and the HoG can be calculated using (reconstructed) reference samples within the reference region. In this case, the template can be located within the reference region, or it can be positioned above / to the left of a neighboring reference region. In this case, the size of the reference region can be the same as the size of the current block. Alternatively, the size of the reference region can differ from the size of the current block. For example, the width of the reference region can be half the width of the current block, and / or the height of the reference region can be half the height of the current block. In this case, the size of the template can also vary within the range of the width and height of the reference region.

[0150] The n intra-frame modes derived from DIMD can be referred to as DIMD-derived modes or DIMD-based modes. DIMD-derived modes can be included as candidates in the MPM list (e.g., the PMPM list).

[0151] When a DIMD technique (or type) is applied to the current block, the first intra-mode derived from DIMD can be stored as the intra-mode for the target block and can be referenced when configuring the MPM list for subsequent blocks.

[0152] When a DIMD technique (or type) is applied to the current block, the first intra-frame mode derived from DIMD can be referred to as the mode used to select the transform kernel (or transform set).

[0153] For weighted and / or weighted average or predictive fusion, weights between predictors are required. According to embodiments of this disclosure, lookup tables (LUTs) can be used to derive these weights. While existing full HoG calculations can be used to compute dimdMode... i However, Habove and Hleft can be calculated separately to calculate dimdMode. i Is it highly dependent on the top template or the left template, that is, dimMode? i Is it position-dependent?

[0154] When dimMode i When it is not position-dependent, the weight (wDimd) i (wPlanar) can be determined based on the HoG size, as in traditional methods.

[0155] When dimMode i When the problem is position-dependent, sample-based mixing can be applied. In this case, weights can be applied differently based on the samples.

[0156] When the upper HoG (e.g., Habove) is equal to or greater than twice the left HoG (e.g., Hleft), the weights can be based on the following equation.

[0157] [Equation 1]

[0158] here, It can represent the difference between the upper HoG and the left HoG, or the absolute value of that difference.

[0159] When the left HoG (e.g., Hleft) is equal to or greater than twice the upper HoG (e.g., Habove), the weights can be based on the following equation.

[0160] [Equation 2]

[0161] here, It can represent the difference between the left HoG and the top HoG, or the absolute value of that difference.

[0162] For DIMD, an additional W column can be used when the upper-right adjacent reference sample is available, and an additional H row can be used when the lower-left adjacent reference sample is available. In other words, the template region including adjacent (reconstructed) samples used for HOG calculation can be based on the availability of adjacent (reconstructed) samples.

[0163] When deriving n intra-frame (prediction) modes in DIMD, n intra-frame mode predictors and planar predictors can be combined. However, when deriving only one intra-frame mode in DIMD, only the intra-frame mode can be used. Alternatively, even when deriving only one intra-frame mode in DIMD, the intra-frame mode can be combined with a planar mode predictor. For example, when the HoG value of the intra-frame mode with the highest HoG in DIMD is equal to or greater than a threshold, only one intra-frame mode can be derived. In another example, when the first HoG value of the intra-frame mode with the highest HoG in DIMD is a specific value or greater than the second HoG value of the intra-frame mode with the second highest HoG, only one intra-frame mode can be derived. Furthermore, among the modes derived in DIMD, n intra-frame prediction modes with HoG equal to or greater than a specific threshold can be selected.

[0164] For a block to which DIMD is applied, the pattern derived from the first DIMD can be stored as the intra-prediction pattern for the current block. The stored pattern can be referenced as the patterns of neighboring blocks referenced for deriving the intra-prediction pattern for subsequent blocks and / or as the pattern for selecting the transform kernel (or transform set).

[0165] There may be situations where the pattern derived by the first DIMD may differ from the actual prediction pattern of the current block (or the best prediction pattern with the highest correlation to the current block). Therefore, in addition to the first intra-prediction pattern derived by DIMD, it is necessary to investigate patterns that can be stored as intra-prediction patterns for the current block. For example, when the HoG of the pattern derived by the first DIMD is greater than a certain threshold, the first pattern can be stored as an intra-prediction pattern for the current block; otherwise, a plane or DC pattern can be stored as an intra-prediction pattern for the current block. Alternatively, by comparing the first predictor (first prediction sample) of the current block generated by DIMD with the second predictor (second prediction sample) generated by the pattern derived by the first DIMD, when the difference between the two is within a certain SAD / SATD, the pattern derived by the first DIMD can be stored as an intra-prediction pattern for the current block; otherwise, a plane or DC pattern can be stored as an intra-prediction pattern for the current block.

[0166] In another example, a candidate list can be configured using m intra-prediction modes derived from DIMD, and any mode in the candidate list can be stored. Here, m can be equal to n, or m can be 2 or 3. In this case, the first predictor (first prediction sample) of the current block generated by DIMD can be compared with the predictors based on the corresponding m intra-prediction modes, and the intra-prediction mode with the lowest SAD / SATD among the m intra-prediction modes can be stored as the intra-prediction mode for the current block.

[0167] Alternatively, for the first predictor (first prediction sample) derived by the DIMD technique, the first intra-prediction mode derived by the DIMD technique (i.e., the intra-prediction mode with the highest HoG based on the HoG of the Sobel filter re-derived for the prediction sample (the first intra-prediction mode)) can be stored as the intra-prediction mode for the current block. In this case, the first intra-prediction mode derived from the template of the current block can be different from the first intra-prediction mode derived from the predictor of the current block.

[0168] For DIMD, DIMD-related information can be communicated using signals. DIMD-related information may include at least one of the following: DIMD enable flag, DIMD application flag, and / or DIMD flag.

[0169] The DIMD enable flag (dimd_enable_flag) can be signaled via high-level syntax (e.g., SPS). When the value of the DIMD enable flag is 1, the DIMD apply flag (dimd_applied_flag) and / or the DIMD flag (dimd_flag) can be signaled.

[0170] The DIMD application flag can be signaled at the PPS, Picture Header (PH), Slice Header (SH), or CTU level. Even when DIMD is available, it can be enabled / disabled at the picture, slice, or CTU level via the DIMD application flag. The DIMD flag can be signaled at the CU, PU, ​​or TU level. The DIMD flag can be signaled prior to the MPM flag (or PMPM flag).

[0171] DIMD-related information can be communicated using signals, for example, as shown below.

[0172] The DIMD flag (dimd_flag) can be signaled before the PMPM flag (pmpm_flag). In this case, when the value of the DIMD flag is 0, the PMPM flag can be signaled, and when the value of the DIMD flag is 1, the signaling of the PMPM flag can be omitted, as can be represented, for example, in the table below.

[0173] [Table 2]

[0174] The DIMD flag (dimd_flag) can be signaled before the reference line index (intra_luma_ref_idx) (in the parsing order). When multiple adjacent reference sample lines are used for intra-prediction of the current block, the reference line index is information indicating one or more of the adjacent reference sample lines. The reference line index can be signaled when the DIMD flag is not 1. When the DIMD flag is not 1 and the reference sample line index indicates 0, the PMPM flag can be signaled, which can be expressed, for example, as shown in the table below.

[0175] [Table 3]

[0176] The DIMD flag (dimd_flag) can be signaled after the reference row index (intra_luma_ref_idx) (in the parsing order). The DIMD flag can be signaled when the reference row index is 0. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample row index indicates 0, as can be expressed, for example, as shown in the table below.

[0177] [Table 4]

[0178] Furthermore, as described above, the MPM list can be configured to efficiently signal the intra-prediction mode for the current block. In the method according to embodiments of this disclosure, multiple MPM lists can be configured. The multiple MPM lists may include a first MPM list and a second MPM list. The first MPM list may be referred to as the primary MPM (PMPM) list, and the second MPM list may be referred to as the secondary MPM (SMPM) list.

[0179] The first and second MPM lists can be configured according to predetermined criteria. For example, a generic MPM list containing k candidates can be configured first, the first n candidates in the generic MPM list can be included in the first MPM list, and the remaining m candidates can be included in the second MPM list. In one example, k can be 22, n can be 6, and / or m can be 16. For example, the first candidate in the generic MPM list (the candidate of the first entry) can always be a planar pattern. In another example, where the planar pattern is signaled based on a separate planar flag (or not_planar_flag), k can be 21, n can be 5, and / or m can be 16.

[0180] At least one of the remaining candidates, excluding the planar pattern, can be derived from adjacent blocks.

[0181] Figure 9 An example of adjacent blocks used to derive the MPM list is shown.

[0182] Reference Figure 9 The adjacent block may include at least one of the following: the left adjacent block (L), the bottom left adjacent block (BL), the top adjacent block (A), the top right adjacent block (AR), and / or the bottom left adjacent block (AL) of the current block.

[0183] Furthermore, the remaining candidates may include DIMD-based patterns (DIMD-derived intra-prediction patterns). For example, intra-prediction patterns derived from neighboring blocks and DIMD-based patterns (hereinafter referred to as sorted intra-prediction patterns) can be sorted in ascending order of SAD cost based on SAD cost (hereinafter referred to as sorted intra-prediction patterns), and then added to the general MPM list or the first MPM list. In this case, only up to a predetermined number (p) of sorted intra-prediction patterns can be extracted and added to the general MPM list or the first MPM list. Here, the SAD cost can be calculated based on the reconstruction samples of the template of the current block and the predictors derived by applying the candidate patterns to the template. Among the sorted intra-prediction patterns, directional prediction patterns can be referred to as sorted directional patterns. For example, p can be 5 or 6. Alternatively, p can be 7 or 8. Planar patterns and / or DC patterns can be excluded from the DIMD-based patterns. This also applies below. DIMD-based patterns can be referred to as DIMD-derived patterns.

[0184] For example, sorted prediction patterns, patterns obtained by adding offsets to (extracted) sorted orientation patterns or subtracting offsets from said (extracted) sorted orientation patterns, and predetermined default patterns can be added until all k entries are filled.

[0185] In another example, the sorting process can be omitted for some or all of the DIMD-based patterns. This is because DIMD-based patterns are highly correlated with the current block. Therefore, the sorting process can be omitted for some or all of the DIMD-based patterns, and the sorting process can be preferentially assigned to the front of the MPM list (e.g., a general MPM list or a first MPM list). In this case, planar patterns can be placed first, and some or all of the DIMD-based patterns can be assigned after the planar patterns. Alternatively, when planar patterns are not included in the MPM list, some or all of the DIMD-based patterns can be assigned first. Alternatively, DIMD-based patterns can be compared with intra-prediction patterns derived from neighboring blocks, and overlapping patterns can be preferentially assigned to the front of the MPM list because these patterns are highly correlated with the current block. That is, the pruning process and the classification process can be merged. For example, conventional methods require redundant pruning processes and separate sorting processes to remove redundant intra-prediction patterns derived from neighboring blocks (group 1) and DIMD-based patterns (group 2), while this method can omit the sorting process by prioritizing redundant patterns while performing redundancy checks.

[0186] Figure 10 Intra-prediction modes derived from neighboring blocks and DIMD-based modes are shown.

[0187] Reference Figure 10 If patterns B and C overlap between two groups, then patterns B and C can be assigned to the MPM list with priority. In this case, for example, patterns B and C can be assigned at the beginning of the MPM list, or assigned immediately after the planar patterns.

[0188] In this case, for example, the priority between pattern B and pattern C can be determined based on the priority in the DIMD-based patterns. Since the priority of DIMD-based patterns is already derived based on HoG, the priority can be determined without a separate SAD-based classification process.

[0189] Intra-prediction modes used to encode previous blocks within a specific region can be stored and reused based on history; this can be called history-based intra-prediction modes (HIPMs). HIPMs can be represented using various terms, such as history-based MPMs (HMPMs) or history-based candidate modes. The specific region can include, for example, CTUs, CTU rows, multiple CTU rows, slices, and tiles. Multiple CTU rows can include, for example, the nth CTU row and the (n-1)th CTU row where the current block is located. In this case, the HIPM list can be reordered based on the frequency of the intra-prediction modes. That is, intra-prediction modes with a high occurrence frequency within a specific region have a higher priority than intra-prediction modes with a low occurrence frequency and can be assigned to an earlier position in the list. When assigned to an earlier position in the list, a lower index value can be used to indicate the mode.

[0190] Figure 11 The intra-prediction mode of the previous block within a specific region is shown.

[0191] Reference Figure 11 When a specific pattern (e.g., pattern #50) has the highest frequency, the HIPM list can be configured to give that pattern high priority. This list can be referred to as a buffer. In this case, for example, pattern #28 has the next highest frequency and can therefore be assigned to the position after pattern #50.

[0192] When configuring the HIPM list, the intra-prediction mode of previous blocks within a specific region may be unavailable, for example, when previous blocks are coded based on inter-frame prediction (inter-frame coding). In this case, the intra-prediction mode of the previous blocks can be treated as planar mode or DC mode. Alternatively, the intra-prediction mode of previous blocks can be omitted when configuring the HIPM list.

[0193] The HIPM list can be used to configure the MPM list. For example, the MPM list can be configured based on x candidates from the HIPM list according to priority. That is, the MPM list can include x candidates from the HIPM list according to priority. Alternatively, the candidates for the MPM list can be derived based on candidates for intra-prediction modes derived from neighboring blocks (first group) and DIMD-based modes (second group) and / or candidates from the HIPM list (third group). In this case, a pruning and sorting process can be performed, and at most the top p items can be extracted through the sorting process.

[0194] As described above, a general MPM list, a first MPM list, and / or a second MPM list can exist. The second MPM list can be divided into four groups. In this case, for example, the PMPM flag can be signaled first, and when the PMPM flag is 1, the PMPM index can be signaled. The PMPM flag can indicate whether the intra-prediction mode for the current block exists in the first MPM list. For example, the GMPM flag can be signaled before the PMPM flag. In this case, when the GMPM flag is 1, the PMPM flag and SMPM flag (described below) can be signaled. When the PMPM flag is 0, the SMPM flag can be signaled first, and when the SMPM flag is 1, the group index (SMPM group index) for the second MPM list can be parsed / signed first, and the mode index (SMPM mode index) can be signaled / parsed subsequently. Information about the intra-prediction mode (e.g., GMPM flag, PMPM flag, PMPM index, SMPM flag, SMPM group index, and / or SMPM mode index) can be signaled using CU syntax. When the GMPM flag is 0, the PMPM and SMPM flags can be deduced to be 0 without signaling / coding. The GMPM flag can also be simply referred to as the MPM flag. The GMPM flag can be omitted depending on the situation.

[0195] For example, information about intra-frame prediction can be signaled as follows. This information can include one or more syntax elements. The same applies below.

[0196] [Table 5]

[0197] For example, smpm_group_idx can be binary-coded with a fixed length, and smpm_mode_idx can also be binary-coded with a fixed length.

[0198] In another example, smpm_group_idx can be truncated Rice (or truncated unary) binary. In this case, smpm_mode_idx can be fixed-length binary. Alternatively, the binary method for smpm_mode_idx can be determined based on the smpm_group_idx value. For example, when the smpm_group_idx value is 0 or 1, smpm_mode_idx can be truncated Rice (or truncated unary) binary, while in other cases, it can be fixed-length binary. smpm_mode_idx can be encoded based on a context model, in which case the context model for smpm_mode_idx can be determined differently based on the size of the current block and / or the smpm_group_idx value. For example, the context model can be indicated based on the context index increment (ctxInc), and the ctxInc for the bin of smpm_mode_idx can be determined differently, for example, based on the smpm_group_idx value as shown below. As described below, smpm_group_flag can be used to replace smpm_group_idx.

[0199] [Table 6]

[0200] [Table 7]

[0201] The second MPM list can be divided into two groups. In this case, the number of candidates in the first group can be equal to or different from the number of candidates in the second group. For example, the number of candidates in the second MPM list can be 16, 12, or 8. In this case, the first group can include 4 or 8 candidates, while the second group can include 8 candidates.

[0202] For example, `smpm_group_flag` can be used instead of `smpm_group_idx`. `smpm_group_flag` indicates whether the intra-prediction mode for the current block is included in the second candidate group of the second MPM list. The binary representation method for `smpm_mode_idx` can be determined based on the `smpm_group_flag` value. For example, when the `smpm_group_flag` value indicates the first group (e.g., a value of 1), `smpm_mode_idx` can be truncated Rice (or truncated unary) binary representation, and in other cases, `smpm_mode_idx` can be binary represented by a fixed length.

[0203] In this case, for example, information about intra-frame prediction can be communicated using signals as follows.

[0204] [Table 8]

[0205] You can use smpm_idc instead of smpm_flag and / or smpm_group_flag (or smpm_group_idx).

[0206] In this case, for example, information about intra-frame prediction can be communicated using signals as follows.

[0207] [Table 9]

[0208] `smpm_idc` indicates whether the intra-prediction mode for the current block is included in the second MPM list, and if so, which group the intra-prediction mode belongs to. `smpm_idc` can be truncated Rice (or truncated unary) binary.

[0209] The following table shows the binary representation of smpm_idc.

[0210] [Table 10]

[0211] smpm_mode_idx can also be called smpm_idx. smpm_idx can be truncated into Rice (or truncated unary) binary form.

[0212] The following table shows the binary representation of smpm_idx.

[0213] [Table 11]

[0214] Alternatively, smpm_idx can be binary based on smpm groups. For example, smpm_idx can be binary based on the smpm_idc value. For instance, when the smpm_idc value is 1, smpm_idx can be truncated Rice (or truncated unary) binary, and when the smpm_idc value is 2, smpm_idx can be fixed-length binary. In this case, for example, the number of candidates in the first group can be 4, and the number of candidates in the second group can be 8. Therefore, for the worst case, the maximum number of binary bins can be matched.

[0215] [Table 12]

[0216] [Table 13]

[0217] As mentioned above, considering the best case, the first group in the second MPM list can be binary-coded with variable length because it is highly relevant to the current block, and considering the worst case, the second group in the second MPM list can be binary-coded with fixed length because it is relatively less relevant to the current block.

[0218] Context-based encoding (e.g., CABAC) can be used for smpm_idx, in which case the context index (or context index increment) of bin 0 can be set differently depending on the situation. The context model of smpm_idx can be determined differently based on the size of the current block and / or the value of smpm_idc. For example, the context index (or context index increment) of bin 0 of smpm_idx can be set differently based on smpm_idc (or smpm_group_idx or smpm_group_flag).

[0219] For example, when the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is greater than 1, the context index increment of bin 0 of smpm_idx can be 1. When the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is 1, the context index increment of bin 0 of smpm_idx can be 0, or vice versa. In the table below, smpm_idc can be replaced by smpm_group_idx or smpm_group_flag. In this case, the value of smpm_group_idx or smpm_group_flag as a condition can be set to be 1 less than the value of smpm_idc.

[0220] [Table 14]

[0221] [Table 15]

[0222] [Table 16]

[0223] [Table 17]

[0224] Through the above context-based coding, the context model can be adaptively assigned to information related to the intra-frame prediction mode, and the entropy coding efficiency can be effectively increased.

[0225] According to embodiments of this disclosure, in addition to existing planar patterns, two or more oriented planar patterns can be added. The oriented planar patterns can include a horizontal planar pattern and a vertical planar pattern.

[0226] Figure 12 Examples of predictions based on horizontal plane patterns and examples of predictions based on vertical plane patterns are shown. Figure 12 In the image, (a) shows an example of prediction based on a horizontal plane pattern, and (b) shows an example of prediction based on a vertical plane pattern.

[0227] Reference Figure 12 In planar horizontal mode, predicted samples are generated using the left reference sample of the target sample in the current block and the top-right reference sample (TR) of the current block. In this case, predicted samples can be generated by performing horizontal linear interpolation using the left reference sample and the top-right reference sample (TR). For example, horizontal linear interpolation can be performed based on the following equation.

[0228] [Equation 3]

[0229] Here, pred(x, y) represents the predicted sample value at the (x, y) coordinate, W represents the width of the current block, and TR represents the x-coordinate of the top-right reference sample. rec(-1, y) can represent the value of the reconstructed reference sample at the (-1, y) coordinate, and rec(TR, -1) can represent the value of the reconstructed reference sample (i.e., the top-right reference sample) at the (TR, -1) coordinate. For reference, the above coordinates can represent the values ​​when the top-left sample position of the current block is (0, 0).

[0230] In planar vertical mode, predicted samples are generated using the top reference sample of the target sample in the current block and the bottom left reference sample of the current block. In this case, predicted samples can be generated by performing vertical linear interpolation using the top reference sample and the bottom left reference sample (BL). For example, vertical linear interpolation can be performed based on the following equation.

[0231] [Equation 4]

[0232] Here, pred(x, y) represents the predicted sample value at coordinates (x, y), H represents the height of the current block, and BL represents the y-coordinate of the lower-left reference sample. rec(x, -1) can represent the value of the reconstructed reference sample at coordinates (x, -1), and rec(-1, BL) can represent the value of the reconstructed reference sample (i.e., the lower-left reference sample) at coordinates (-1, BL). For reference, the above coordinates can represent the values ​​when the upper-left sample position of the current block is (0, 0).

[0233] Oriented planar modes can also include planar diagonal modes.

[0234] Figure 13 An example of prediction based on a planar diagonal pattern is shown.

[0235] Reference Figure 13 In the diagonal planar mode, predicted samples can be generated through linear interpolation of two reference samples in the diagonal direction. For example, predicted samples can be generated through linear interpolation of the upper-right and lower-left diagonal reference samples of the target sample within the current block. In this case, for the target sample located in the lower-right direction within the current block (e.g., P3, P6, P9, P7, P10, P11, P12, P13, P14, or P15), the upper-right and lower-left adjacent reference samples of the current block can be used. For example, TR and BL can be used to predict target samples P3, P6, P9, or P12. In this case, predicted samples can be generated through linear interpolation of TR and BL.

[0236] The availability of directional planar mode can be determined based on high-level syntax and block size. For example, directional planar mode enabling flags (e.g., `dir_planar_enabled_flag`) can be signaled using high-level syntax. Here, high-level syntax can include PPS, SPS, image headers, and slice headers. The `constraint_flag` for directional planar mode can be signaled within general constraint information. For example, general constraint information can include `no_dir_planar_constraint_flag`, which indicates whether directional planar mode is restricted.

[0237] In addition, when the block size meets, for example, width Oriented planar mode is allowed when height > 64, width <= 128, and height <= 128. Alternatively, when the block size meets the width requirement... Oriented planar mode is allowed when height > 128, width <= 256, and height <= 256. Alternatively, oriented planar mode is allowed only when the current block is not a square. Alternatively, horizontal and vertical planar modes are allowed when the current block is not a square, while diagonal planar mode is allowed when the current block is a square. Alternatively, horizontal planar mode is allowed when the width of the current block is greater than its height, and vertical planar mode is allowed when the height of the current block is greater than its width. Alternatively, horizontal planar mode is allowed when the width of the current block is less than its height, and vertical planar mode is allowed when the height of the current block is less than its width. Alternatively, horizontal planar mode is disallowed when the width of the current block is greater than its height by a certain threshold, and vertical planar mode is disallowed when the height of the current block is greater than its width by a certain threshold. Alternatively, when the width of the current block is less than a certain threshold or less than its height, a planar horizontal mode may be disallowed, and when the height of the current block is less than a certain threshold or less than its width, a planar vertical mode may be disallowed. Therefore, signaling for the selection information (e.g., directional_idx) described below can be omitted depending on the conditions.

[0238] When the Oriented Plane Mode Enable flag allows Oriented Plane Mode and / or when the block size condition is met, information about the Oriented Plane Mode, which will be described below, can be signaled.

[0239] When the current block is not square, directional planar patterns are allowed. For example, when the current block is square, prediction-related information may not include selection information. In this case, selection information can be deduced into specific values ​​without encoding / decoding.

[0240] For example, a horizontal planar mode might not be allowed when the width of the current block is greater than its height. In this case, the candidate modes may not include the horizontal planar mode. That is, selection information can be used to select one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar mode is applied to the current block.

[0241] For example, a planar vertical mode might not be allowed when the width of the current block is greater than its height. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0242] For example, a horizontal planar pattern might not be allowed if the width of the current block is greater than a first threshold or more than its height. In this case, the candidate patterns may not include the horizontal planar pattern. That is, selection information can be used to select one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar pattern is applied to the current block.

[0243] When the width of the current block is greater than a second threshold or more than its height, the planar vertical mode may be disallowed. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be the same as the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0244] For example, a horizontal planar mode might not be allowed when the height of the current block is greater than its width. In this case, the candidate modes may not include the horizontal planar mode. That is, selection information can be used to select one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar mode is applied to the current block.

[0245] For example, a planar vertical mode might not be allowed when the height of the current block is greater than its width. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0246] For example, a horizontal planar pattern might not be allowed if the height of the current block is greater than a first threshold or more than its width. In this case, the candidate patterns may not include the horizontal planar pattern. That is, selection information can be used to select one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar pattern is applied to the current block.

[0247] When the height of the current block is greater than a second threshold or more than its width, the planar vertical mode may be disallowed. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be the same as the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0248] Oriented planar modes (e.g., horizontal and vertical planar modes) can be signaled, for example, as follows.

[0249] [Table 18]

[0250] When a planar mode is applied to the current block, the horizontal and vertical planar modes can be signaled as additional modes.

[0251] The directional_flag is a flag indicating whether a directional plane mode (e.g., horizontal plane mode or vertical plane mode) is applied to the current block.

[0252] `directional_idx` indicates the selection information for the mode to be applied to the current block from among candidate modes, including planar horizontal and planar vertical modes. `directional_idx` can be referred to as, for example, `planar_hor_flag` or `planar_ver_flag`.

[0253] Flag information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMD-based pattern of the current block, and / or the flag information values ​​of the current block's neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment for the flag information can be derived based on the flag information values ​​of the left neighboring block and the above neighboring block.

[0254] Selection information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMD-based pattern of the current block, and / or the selection information values ​​of the current block's neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment can be determined based on the size of the current block, the DIMD-based pattern of the current block, and / or the selection information values ​​of the current block's neighboring blocks.

[0255] Furthermore, the if (INTRA_PLANAR) check is a conditional statement used to determine whether the planar pattern is applied to the current block, and can be replaced by if (!not_planar_flag) if the planar pattern is signaled separately.

[0256] [Table 19]

[0257] [Table 20]

[0258] You can replace mpm_flag with pmpm_flag, and you can replace mpm_idx with pmpm_idx.

[0259] You can (for example) truncate Rice (or truncate unary) binaryization of directional_idx.

[0260] [Table 21]

[0261] For example, type 1 could correspond to a horizontal planar mode, type 2 to a vertical planar mode, and type 3 to a diagonal planar mode. However, this example is for illustrative purposes, and the indexing and / or binarization of the directional_idx for the current block can be adaptively performed based on the size of the current block, the DIMD-based mode, etc. For example, when the height of the current block is greater than its width, type 1 could correspond to a horizontal planar mode, type 2 to a vertical planar mode, and type 3 to a diagonal planar mode. When the width of the current block is greater than its height, type 1 could correspond to a vertical planar mode, type 2 to a horizontal planar mode, and type 3 to a diagonal planar mode. When the current block is a square, type 1 could correspond to a diagonal planar mode, type 2 to a horizontal planar mode, and type 3 to a vertical planar mode. Alternatively, when the current block is a square, type 1 could correspond to a diagonal planar mode, type 2 to a vertical planar mode, and type 3 to a horizontal planar mode.

[0262] Instead of the omitted `directional_idx`, the pattern to be applied among candidate patterns, including planar horizontal and planar vertical patterns, can be determined based on TIMD or DIMD. For example, whether to apply the planar vertical or planar horizontal pattern can be implicitly determined based on the HoG derived from DIMD.

[0263] When determining the availability of planar horizontal mode and planar vertical mode by comparing the width and height of the current block as described above, it is possible to determine which mode, planar horizontal mode or planar vertical mode, should be applied in the absence of directional_idx signaling.

[0264] Syntax elements such as dir_planar_idc can be used instead of directional_flag and directional_idx. dir_planar_idc can be called planar_directional_idc or directional_idc.

[0265] [Table 22]

[0266] [Table 23]

[0267] The `dir_planar_idc` parameter indicates whether the current block should be treated as a normal planar, horizontal, or vertical planar. `dir_planar_idc` can be truncated (or unary) binary.

[0268] [Table 24]

[0269] When a directional plane mode (plane horizontal mode or plane vertical mode) is applied to the current block, intra-frame prediction for the current block storage can be determined based on a predetermined method.

[0270] For example, the intra-prediction mode of the current block (which is referenced in the process of deriving the intra-prediction mode for subsequent blocks and generating the MPM list) can be the (normal) planar mode.

[0271] When selecting a transform kernel for the primary / secondary (inverse) transform of the current block, the selection can also be based on the intra-prediction mode of the current block. For example, the transform set including the transform kernel can be determined based on the intra-prediction mode of the current block. In this case, when selecting the transform kernel, the referenced intra-prediction mode can be mapped as follows: if a horizontal planar mode is applied to the current block, it is mapped to a horizontal intra-prediction mode; if a vertical planar mode is applied to the current block, it is mapped to a vertical intra-prediction mode. Furthermore, the scan order for scanning (quantizing) the transform coefficients can be determined based on the intra-prediction mode of the current block. In this case, for example, to determine the scan order, the following can be considered: if a horizontal planar mode is applied to the current block, a horizontal intra-prediction mode is applied; if a vertical planar mode is applied to the current block, a vertical intra-prediction mode is applied. For example, when it is determined that a horizontal intra-prediction mode is applied, the scan order can be horizontal, and when it is determined that a vertical intra-prediction mode is applied, the scan order can be vertical.

[0272] In this case, the problem arises that the current block stores two or more intra-prediction modes, which leads to asymmetric memory buffer allocation in the implementation between cases where a horizontal or vertical planar mode is applied and cases where a horizontal or vertical planar mode is not applied.

[0273] Therefore, a method is needed to reconcile the intra prediction modes stored for reference when deriving intra prediction modes for subsequent blocks with the intra prediction modes stored for selecting the transform kernel for the current block. Thus, a method is needed that stores the prediction modes used to derive intra prediction modes for adjacent blocks as horizontal / vertical modes, or, as usual, uses planar modes as prediction modes for deriving the primary / secondary transform kernels, but changes the transform kernels within the primary / secondary transform set corresponding to the planar modes.

[0274] For example, the intra-prediction mode of the current block (which is referenced in the process of deriving the intra-prediction mode for subsequent blocks and generating the MPM list) can be mapped to a horizontal intra-prediction mode when a planar horizontal mode is applied to the current block, and to a vertical intra-prediction mode when a planar vertical mode is applied to the current block. The following diagram illustrates the mapping when a directional planar mode is applied to the current block.

[0275] [Table 25]

[0276] In other words, in this case, the number of intra-prediction modes that need to be stored for the current block can be reduced to one.

[0277] Alternatively, separate pattern numbers # can be assigned to the horizontal and vertical plane patterns as follows. These pattern numbers # are for illustrative purposes, and other reserved numbers may be assigned.

[0278] [Table 26]

[0279] In this case, the number of intra-prediction modes that need to be stored for the current block can be fixed at 1.

[0280] When referenced for deriving intra-prediction modes for subsequent blocks or for deriving transform kernels, a separate prediction mode mapping process can be performed.

[0281] For example, the storage can include the pattern numbers for horizontal and vertical plane modes, but in practice, these modes can be mapped and utilized as follows.

[0282] [Table 27]

[0283] When a directional planar mode is applied to the current block, the stored mode is a planar mode. However, when deriving the mode referenced for deriving the intra-prediction mode of subsequent blocks and / or for selecting the transform kernel (or transform set), the directional planar selection information (directional_idx or dir_planar_idc) can be checked, and the referenced mode can be adaptively derived based on this.

[0284] For example, as shown in the table below, when applying a directional plane mode (plane horizontal mode or plane vertical mode), the stored mode can be a plane mode.

[0285] [Table 28]

[0286] In this case, for example in the transformation kernel derivation operation, the value of the orientation plane selection information can be referenced, as shown in the following figure.

[0287] Figure 14 The derivation of the transformation kernel for oriented plane modes is shown.

[0288] Reference Figure 14 When `predModeIntra`, which is the intra-prediction mode for the current block, is equal to the intra-planar mode and `directional_flag` is 1, `predModeIntra` can be modified. In this case, for example, when `directional_idx` is 0 or a planar horizontal mode is applied, `predModeIntra` can be mapped to the intra-planar mode. Similarly, when `directional_idx` is 1 or a planar vertical mode is applied, `predModeIntra` can be mapped to the intra-planar mode. And when `directional_idx` is 2 or a planar diagonal mode is applied, `predModeIntra` can be mapped to the intra-planar mode.

[0289] When deriving the transform kernel for the current block, the selection of the transform set and / or the derivation of the transform kernel can be performed using predModeIntra after mapping.

[0290] The aforementioned method can solve the problem of asymmetric memory buffer allocation for directional plane modes, and can coordinate the intra-prediction modes stored for reference when deriving intra-prediction modes for subsequent blocks and the intra-prediction modes stored for selecting transform kernels for the current block based on a single stored intra-prediction mode.

[0291] Figure 15 A video / image encoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 15The method disclosed in the article can be derived from Figure 2 The encoding device disclosed in the document executes the code. Specifically, for example, Figure 15 S1500 to S1520 can be performed by the predictor 220 of the encoding device 200, and Figure 15 S1530 can be executed by the entropy encoder 240 of the encoding device 200. Figure 15 The methods disclosed herein may include the aforementioned embodiments.

[0292] Reference Figure 15 The encoding device determines the orientation plane mode for the current block (S1500). The encoding device determines whether to apply the orientation plane mode to the current block, and when the orientation plane mode is applied to the current block, a specific orientation plane mode can be derived.

[0293] The orientation plane mode may include at least one of the following: plane vertical mode, plane horizontal mode, or plane diagonal mode.

[0294] The encoding device generates a prediction sample for the current block based on the orientation plane pattern (S1510). In this case, the encoding device can derive multiple reference samples for the target sample within the current block based on the orientation plane pattern, and generate a prediction sample for the target sample based on the multiple reference samples. The prediction sample for the target sample can be generated using a distance-based weighted sum of multiple reference samples. In this case, the distance for the distance-based weighted sum can be either horizontal or vertical distance.

[0295] Multiple reference samples may include a first reference sample and a second reference sample.

[0296] As an example, assuming the orientation plane mode is a horizontal plane mode, the first reference sample can be located to the left of the target sample, and the second reference sample can be located to the upper right of the current block. The second reference sample can be located to the right of a third reference sample located above the target sample. The second reference sample can be adjacent to a sample located at the upper right sample position within the current block. For example, the second reference sample can be located above the sample at the upper right sample position within the current block. Alternatively, the second reference sample can be located to the upper right of the sample at the upper right sample position within the current block.

[0297] As another example, based on the case where the orientation plane mode is a planar vertical mode, the first reference sample can be located above the target sample, and the second reference sample can be located in the lower left direction of the current block. The second reference sample can be located below a third reference sample located to the left of the target sample. The second reference sample can be adjacent to a sample located at the lower left sample position within the current block. For example, the second reference sample can be located to the left of a sample at the lower left sample position within the current block. Alternatively, the second reference sample can be located to the lower left of a sample at the lower left sample position within the current block.

[0298] As another example, based on the case where the orientation plane mode is a diagonal planar mode, the first reference sample can be located in the upper right diagonal direction of the target sample, and the second reference sample can be located in the lower left diagonal direction of the target sample. The first reference sample can be located to the right of the third reference sample, which is located above the target sample. The second reference sample can be located below the fourth reference sample, which is located to the left of the target sample.

[0299] The number of (adjacent) reference samples used when the planar horizontal mode or the planar horizontal mode is applied to the current block can be set differently than the number of (adjacent) reference samples used when the planar diagonal mode is applied to the current block.

[0300] The encoding device generates prediction-related information based on the directional plane pattern (S1520). The prediction-related information may include the information disclosed in the table above. For example, the prediction-related information may include selection information indicating that one of the candidate patterns, including the horizontal plane pattern and the vertical plane pattern, is applied to the current block. In this case, for example, the number of candidate patterns may be determined based on the size of the current block.

[0301] When the current block is not square, directional planar patterns are allowed. For example, when the current block is square, prediction-related information may not include selection information. In this case, selection information can be deduced as a specific value without being encoded / decoded.

[0302] For example, when the width of the current block is greater than its height, a horizontal planar mode may not be allowed. In this case, the candidate modes may not include the horizontal planar mode. That is, the selection information can indicate one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that a vertical planar mode is applied to the current block.

[0303] For example, when the width of the current block is greater than its height, the planar vertical mode may not be allowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information can indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0304] For example, when the width of the current block is greater than a first threshold or more, the horizontal planar pattern may not be allowed. In this case, the candidate patterns may not include the horizontal planar pattern. That is, the selection information may indicate one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the vertical planar pattern is applied to the current block.

[0305] When the width of the current block is greater than a second threshold or more than its height, the planar vertical mode may be disallowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be equal to the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0306] For example, when the height of the current block is greater than its width, a horizontal planar mode may not be allowed. In this case, the candidate modes may not include the horizontal planar mode. That is, the selection information can indicate one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that a vertical planar mode is applied to the current block.

[0307] For example, when the height of the current block is greater than its width, the planar vertical mode may not be allowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information can indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0308] For example, when the height of the current block is greater than a first threshold or more, the horizontal planar pattern may not be allowed. In this case, the candidate patterns may not include the horizontal planar pattern. That is, the selection information may indicate one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the vertical planar pattern is applied to the current block.

[0309] When the height of the current block is greater than the second threshold or more than its width, the planar vertical mode may not be allowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information can select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be equal to the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0310] For example, oriented plane mode may be available when the product of the width and height of the current block is greater than 64 or 128. Additionally, oriented plane mode may be allowed when the width of the current block is 128 or 256 or less, and the height is 128 or 256 or less.

[0311] The availability of a planar mode can be determined based on high-level syntax and block size. For example, a planar mode enabling flag (e.g., `dir_planar_enabled_flag`) can be signaled using high-level syntax. Here, high-level syntax can include PPS, SPS, image headers, slice headers, etc. Furthermore, a constraint flag for the planar mode can be signaled within general constraint information. For example, general constraint information can include a `no_dir_planar_contraint_flag` indicating whether the planar mode is constrained.

[0312] Prediction-related information may include MPM marker information, non-planar marker information, MPM index information, etc.

[0313] The selection information for the current block can be indicated by a signal following the MPM flag information and / or non-planar flag information, indicating that one of the candidate modes, including the horizontal and vertical planar modes, is applied. The signaling order (parsing order) of the MPM index information is later than the signaling order (parsing order) of the selection information, and the MPM index information may not be signaled when the selection information is signaled. The MPM index information may include the aforementioned PMPM index information and / or SMPM index information (e.g., smpm_mode_idx, etc.).

[0314] Selection information can be based on truncated Rice binary. Selection information can also be based on truncated unary binary. Selection information can also be based on fixed-length binary. For example, as described in Table 21 above, the binary values ​​for the horizontal planar mode and the vertical planar mode can be set differently based on the size of the current block. As an example, when the height of the current block is greater than its width, the index value of the selection information for the horizontal planar mode can be 0, and the index value of the selection information for the vertical planar mode can be 1. As an example, when the width of the current block is greater than its height, the index value of the selection information for the vertical planar mode can be 0, and the index value of the selection information for the horizontal planar mode can be 1.

[0315] Prediction-related information may include flags indicating whether an orientation plane mode is applied to the current block. In this case, for example, when the orientation plane flag value is 1, selection information can be signaled. Based on the flag value being 1, it can be determined, based on the size of the current block, which mode—horizontal or vertical—is applied.

[0316] Flag information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMM-based pattern of the current block, and / or the values ​​of flag information for the current block's neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment for the flag information can be derived based on the values ​​of the flag information for the left neighboring block and the flag information for the top neighboring block.

[0317] Selection information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMM-based pattern of the current block, and / or the values ​​of selection information for neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment can be determined based on the size of the current block, the DIMM-based pattern of the current block, and / or the values ​​of selection information for neighboring blocks.

[0318] The scan order applied to the current block can be set differently based on the orientation plane mode. The scan order applied to the current block can include the scan order of the (quantized) transform coefficient scans for the current block. As an example, when the horizontal plane mode is applied, a first scan order is applied as the scan order of the transform coefficient scans for the current block, and when the vertical plane mode is applied to the current block, a second scan order is applied as the scan order of the transform coefficient scans for the current block, and the second scan order can be different from the first scan order. Additionally, when the diagonal plane mode is applied to the current block, a third scan order is applied as the scan order of the transform coefficient scans for the current block, and the third scan order can be different from the first scan order and / or the second scan order. As an example, the first scan order can be a horizontal scan order, the second scan order can be a vertical scan order, and the third scan order can be a diagonal scan order. As another example, the first scan order can be a vertical scan order, the second scan order can be a horizontal scan order, and the third scan order can be a diagonal scan order.

[0319] For example, when the orientation plane mode is applied to the current block, the intra prediction mode stored for the current block and the intra prediction mode used to select the transform set for the current block can be based on at least one of Tables 25 to 28 above.

[0320] For example, when the orientation plane mode is applied to the current block, the intra prediction mode stored for the current block can be different from the intra prediction mode used to select the transform set for the current block.

[0321] For example, when the directional plane mode is applied to the current block, the transform set selected for the current block can be equal to the transform set selected for the current block when either the horizontal intra-prediction or the vertical intra-prediction is applied to the current block.

[0322] The encoding device encodes image information, including prediction-related information (S1530). For example, the prediction-related information may include the CU syntax for the current block. The image information may be referred to as video information.

[0323] Furthermore, according to embodiments of this disclosure, image information may include various types of information. For example, image information may include information disclosed in at least one of the tables described above.

[0324] In addition, image information may include residual information. Residual information is information about the residual samples. Residual information may include information about the quantized transform coefficients for the residual samples.

[0325] Encoded image information can be output as a bitstream. The bitstream can be sent to a decoding device via a network or storage medium. For example, image data including a bitstream can be sent to the decoding device by a sending device (or transmitter). In this case, the image data including the bitstream can be sent to the decoding device via a streaming server.

[0326] Furthermore, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This is to derive the same prediction result in the encoding device as the prediction result performed in the decoding device, thereby improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples, reconstructed blocks) in memory and use it as a reference image for inter-frame prediction. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0327] According to the above implementation, orientation plane modes can be efficiently indicated. Furthermore, the problem of asymmetric memory buffer allocation for orientation plane modes can be solved, and the intra-prediction modes stored for reference when deriving intra-prediction modes for subsequent blocks based on a stored intra-prediction mode, as well as the intra-prediction modes stored for selecting the transform kernel of the current block, can be coordinated.

[0328] Figure 16 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 16 The method disclosed in the article can be derived from Figure 3 The decoding device disclosed in the document performs the operation. Specifically, for example, Figure 16 S1600 can be executed by the entropy decoder 310 of the decoding device 300, and S1610 to S1620 can be executed by the predictor 330 of the decoding device 300. Figure 16 The methods disclosed herein may include the aforementioned embodiments.

[0329] Reference Figure 16 The decoding device obtains prediction-related information through the bitstream (S1600). The decoding device can obtain image information including prediction-related information through the bitstream. As mentioned above, the image information may also include residual information.

[0330] Prediction-related information may include the information disclosed in the table above. For example, prediction-related information may include selection information indicating that one of the candidate patterns, including horizontal and vertical plane patterns, is applied to the current block. In this case, for example, the number of candidate patterns may be determined based on the size of the current block.

[0331] When the current block is not square, directional planar patterns are allowed. For example, when the current block is square, prediction-related information may not include selection information. In this case, selection information can be deduced as a specific value without being encoded / decoded.

[0332] For example, when the width of the current block is greater than its height, a horizontal planar mode may not be allowed. In this case, the candidate modes may not include the horizontal planar mode. That is, the selection information can indicate one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that a vertical planar mode is applied to the current block.

[0333] For example, when the width of the current block is greater than its height, the planar vertical mode may not be allowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information can indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0334] For example, when the width of the current block is greater than a first threshold or more, the horizontal planar pattern may not be allowed. In this case, the candidate patterns may not include the horizontal planar pattern. That is, the selection information may indicate one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the vertical planar pattern is applied to the current block.

[0335] When the width of the current block is greater than a second threshold or more than its height, the planar vertical mode may be disallowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be equal to the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0336] For example, when the height of the current block is greater than its width, a horizontal planar mode may not be allowed. In this case, the candidate modes may not include the horizontal planar mode. That is, the selection information can indicate one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that a vertical planar mode is applied to the current block.

[0337] For example, when the height of the current block is greater than its width, the planar vertical mode may not be allowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information can indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0338] For example, when the height of the current block is greater than a first threshold or more, the horizontal planar pattern may not be allowed. In this case, the candidate patterns may not include the horizontal planar pattern. That is, the selection information may indicate one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the vertical planar pattern is applied to the current block.

[0339] When the height of the current block is greater than a second threshold or more than its width, the planar vertical mode may not be allowed. In this case, the candidate modes may not include the planar vertical mode. That is, the selection information may indicate one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be equal to the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0340] For example, oriented plane mode can be available when the product of the width and height of the current block is greater than 64 or 128. Additionally, oriented plane mode can be allowed when the width of the current block is less than or equal to 128 or 256, and the height is less than or equal to 128 or 256.

[0341] The availability of a planar mode can be determined based on high-level syntax and block size. For example, a planar mode enabling flag (e.g., `dir_planar_enabled_flag`) can be signaled using high-level syntax. Here, high-level syntax can include PPS, SPS, image headers, slice headers, etc. Furthermore, a constraint flag for the planar mode can be signaled within general constraint information. For example, general constraint information can include a `no_dir_planar_contraint_flag` indicating whether the planar mode is constrained.

[0342] Prediction-related information may include MPM marker information, non-planar marker information, MPM index information, etc.

[0343] The selection information for the current block can be indicated by a signal following the MPM flag information and / or non-planar flag information, indicating that one of the candidate modes, including the horizontal and vertical planar modes, is applied. The signaling order (parsing order) of the MPM index information is later than the signaling order (parsing order) of the selection information, and the MPM index information may not be signaled when the selection information is signaled. The MPM index information may include the aforementioned PMPM index information and / or SMPM index information (e.g., smpm_mode_idx, etc.).

[0344] Selection information can be based on truncated Rice binary. Selection information can also be based on truncated unary binary. Selection information can also be based on fixed-length binary. For example, as described in Table 21 above, the binary values ​​for the horizontal planar mode and the vertical planar mode can be set differently based on the size of the current block. As an example, when the height of the current block is greater than its width, the index value of the selection information for the horizontal planar mode can be 0, and the index value of the selection information for the vertical planar mode can be 1. As an example, when the width of the current block is greater than its height, the index value of the selection information for the vertical planar mode can be 0, and the index value of the selection information for the horizontal planar mode can be 1.

[0345] Prediction-related information may include flags indicating whether an orientation plane mode is applied to the current block. In this case, for example, when the orientation plane flag value is 1, selection information can be signaled. Based on the flag value being 1, it can be determined which mode—horizontal or vertical—is applied, based on the size of the current block.

[0346] Flag information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMM-based pattern of the current block, and / or the values ​​of flag information for the current block's neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment for the flag information can be derived based on the values ​​of the flag information for the left neighboring block and the flag information for the top neighboring block.

[0347] Selection information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMM-based pattern of the current block, and / or the values ​​of selection information for neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment can be determined based on the size of the current block, the DIMM-based pattern of the current block, and / or the values ​​of selection information for neighboring blocks.

[0348] The scan order applied to the current block can be set differently based on the orientation plane mode. The scan order applied to the current block can include the scan order of the (quantized) transform coefficient scans for the current block. As an example, when the horizontal plane mode is applied, a first scan order is applied as the scan order of the transform coefficient scans for the current block, and when the vertical plane mode is applied to the current block, a second scan order is applied as the scan order of the transform coefficient scans for the current block, and the second scan order can be different from the first scan order. Additionally, when the diagonal plane mode is applied to the current block, a third scan order is applied as the scan order of the transform coefficient scans for the current block, and the third scan order can be different from the first scan order and / or the second scan order. As an example, the first scan order can be a horizontal scan order, the second scan order can be a vertical scan order, and the third scan order can be a diagonal scan order. As another example, the first scan order can be a vertical scan order, the second scan order can be a horizontal scan order, and the third scan order can be a diagonal scan order.

[0349] For example, when the orientation plane mode is applied to the current block, the intra prediction mode stored for the current block and the intra prediction mode selected for the transform set of the current block can be based on at least one of Tables 25 to 28 above.

[0350] For example, when the orientation plane mode is applied to the current block, the intra prediction mode stored for the current block and the intra prediction mode selected for the transform set of the current block can be different.

[0351] For example, the transform set selected for the current block when the directional plane mode is applied can be the same as the transform set selected for the current block when horizontal intra-prediction or vertical intra-prediction is applied.

[0352] The decoding device derives the orientation plane pattern of the current block based on prediction-related information (S1610). The decoding device determines whether the orientation plane pattern is applied to the current block, and when the orientation plane pattern is applied to the current block, it can derive a specific orientation plane pattern.

[0353] As described above, the orientation plane mode may include at least one of the following: plane vertical mode, plane horizontal mode, or plane diagonal mode.

[0354] The decoding device generates a prediction sample based on the orientation plane pattern (S1620). In this case, the decoding device can derive multiple reference samples for the target sample within the current block based on the orientation plane pattern, and generate a prediction sample for the target sample based on the multiple reference samples. The prediction sample for the target sample can be generated using a distance-based weighted sum of the multiple reference samples. In this case, the distance for the distance-based weighted sum can be either a horizontal distance or a vertical distance.

[0355] Multiple reference samples may include a first reference sample and a second reference sample.

[0356] As an example, assuming the orientation plane mode is a horizontal plane mode, the first reference sample can be located to the left of the target sample, and the second reference sample can be located to the upper right of the current block. The second reference sample can be located to the right of a third reference sample located above the target sample. The second reference sample can be adjacent to a sample located at the upper right sample position within the current block. For example, the second reference sample can be located above a sample at the upper right sample position within the current block. Alternatively, the second reference sample can be located to the upper right of a sample at the upper right sample position within the current block.

[0357] As another example, assuming the orientation plane mode is a planar vertical mode, the first reference sample can be located above the target sample, and the second reference sample can be located in the lower left direction of the current block. The second reference sample can be located below a third reference sample located to the left of the target sample. The second reference sample can be adjacent to a sample located at the lower left sample position within the current block. For example, the second reference sample can be located to the left of a sample at the lower left sample position within the current block. Alternatively, the second reference sample can be located to the lower left of a sample at the lower left sample position within the current block.

[0358] As another example, based on the case where the orientation plane mode is a diagonal planar mode, the first reference sample can be located in the upper right diagonal direction of the target sample, and the second reference sample can be located in the lower left diagonal direction of the target sample. The first reference sample can be located to the right of the third reference sample, which is located above the target sample. The second reference sample can be located below the fourth reference sample, which is located to the left of the target sample.

[0359] The number of (adjacent) reference samples used when the planar horizontal mode or the planar horizontal mode is applied to the current block can be set differently than the number of (adjacent) reference samples used when the planar diagonal mode is applied to the current block.

[0360] As mentioned above, depending on the circumstances, a prediction sample filtering process can be further performed for all or some of the prediction samples of the current block.

[0361] The decoding device can generate reconstructed samples based on the predicted samples of the current block. For example, the decoding device can generate reconstructed samples for the current block based on the residual samples and predicted samples for the current block. Residual samples for the current block can be generated based on the received residual information. Additionally, as an example, the decoding device can generate a reconstructed image including the reconstructed samples. As described above, in-loop filtering processes, etc., can then be further applied to the reconstructed image.

[0362] According to the above implementation, orientation plane modes can be efficiently indicated. Furthermore, the problem of asymmetric memory buffer allocation for orientation plane modes can be solved, and the intra-prediction modes stored for reference when deriving intra-prediction modes for subsequent blocks based on a stored intra-prediction mode, as well as the intra-prediction modes stored for transform kernel selection in the current block, can be coordinated.

[0363] Although the method is described as a series of steps or blocks based on the flowchart in the foregoing embodiments, the embodiments are not limited to the order of the steps, and a step may occur in a different order or simultaneously with another step described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps may be deleted without affecting the scope of the embodiments of this disclosure.

[0364] The aforementioned methods according to embodiments of the present disclosure can be implemented in software, and the encoding and / or decoding devices according to the present disclosure can be included in apparatus for performing image processing, such as TVs, computers, smartphones, set-top boxes, and display devices.

[0365] The embodiments described above can be implemented in the form of a recording medium including computer-executable (program) instructions, such as program modules executed by a computer. These modules can be stored in memory and executed by a processor. The memory can reside internally or externally to the processor and can be connected to the processor via various known means. The computer-readable medium can be any available medium accessible to a computer and can include volatile and non-volatile media, as well as removable and non-removable media. Furthermore, the computer-readable medium can include both computer storage media and communication media. Computer storage media can include volatile and non-volatile media, as well as removable and non-removable media, implemented using any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically include computer-readable instructions, data structures, program modules, other data in modulated data signals such as carrier waves, or other transmission mechanisms, and include any information delivery medium.

[0366] Furthermore, the embodiments of this disclosure described above can be implemented as a computer program (or computer program product) including computer-executable instructions. The computer program may include programmable machine instructions processed by a processor and may be implemented using a high-level programming language, an object-oriented programming language, assembly language, or machine language. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical media, or solid-state drive (SSD)).

[0367] Therefore, when the above-described computer program is executed by a computing device, embodiments of the present disclosure can be implemented. The computing device may include at least some of a processor, memory, storage devices, high-speed interfaces connected to the memory and high-speed expansion ports, and low-speed interfaces connected to low-speed buses and storage devices. These components may be interconnected via various buses and may be mounted on a common motherboard or otherwise suitably mounted.

[0368] The processor can process instructions within a computing device. These instructions may include instructions stored in memory or a storage device to display graphical information for providing a graphical user interface (GUI) on an external input / output device, such as a display connected to a high-speed interface. In another embodiment, multiple processors and / or multiple buses may be suitably used with multiple memories and memory types. Furthermore, the processor can be implemented as a chipset comprising multiple independent analog and / or digital processor chips.

[0369] Memory stores information within a computing device. For example, memory may include volatile memory cells or a set of volatile memory cells. In another example, memory may include non-volatile memory cells or a set of non-volatile memory cells. Memory may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0370] Storage devices can provide high-capacity storage space for computing devices. Storage devices can be computer-readable media or components that include computer-readable media. For example, storage devices can include devices within a storage area network (SAN) or other components, and can be floppy disk devices, hard disk devices, optical disk devices, magnetic tape devices, flash memory, other similar semiconductor memory devices, or device arrays.

[0371] The network can be implemented as a wired network (such as a local area network (LAN), a wide area network (WAN), or a value-added network (VAN)) or various types of wireless networks (such as mobile radio communication networks or satellite communication networks).

[0372] Although this disclosure has been described with reference to embodiments shown in the accompanying drawings, these embodiments are merely exemplary. Those skilled in the art will understand that various modifications and variations of the embodiments are possible. That is, the scope of this disclosure is not limited to the described embodiments, and various modifications and changes made by those skilled in the art based on the basic concepts defined in the appended claims also fall within the scope of the claims. Therefore, the true technical scope of this disclosure should be determined by the technical spirit of the appended claims.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: Obtain prediction-related information through bitstream; Based on the predicted information, derive the directional plane pattern for the current block; as well as Based on the directional plane pattern, a prediction sample is generated for the current block. The orientation plane mode includes at least one of a plane vertical mode, a plane horizontal mode, or a plane diagonal mode.

2. The image decoding method according to claim 1, wherein, The step of generating the predicted sample for the current block based on the directional plane pattern includes: Based on the directional plane pattern, derive multiple reference samples for the target sample within the current block; and A predicted sample for the target sample is generated based on the multiple reference samples.

3. The image decoding method according to claim 2, wherein, The plurality of reference samples includes a first reference sample and a second reference sample, and Specifically, this is based on the case where the directional plane mode is a horizontal plane mode. The first reference sample is located to the left of the target sample, and The second reference sample is located in the upper right direction of the current block.

4. The image decoding method according to claim 2, wherein, The plurality of reference samples includes a first reference sample and a second reference sample, and Specifically, this is based on the case where the orientation plane mode is a planar vertical mode. The first reference sample is located above the target sample, and The second reference sample is located in the lower left direction of the current block.

5. The image decoding method according to claim 2, wherein, The plurality of reference samples includes a first reference sample and a second reference sample, and Specifically, this is based on the case where the directional plane mode is a diagonal plane mode. The first reference sample is located diagonally to the upper right of the target sample, and The second reference sample is located in the lower left diagonal direction of the target sample.

6. The image decoding method according to claim 1, wherein, The prediction-related information includes selection information, which indicates that one of the candidate patterns, including the horizontal plane pattern and the vertical plane pattern, is applied to the current block. The number of candidate patterns is determined based on the size of the current block.

7. The image decoding method according to claim 1, wherein, The orientation plane mode is allowed when the current block is not a square.

8. The image decoding method according to claim 7, wherein, When the width of the current block is greater than the height of the current block, the horizontal planar mode is not allowed, and Specifically, the planar vertical mode is not allowed when the width of the current block is greater than the height of the current block.

9. The image decoding method according to claim 7, wherein, When the width of the current block is greater than the height of the current block by a first threshold or more, the horizontal planar mode is not allowed, and Specifically, the planar vertical mode is not allowed when the width of the current block is greater than the height of the current block by a second threshold or more.

10. The image decoding method according to claim 1, wherein, The orientation plane mode is available when the product of the width and height of the current block is greater than 64 or 128.

11. The image decoding method according to claim 1, wherein, The prediction-related information includes selection information, which indicates that one of the candidate patterns, including the horizontal plane pattern and the vertical plane pattern, is applied to the current block. The selection information is signaled after the most probable mode (MPM) flag information.

12. The image decoding method according to claim 1, wherein, The prediction-related information includes flag information indicating whether the orientation plane pattern is applied to the current block, and Specifically, if the value of the flag information is 1, then the size of the current block determines whether the horizontal plane mode or the vertical plane mode is applied.

13. The image decoding method according to claim 1, wherein, The prediction-related information includes selection information, which indicates that one of the candidate patterns, including the horizontal plane pattern and the vertical plane pattern, is applied to the current block. The selection information is based on truncated Rice binary.

14. The image decoding method according to claim 13, wherein, Based on the size of the current block, the binary values ​​for the horizontal plane mode and the binary values ​​for the vertical plane mode are set differently.

15. The image decoding method according to claim 14, wherein, Since the height of the current block is greater than the width of the current block, the index value of the selection information for the horizontal plane mode is 0, and the index value of the selection information for the vertical plane mode is 1.

16. The image decoding method according to claim 1, wherein, Based on the plane horizontal pattern being applied to the current block, the first scan order is applied as the scan order for the transform coefficient scans of the current block. Wherein, based on the planar vertical pattern applied to the current block, the second scan order is applied as the scan order of the transform coefficient scan for the current block, and The second scanning order is different from the first scanning order.

17. The image decoding method according to claim 1, wherein, Based on the orientation plane mode being applied to the current block, the intra-prediction mode stored for the current block is different from the intra-prediction mode selected for the transform set of the current block.

18. The image decoding method according to claim 1, wherein, When the orientation plane mode is applied to the current block, the transform set selected for the current block is the same as the transform set selected for the current block when horizontal intra-prediction or vertical intra-prediction is applied to the current block.

19. An image encoding method performed by an encoding device, the image encoding method comprising the following steps: Determine the orientation plane mode for the current block; Based on the directional plane pattern, a prediction sample is generated for the current block; Generate prediction-related information based on the directional plane pattern; as well as The image information, including the prediction-related information, is encoded. The orientation plane mode includes at least one of a plane vertical mode, a plane horizontal mode, or a plane diagonal mode.

20. A method for transmitting image data, the method comprising the following steps: Obtaining a bitstream generated by an image encoding method, wherein the image encoding method includes the following steps: determining a directional plane pattern for the current block, generating prediction samples for the current block based on the directional plane pattern, generating prediction-related information based on the directional plane pattern, and encoding image information including the prediction-related information; and Send the image data including the bitstream. The orientation plane mode includes at least one of a plane vertical mode, a plane horizontal mode, or a plane diagonal mode.

Images