Image encoding method using template-based intra prediction mode derivation and apparatus therefor

Abstract

The image decoding method according to embodiments of the present disclosure includes the following steps: determining a filter for a template applied to a current block; deriving n intra-prediction modes for the current block based on the template and the filter; and generating a prediction block for the current block based on the n intra-prediction modes and a non-directional mode, wherein the n intra-prediction modes include a directional intra-prediction mode and the non-directional mode is a planar mode or a DC mode.

Description

Technical Field

[0001] This disclosure relates to image / video coding methods and image / video coding devices. Background Technology

[0002] Image / video encoding is used in a variety of applications, such as digital storage media, television broadcasting, video streaming services, and real-time communications. Recently, the demand for high-resolution and high-quality images / videos has been increasing across various fields.

[0003] As images / videos achieve higher resolution and quality, their data size increases, leading to a relative increase in the amount of information or bits transmitted. Consequently, the costs of transmission and storage rise when using conventional media such as wired and wireless broadband lines to transmit image data or when using existing storage media to store image / video data.

[0004] Recently, there has been an increasing interest in and demand for immersive media such as virtual reality (VR), augmented reality (AR), mixed reality (MR) content, and holograms. Furthermore, there is a growing number of attempts to use immersive media to deliver immersive experiences in gaming, education, healthcare, real estate, and marketing.

[0005] Therefore, efficient image / video compression technology is needed to efficiently compress, send, store, and play high-resolution and high-quality image / video information with various characteristics. Summary of the Invention

[0006] Technical solution

[0007] According to embodiments of this disclosure, a method and apparatus for improving video / image coding efficiency are provided.

[0008] According to embodiments of this disclosure, a video / image coding method and a video / image coding apparatus based on intra-frame prediction are provided.

[0009] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes the following steps: determining a filter for a template applied to a current block; deriving n intra-frame prediction modes for the current block based on the template and the filter; and generating a prediction block for the current block based on the n intra-frame prediction modes and a non-directional mode, wherein the n intra-frame prediction modes include a directional intra-frame prediction mode, and the non-directional mode is a planar mode or a DC mode.

[0010] According to embodiments of this disclosure, an image encoding method performed by an encoding device is provided. The method includes the following steps: determining a filter for a template applied to a current block; deriving n intra-frame prediction modes for the current block based on the template and the filter; generating a prediction block for the current block based on the n intra-frame prediction modes and a non-directional mode; generating prediction-related information; and encoding image information including the prediction-related information, wherein the n intra-frame prediction modes include directional intra-frame prediction modes, and the non-directional modes are planar modes or DC modes.

[0011] According to embodiments of the present disclosure, a decoding apparatus for image decoding is provided. The decoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: determining a filter for a template applied to a current block; deriving n intra-frame prediction modes for the current block based on the template and the filter; and generating a prediction block for the current block based on the n intra-frame prediction modes and a non-directional mode, wherein the n intra-frame prediction modes include a directional intra-frame prediction mode, and the non-directional mode is a planar mode or a DC mode.

[0012] According to embodiments of the present disclosure, an encoding apparatus for image encoding is provided. The encoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: determining a filter for a template applied to a current block; deriving n intra-frame prediction modes for the current block based on the template and the filter; generating a prediction block for the current block based on the n intra-frame prediction modes and a non-directional mode; generating prediction-related information; and encoding image information including the prediction-related information, wherein the n intra-frame prediction modes include directional intra-frame prediction modes, and the non-directional modes are planar modes or DC modes.

[0013] According to embodiments of the present disclosure, a method for transmitting video / image data is provided, the video / image data comprising a bitstream generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0014] According to embodiments of the present disclosure, an apparatus for transmitting video / image data is provided, the video / image data comprising a bitstream generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0015] According to embodiments of the present disclosure, a computer-readable storage medium is provided that stores a program for performing a method according to at least one embodiment of the present disclosure.

[0016] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded video / image information generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0017] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded information or encoded video / image information, the encoded information or encoded video / image information causing a decoding device to perform a video / image decoding method according to at least one embodiment of the present disclosure.

[0018] Beneficial effects

[0019] According to the embodiments of this disclosure, the overall video / image compression efficiency can be improved.

[0020] According to embodiments of this disclosure, prediction performance for the current block can be improved.

[0021] According to embodiments of this disclosure, intra-prediction patterns for the current block can be efficiently derived based on templates.

[0022] According to embodiments of this disclosure, the optimal prediction block for the current block can be derived while reducing side information for intra-frame prediction mode signaling.

[0023] According to embodiments of this disclosure, an intra-prediction mode for the current block storage performing template-based intra-prediction and / or an intra-prediction mode for selecting the transform kernel for the current block can be efficiently determined.

[0024] According to embodiments of this disclosure, intra-prediction patterns derived from DIMD can be used as representative patterns for specific prediction techniques or inter-prediction blocks, thereby improving correlation with neighboring blocks in intra-prediction. Attached Figure Description

[0025] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0026] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied.

[0027] Figure 3 This diagram schematically illustrates the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0028] Figure 4 The intra-frame prediction process is illustrated.

[0029] Figure 5 An example of a video / image coding method based on intra-frame prediction is shown.

[0030] Figure 6 An example of a video / image decoding method based on intra-frame prediction is shown.

[0031] Figure 7 An example of a directional intra-frame prediction mode is shown.

[0032] Figure 8 An example of adjacent blocks used to derive the MPM list is shown.

[0033] Figure 9 Intra-prediction modes derived from neighboring blocks and DIMD-based modes are shown.

[0034] Figure 10 The intra-prediction mode of the previous block within a specific region is shown.

[0035] Figure 11 Examples of predictions based on a horizontal plane pattern and examples of predictions based on a vertical plane pattern are shown.

[0036] Figure 12 An example of prediction based on a planar diagonal pattern is shown.

[0037] Figure 13 The derivation of the transformation kernel for oriented plane modes is shown.

[0038] Figure 14 This shows an example of TIMD application when the current block is a square block.

[0039] Figure 15 This shows an example of TIMD application when the current block is a non-square block.

[0040] Figure 16 An example of intra-frame TMP technology is shown.

[0041] Figure 17 An example of SGPM's partitioning mode is shown.

[0042] Figure 18 An example of a candidate SGPM signaling is shown.

[0043] Figure 19 An example of an SGPM template is shown.

[0044] Figure 20 The differences between candidates based on the SGPM transpose flag are shown.

[0045] Figure 21 This shows an example of intraTMP being applied to a partition of the current block where SGPM is applied.

[0046] Figure 22An example of template-based HoG computation in DIMD is shown.

[0047] Figure 23 A video / image encoding method according to an embodiment of the present disclosure is illustrated schematically.

[0048] Figure 24 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Detailed Implementation

[0049] Because this disclosure can have various variations and implementations, specific embodiments are illustrated in the accompanying drawings and will be described in detail. However, it should be understood that the embodiments of this disclosure are not intended to be limited to a particular embodiment. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the technical spirit of this disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise. As used herein, the term “and / or” includes any one and all combinations of two or more associated listed items. As used herein, the terms “comprising,” “including,” and “having” specify the presence of stated features, numbers, operations, elements, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, numbers, operations, elements, components, and / or combinations thereof. In this disclosure, the use of the term “may” in connection with examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature, but all examples are not limited thereto, and the corresponding feature or configuration may be omitted.

[0050] For ease of explanation of the different features, each component in the accompanying drawings described in this disclosure is shown independently, but this does not mean that each component is implemented as a separate piece of hardware or separate piece of software. For example, two or more of these components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which components are integrated and / or separated are also included within the scope of this disclosure without departing from its spirit.

[0051] In this disclosure, “A or B” may mean “A only”, “B only”, or “both A and B”. In other words, “A or B” in this disclosure may be interpreted as “A and / or B”. For example, “A, B or C” in this disclosure may mean “A only”, “B only”, “C only”, or “any one and all combinations of A, B and C”.

[0052] The forward slash ( / ) or comma used in this article can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0053] In this disclosure, "at least one of A and B" may mean "only A", "only B" or "both A and B". Furthermore, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted in the same way as "at least one of A and B".

[0054] In this disclosure, "at least one of A, B, and C" may mean "only A", "only B", "only C", or "any one and all combinations of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" may mean "at least one of A, B, and C".

[0055] The parentheses used in this disclosure can indicate "for example". Specifically, when indicated as "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" in this disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when indicated as "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0056] In this disclosure, the technical features explained individually in a single figure may be implemented independently or simultaneously.

[0057] This disclosure relates to video / image coding. For example, the methods / implementations described in this disclosure can be applied to methods disclosed in enhanced compression models or the H.267 standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the AOMedia Video2 (AV2) standard or next-generation video / image coding standards (e.g., H.268 and H.269).

[0058] In this disclosure, encoding may include encoding and / or decoding. In this disclosure, image encoding may be used interchangeably with video encoding.

[0059] In this disclosure, video can refer to a collection of images over time. An image typically refers to a unit representing a single image at a specific time, while a slice / tile refers to a unit that forms part of an image during encoding. A slice / tile may include one or more coding tree units (CTUs). A single image may include one or more slices / tiles. A tile may represent a rectangular area of ​​a CTU within a specific tile row and column of an image.

[0060] A single image can be divided into two or more sub-images. A sub-image can be a rectangular region of one or more slices of the image.

[0061] A pixel or cell can refer to the smallest unit that makes up a picture (or image). The term "sample" can be used in contrast to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the luminance component (pixel / pixel value) or only the chrominance component (pixel / pixel value).

[0062] A unit can represent a basic unit of image processing. A unit may include a specific region of an image and at least one of the information associated with that region. A single unit may include a luminance block and two chrominance (e.g., Cb and cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M ​​columns and N rows.

[0063] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, throughout the drawings, the same reference numerals may be used to denote similar elements, and redundant descriptions of similar elements may be omitted.

[0064] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0065] Reference Figure 1 A video / image encoding system may include a first device (encoding device) and a second device (decoding device). The first device may deliver encoded video / image information or data to the second device in the form of a file or stream via a digital storage medium or network.

[0066] A video / image encoding system may also include a video / image acquisition device and a video / image renderer. The video / image acquisition device may be included in the encoding device, or it may be configured as a separate device or an external component. The video / image renderer may be included in the decoding device, or it may be configured as a separate device or an external component.

[0067] The first device may include a transmitter as an internal component, or a transmitter as a separate device or an external component.

[0068] The second device may include a receiver as an internal component, or as a receiver as a separate device or an external component.

[0069] An encoding device can be called an encoder, and a decoding device can be called a decoder. A transmitter can be included in the encoding device. A receiver can be included in the decoding device. A renderer can include a display, and the display can be configured as a separate device or an external component.

[0070] Decoding and encoding devices using embodiments of this disclosure can be included in multimedia broadcasting transmitters / receivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices such as video communication, mobile streaming devices, storage media, portable cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, Internet streaming service providers, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicle terminals), aircraft terminals, and ship terminals), and medical video devices, and can be used to process video signals or data signals. For example, over-the-top (OTT) video devices can include game consoles, Blu-ray players, Internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0071] A video / image acquisition device can acquire video / image sources. The video / image acquisition device can acquire video / images through processes of capturing, compositing, or generating video / images. The video / image acquisition device may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras and a video / image archive including previously captured video / images. The video / image generation device may include, for example, a portable camcorder, a computer, a tablet PC, and a smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, in which case the process of generating relevant data can replace the video / image capture process. The video / image source can perform a video / image preprocessing process to input the optimized video / image into the encoder.

[0072] Encoding devices can encode input video / images. For compression and encoding efficiency, encoding devices can perform a series of processes such as prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output as a bitstream.

[0073] A transmitter can send encoded images / image information or data, output as a bitstream, to a receiver in a receiving device via a digital storage medium or network, either as a file or a stream. The encoded images / image information or data output as a bitstream can be sent to the receiver via a streaming server. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0074] During the sending or receiving of bitstreams, the streaming server may temporarily store the bitstreams. The streaming server sends multimedia data to the user's device via a web server based on user requests, and the web server acts as an intermediary for notifying the user of available services. When a user requests a desired service from the web server, the web server forwards the request to the streaming server, and the streaming server sends the multimedia data to the user. The content streaming system may include a separate control server, in which case the control server controls the commands / responses between devices in the content streaming system.

[0075] A streaming server can receive content from media storage devices and / or encoding devices. For example, when receiving content from an encoding device, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.

[0076] Decoding devices can decode video / images by performing a series of processes corresponding to the operations of encoding devices, such as dequantization, inverse transform, and prediction.

[0077] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.

[0078] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied. In the following, the encoding apparatus may include image encoding apparatus and / or video encoding apparatus.

[0079] Reference Figure 2The encoding device 200 may include an image partitioner 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor and an intra-frame predictor. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image partitioner 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured as at least one hardware component (e.g., an encoder chipset or processor). The memory 270 may include a decoded picture buffer (DPB) or may be configured as a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0080] Image partitioner 210 can partition an input image (or picture or frame) input to encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, the coding unit can be recursively partitioned from coding tree unit (CTU) or maximum coding unit (LCU) according to a quadtree-binary-tritree (QTBTTT) structure. For example, a single coding unit can be partitioned into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, and a binary tree structure and / or a ternary structure can be applied later. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding unit that is no longer partitioned. In this case, based on the encoding efficiency according to the image characteristics, the maximum coding unit can be used as the final coding unit, or if necessary, the coding unit can be recursively partitioned into deeper coding units, thereby using the coding unit with the optimal size as the final coding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described below. In another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be divided or partitioned from the final encoding unit described above. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving the residual signal from the transform coefficients.

[0081] Depending on the context, the term "unit" can be used interchangeably with the terms "area" or "region". Generally, an M×N block can represent an array of samples or transform coefficients arranged in M ​​rows and N columns. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. "Sample" can be used as a term corresponding to a pixel or cell in a single picture (or image).

[0082] Encoding device 200 generates a residual signal (residual signal, residual block, or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the predictor from the input image signal (original block or original sample array), and sends the generated residual signal to converter 232. In this case, as shown, the component in encoder 200 used to subtract the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) can be referred to as subtractor 231. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The predictor can generate various information about the prediction, such as prediction mode information, and can send the generated information to entropy encoder 240, as described below in the description of each prediction mode. The information about the prediction can be encoded by entropy encoder 240 and output as a bitstream.

[0083] An intra-frame predictor can refer to samples within the current image to predict the current block. The referenced samples can be located as neighboring (adjacent to) the current block, or, depending on the prediction mode, as being located away from the current block. In intra-frame prediction, prediction modes can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC modes and planar modes. Depending on the granularity of the prediction direction, directional modes can include, for example, 33 or 65 directional prediction modes. However, this example is for illustration only, and more or fewer directional prediction modes can be used depending on the configuration. The intra-frame predictor can determine the prediction mode applied to the current block based on the prediction modes applied to neighboring blocks.

[0084] Inter-frame predictors can derive predicted blocks for the current block based on reference blocks (reference sample arrays) specified by motion vectors on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted based on the correlation between motion information between neighboring blocks and the current block, at the block, sub-block, or sample level. Motion information can include motion vectors and reference image indices. Motion information can also include inter-frame prediction direction (L0 prediction, L1 prediction, and Bi prediction) information. In inter-frame prediction, neighboring blocks can include spatially adjacent blocks existing within the current image and temporally adjacent blocks existing in the reference image. The reference image including the reference block and the reference image including the temporally adjacent block can be the same or different. A temporally adjacent block can be referred to as a co-located reference block or a co-located CU (colCU), and a reference image including a temporally adjacent block can also be referred to as a co-located image (colPic). For example, the inter-frame predictor can configure a motion information candidate list based on neighboring blocks and can generate information indicating candidates for deriving motion vectors and / or reference image indices for the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip and merge modes, the inter-frame predictor can use motion information about neighboring blocks as motion information about the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using motion vectors from neighboring blocks as motion vector predictors and signaling the motion vector difference.

[0085] Predictor 220 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also simultaneous intra-frame prediction and inter-frame prediction, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for predicting blocks. The IBC prediction mode or palette mode can be used for, for example, screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame based on the block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0086] The predicted signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graphical-Based Transform (GBT), or Conditional Nonlinear Transform (CNT).

[0087] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal as a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector based on the coefficient scan order, and can generate information about the transform coefficients based on the one-dimensional vector form of the quantized transform coefficients. Entropy encoder 240 can perform various encoding methods, such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). Entropy encoder 240 can encode information required for video / image reconstruction (e.g., values ​​of syntax elements) other than the quantized transform coefficients, either together or separately. The encoded information (e.g., encoded video / image information) can be sent or stored as a bitstream based on the Network Abstraction Layer (NAL). The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Furthermore, the video / image information may also include general constraint information. In this disclosure, information and / or syntax elements transmitted from the encoding device / signed to the decoding device may be included in the video / image information. The video / image information may be encoded by the aforementioned encoding process and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) and / or memory (not shown) for transmitting and / or storing signals output from the entropy encoder 240 may be configured as internal / external components of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0088] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 234 and inverse transform unit 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from the predictor to generate a reconstructed signal (reconstructed image, reconstructed block, or array of reconstructed samples). When no residual is available for processing the target block (e.g., when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current image, or for inter-frame prediction of the next image after filtering, as described below.

[0089] Luminance mapping and chroma scaling (LMCS) can be applied to image encoding and / or reconstruction processing.

[0090] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270, specifically in the DPB of memory 270. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, and bilateral filtering. Filter 260 can generate filtering-related information and can send the generated information to entropy encoder 240. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.

[0091] The modified reconstructed image sent to memory 270 can be used as a reference image in the inter-frame predictor. When inter-frame prediction is applied via the modified reconstructed image, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can improve encoding efficiency.

[0092] The DPB of memory 270 can store modified reconstructed images for use as reference images in the inter-frame predictor. Memory 270 can store motion information about blocks in the current image from which motion information is derived (or encoded) and / or about blocks in already reconstructed images. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current image and can send these reconstructed samples to the intra-frame predictor.

[0093] Figure 3This diagram schematically illustrates the configuration of a video / image decoding apparatus to which embodiments of the present disclosure may be applied. Hereinafter, the decoding apparatus may include an image decoding apparatus and / or a video decoding apparatus.

[0094] Reference Figure 3 The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor and an intra-frame predictor. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to embodiments, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be configured as a single hardware component (e.g., a decoder chipset or processor). The memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0095] When the input includes a bitstream containing video / image information, the decoding device 300 can determine the appropriate response based on the input data. Figure 2 The decoding device 300 reconstructs the image by processing video / image information in the encoding device. For example, the decoding device 300 can derive units / blocks based on block partitioning information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied to the encoding device. Therefore, the processing unit used for decoding can be, for example, an encoding unit, and can be partitioned from encoding tree units or maximum encoding units according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding units. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced via a reproduction device.

[0096] Decoding device 300 can receive signals output from encoding device in the form of a bitstream, and the received signals can be decoded by entropy decoder 310. For example, entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). Furthermore, the video / image information may also include general constraint information. Decoding device can further decode the picture based on the information about the parameter sets and / or general constraint information. In this disclosure, the information and / or syntax elements transmitted / received by signals, as described below, can be decoded through a decoding process and can be obtained from the bitstream. For example, entropy decoder 310 can decode the information in the bitstream based on an encoding method (such as exponential Golomb coding, CAVLC, or CABAC) and can output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients for the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream. It can determine a context model using information about the target syntax element and decoding information about adjacent and target blocks, or information about symbols / bins decoded in previous stages. It can generate symbols corresponding to the values ​​of each syntax element by predicting the occurrence probability of bins based on the determined context model and performing arithmetic decoding on the bins. Here, after determining the context model, the CABAC entropy decoding method can update the context model using information about the decoded symbols / bins for the next symbol / bin. Prediction-related information from the information decoded by the entropy decoder 310 can be provided to the predictor 330, and the residual values ​​obtained through entropy decoding in the entropy decoder 310 (i.e., quantized transform coefficients and related parameter information) can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, or residual sample arrays). Filter-related information from the information decoded by the entropy decoder 310 can be provided to the filter 350. A receiver (not shown) for receiving signals output from the encoding device can also be configured as an internal / external element of the decoding device 300, or the receiver can be a component of the entropy decoder 310. The decoding device according to this disclosure can be referred to as a video / image / picture decoding device and can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, and a predictor 330.

[0097] Dequantizer 321 can dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the reordering can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can perform dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information) and obtain the transform coefficients.

[0098] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block or residual sample array).

[0099] The predictor can perform predictions for the current block and generate prediction blocks that include prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on prediction-related information output from the entropy decoder 310, and determine a specific intra-frame / inter-frame prediction mode.

[0100] Predictor 330 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also simultaneous intra-frame prediction and inter-frame prediction, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for predicting blocks. The IBC prediction mode or palette mode can be used for, for example, screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame based on the block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0101] Intra-frame predictors can refer to samples within the current image to predict the current block. The referenced samples can be located as neighbors of the current block (adjacent to the current block) or, depending on the prediction mode, as distant from the current block. In intra-frame prediction, prediction modes can include multiple non-directional modes and multiple directional modes. The intra-frame predictor can determine the prediction mode applied to the current block based on the prediction modes applied to neighboring blocks.

[0102] An inter-frame predictor can derive a predicted block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference image. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference image indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, and Bi prediction) information. In inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing within the current image and temporally adjacent blocks existing in the reference image. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and can derive the motion vector and / or reference image index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode for the current block.

[0103] Adder 340 can add the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from predictor 330 to generate a reconstruction signal (reconstructed image, reconstruction block, or reconstruction sample array). When no residual exists for the processing target block, such as when a skip mode is applied, the prediction block can be used as a reconstruction block.

[0104] Adder 340 can be referred to as a reconstructor or reconstruction block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current image, or it can be output after being filtered as described below, or used for inter-frame prediction of the next image.

[0105] Luminance mapping and chroma scaling (LMCS) can be applied to the image decoding process.

[0106] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be sent to memory 360, specifically to the DPB in memory 360. Various filtering methods can include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, and bilateral filtering.

[0107] The (modified) reconstructed image stored in the DPB of memory 360 can be used as a reference image in the inter-frame predictor. Memory 360 can store motion information about blocks from which motion information in the current image is derived (or decoded) and / or about blocks in already reconstructed images. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current image and can send the reconstructed samples to the intra-frame predictor.

[0108] The embodiments described in this specification for the filter 260 and predictor 210 of the encoding device 200 can also be applied equivalently or accordingly to the filter 350 and predictor 330 of the decoding device 300.

[0109] As described above, prediction is performed during video encoding to improve compression efficiency. This prediction generates a prediction block that includes prediction samples for the current block, which is the target block for encoding. The prediction block includes prediction samples in the spatial domain (or pixel domain). The prediction block is derived identically in both the encoding and decoding devices, and the encoding device can improve image encoding efficiency by signaling information about the residual between the original block and the prediction block (residual information) instead of the original sample values ​​of the original block to the decoding device. The decoding device can derive a residual block including residual samples based on the residual information, generate a reconstructed block including reconstructed samples by combining the residual block and the prediction block, and generate a reconstructed image including the reconstructed block.

[0110] Residual information can be generated through transformation and quantization processes. For example, an encoding device can derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and signal the relevant residual information to the decoding device (via a bitstream). The residual information may include information such as the values ​​and locations of the quantized transform coefficients, transform technique, transform kernel, and quantization parameters. The decoding device can perform dequantization / inverse transform processes based on the residual information to derive residual samples (or residual blocks). The decoding device can generate a reconstructed image based on the prediction block and the residual block. The encoding device can perform dequantization / inverse transform on the quantized transform coefficients to derive residual blocks for reference in inter-frame prediction of subsequent images, and can generate a reconstructed image based on the residual blocks.

[0111] In this disclosure, at least one of quantization / dequantization and / or transform / inverse transform may be omitted. When quantization / dequantization is omitted, the quantized transform coefficients may be referred to as transform coefficients. When transform / inverse transform is omitted, the transform coefficients may be referred to as coefficients or residual coefficients, or, for the sake of consistency in the expression, may still be referred to as transform coefficients.

[0112] Furthermore, in this disclosure, quantized transform coefficients and transform coefficients can be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, residual information can include information about the transform coefficients, and this information can be signaled via residual coding syntax. Transform coefficients can be derived based on residual information (or information about transform coefficients), and scaled transform coefficients can be derived by the inverse transform (scaling) of the transform coefficients. Residual samples can be derived based on the inverse transform (scaling) of the scaled transform coefficients. These details can be equivalently applied to or described in other parts of this disclosure.

[0113] Intra-frame prediction can refer to generating a prediction sample for the current block based on reference samples within the image to which the current block belongs (hereinafter, the current image). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. Neighboring reference samples for the current block may include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, neighboring reference samples for the current block may include multiple rows of upper adjacent samples and multiple columns of left adjacent samples.

[0114] Some neighboring reference samples for the current block may not have been decoded yet, or may be unavailable. In this case, the decoder can configure the neighboring reference samples to be used for prediction by filling in or replacing unavailable samples with available samples.

[0115] When deriving neighboring reference samples, the predicted samples for the current block can be derived based on the neighboring reference samples and intra-prediction mode / type information. Here, the intra-prediction mode can indicate one of a non-directional prediction mode and a directional prediction mode representing the spatial correlation used for intra-prediction. The directional prediction mode can be referred to as the angular prediction mode, and the non-directional prediction mode can be referred to as the non-angular prediction mode. The intra-prediction type can indicate various prediction types used to perform intra-prediction. Intra-prediction types can include, for example, multiple reference lines (MRL), intra-segmentation sub-partitions (ISP), position-dependent intra-prediction (PDPC), matrix-weighted intra-prediction or matrix-based intra-prediction (MIP), cross-component linear model (CCLM), multi-model linear model (MMLM), decoder-side intra-mode derivation (DIMD), fusion of chroma intra-prediction modes, intra-template matching, fusion of template-based intra-mode derivation (TIMD), intra-prediction fusion, cross-component convolutional model (CCCM), cross-component prediction (CCP), and spatial geometric partitioning mode (SGPM). Depending on the situation, intra-prediction can be performed using intra-prediction mode and / or intra-prediction type.

[0116] Specifically, the intra-frame prediction process may include an intra-frame prediction mode / type determination operation, a reference sample derivation operation, and a prediction sample derivation operation based on the intra-frame prediction mode / type. Furthermore, post-filtering operations for the derived prediction samples may be performed as needed.

[0117] Figure 4 The intra-frame prediction process is illustrated.

[0118] Reference Figure 4 As described above, the intra-frame prediction process may include operations to determine the intra-frame prediction mode / type, operations to derive reference samples, and operations to perform intra-frame prediction (generate prediction samples). As described above, the intra-frame prediction process can be performed in both the encoding and decoding devices.

[0119] The encoding device determines the intra-frame prediction mode / type (S400). As described above, the encoding device may include an encoding device and / or a decoding device.

[0120] The encoding device can determine the intra prediction mode / type applicable to the current block from the various intra prediction modes / types described in this disclosure, and can generate prediction-related information. The prediction-related information may include intra prediction mode information indicating the intra prediction mode applied to the current block and / or intra prediction type information indicating the intra prediction type applied to the current block. The decoding device can determine the intra prediction mode / type applicable to the current block based on the prediction-related information.

[0121] For example, when applying intra-prediction, the intra-prediction mode to be applied to the current block can be determined based on the intra-prediction modes of neighboring blocks. For instance, the coding device can select one of the most probable mode (MPM) candidates from a list of most probable modes derived from the intra-prediction modes and / or additional candidate modes of the current block's neighboring blocks (e.g., the left and / or upper neighboring blocks) based on received index information, or it can select one of the remaining intra-prediction modes not included in the MPM candidates based on remaining MPM information (remaining intra-prediction mode information). The MPM list can be configured to include or exclude planar modes as candidates.

[0122] The encoding device can be configured with a list of most probable modes (MPMs) for the current block. The MPM list can also be referred to as the MPM candidate list. Here, MPM can refer to a mode used in intra-frame predictive mode coding to improve coding efficiency by considering the similarity between the current block and neighboring blocks.

[0123] The encoding device can perform prediction based on various intra-prediction modes and determine the optimal intra-prediction mode based on rate distortion optimization (RDO). In this case, the encoding device can determine the optimal intra-prediction mode using MPM candidates configured in the MPM list, or it can determine the optimal intra-prediction mode using any other intra-prediction modes besides those in the MPM list. Specifically, for example, when the intra-prediction type of the current block is not the normal intra-prediction type but a specific type (e.g., DIMD, TIMD, MRL, or ISP), the encoding device can determine the optimal intra-prediction mode by only considering the MPM candidates as intra-prediction mode candidates for the current block. That is, in this case, the intra-prediction mode for the current block can be determined only from the MPM candidates, and the MPM flag can be left unencoded / signaled. In this case, the decoding device can estimate that the MPM flag is 1 without receiving the MPM flag separately.

[0124] Typically, when the intra-prediction mode for the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device generates an MPM index (mpm idx) indicating one of the MPM candidates. When the intra-prediction mode for the current block is not even in the MPM list, the encoding device generates MPM remainder information (remaining intra-prediction mode information) indicating the same mode as the intra-prediction mode for the current block among the remaining intra-prediction modes (and planar modes) not included in the MPM list. MPM remainder information may include, for example, the intra_luma_mpm_remainder syntax element.

[0125] The decoding device obtains intra-prediction mode information from the bitstream. As described above, the intra-prediction mode information may include at least one of an MPM flag, an MPM index, and remaining MPM information (remaining intra-prediction mode information). The decoding device may configure an MPM list. The MPM list is configured to be the same as the MPM list configured by the encoding device. That is, the MPM list may include intra-prediction modes of adjacent blocks, or it may also include specific intra-prediction modes according to a predetermined method.

[0126] The decoding device can determine the intra-prediction mode for the current block based on the MPM list and intra-prediction mode information. For example, when the value of the MPM flag is 1, the decoding device can deduce the candidate indicated by the MPM index among the MPM candidates in the MPM list as the intra-prediction mode for the current block.

[0127] In another example, when the value of the MPM flag is 0, the decoding device can deduce the intra-prediction mode indicated by the remaining intra-prediction mode information (which may be called the MPM remaining information) in the remaining intra-prediction modes as the intra-prediction mode for the current block.

[0128] The encoding device derives reference samples for the current block (S410). The reference samples may include adjacent reference samples of the current block. The adjacent reference samples of the current block may include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, the adjacent reference samples of the current block may include multiple rows of adjacent samples above and multiple columns of adjacent samples to the left.

[0129] The encoding device performs intra-frame prediction on the current block to derive prediction samples (S430). The encoding device can derive prediction samples based on the intra-frame prediction mode / type and reference samples. The encoding device can derive reference samples from the reference samples of the current block according to the intra-frame prediction mode of the current block, and can derive prediction samples of the current block based on the reference samples.

[0130] Intra-prediction-based coding processes can schematically include, for example, the following.

[0131] Figure 5 An example of a video / image coding method based on intra-frame prediction is shown.

[0132] Reference Figure 5The process can be as follows: S500 can be executed by the predictor of the encoding device, S505 can be executed by the residual processor of the encoding device, and S510 or S515 can be executed by the entropy encoder of the encoding device. Specifically, prediction-related information can be derived by the predictor and encoded by the entropy encoder. Residual information can be derived by the residual processor and encoded by the entropy encoder. Residual information is information about residual samples. Residual information may include information about the quantized transform coefficients for the residual samples. As mentioned above, transform coefficients can be derived from residual samples by the transformer of the encoding device, and quantized transform coefficients can be derived from transform coefficients by the quantizer. The information about the quantized transform coefficients can be encoded in the entropy encoder through a residual encoding process.

[0133] The encoding device performs intra-prediction for the current block (S500). The encoding device derives the intra-prediction mode / type for the current block, derives reference samples for the current block, and generates prediction samples within the current block based on the intra-prediction mode / type and the reference samples. The processes for determining the intra-prediction mode / type, deriving adjacent reference samples, and generating prediction samples can be performed simultaneously, or one process can be performed before the other. The encoding device can determine the mode / type applicable to the current block from multiple intra-prediction modes / types. The encoding device can compare the RD costs for the intra-prediction mode / type and determine the optimal intra-prediction mode / type for the current block.

[0134] Encoding devices can perform a predictive sample filtering process. Predictive sample filtering can also be called post-filtering. Some or all of the predicted samples can be filtered through this process. In some cases, the predictive sample filtering process can be omitted.

[0135] The encoding device generates residual samples for the current block based on the predicted samples (S505). The encoding device can compare the predicted samples with the original samples of the current block based on the phase and derive the residual samples.

[0136] The encoding device can encode image / video information, including information about intra-frame prediction (prediction-related information) and / or information about residual samples (residual information) (S510 or S515). Prediction-related information may include information about the intra-frame prediction mode and information about the intra-frame prediction type. The encoding device can output the encoded image / video information as a bitstream. The output bitstream can be sent to the decoding device via a storage medium or network.

[0137] Residual information can include residual coding syntax elements. The coding device can derive quantized transform coefficients by transforming / quantizing the residual samples. The residual information can include information about the quantized transform coefficients.

[0138] As described above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks). To do this, the encoding device can derive (modified) residual samples by performing dequantization / inverse transform on the quantized transform coefficients. The reason for transforming / quantizing the residual samples and then performing dequantization / inverse transform on them is, as mentioned above, to derive the same residual samples as those derived in the decoding device. The encoding device can generate a reconstructed block that includes reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. A reconstructed image for the current image can be generated based on the reconstructed block. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0139] Decoding devices can perform operations corresponding to those performed by encoding devices. Intra-frame prediction-based video / image decoding processes may include, for example, the following.

[0140] Figure 6 An example of a video / image decoding method based on intra-frame prediction is shown.

[0141] Reference Figure 6 S600 can be executed by the entropy decoder of the decoding device, S610 can be executed by the predictor of the decoding device, S615 can be executed by the residual processor of the decoding device, and S620 can be executed by the adder or reconstructor of the decoding device.

[0142] Specifically, the decoding device obtains image / video information from the bitstream (S600). The image / video information may include prediction-related information and / or residual information.

[0143] The decoding device performs intra-frame prediction based on prediction-related information (S610). The decoding device can derive an intra-frame prediction mode / type for the current block based on the prediction-related information, derive a reference sample for the current block, and generate prediction samples within the current block based on the intra-frame prediction mode / type and the reference sample. In this case, the decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Some or all of the prediction samples can be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process can be omitted.

[0144] The decoding device performs residual processing based on the residual information (S615). The decoding device can derive residual samples for the current block based on the residual information. Specifically, the dequantizer of the residual processor can derive the transform coefficients by dequantizing the quantized transform coefficients derived based on the residual information, and the inverse transformer of the residual processor can derive residual samples for the current block by performing an inverse transform on the transform coefficients.

[0145] The decoding device generates a reconstructed block / image (S620). The decoding device can generate reconstructed samples for the current block based on predicted samples and / or residual samples, and can derive a reconstruction including the reconstructed samples. A reconstructed image for the current image can be generated based on the reconstructed block. As described above, in-loop filtering processes, etc., can be further applied to the reconstructed image.

[0146] Information about the intra-prediction mode and / or the intra-prediction type can be encoded / decoded using the binarylization and encoding methods described in this disclosure. For example, information about the intra-prediction mode and / or the intra-prediction type can be binarylized using fixed-length binarylization, truncated Rice binarylization, or truncated unary binarylization. For example, information about the intra-prediction mode and / or the intra-prediction type can be encoded / decoded using entropy coding (e.g., CABAC or CAVLC).

[0147] Figure 7 An example of a directional intra-frame prediction mode is shown. Figure 7 An example can be shown that includes 65 directional intra-frame prediction modes.

[0148] Reference Figure 7 Directional intra-prediction modes can include modes #2 through #65. In this case, mode #50 could represent a vertical intra-prediction mode, and mode #18 could represent a horizontal intra-prediction mode. However, this example is for illustration, and the number and type of candidate intra-prediction modes can vary. One or more non-directional intra-prediction modes can be considered; for example, mode #0 could represent an intra-plane prediction mode (plane mode), and mode #1 could represent an intra-DC prediction mode (DC mode).

[0149] Intra-frame prediction modes can be assigned mode numbers, as shown in the table below, for example.

[0150] [Table 1]

[0151] As described above, the MPM list can be configured to efficiently signal the intra-prediction mode for the current block. In the method according to embodiments of this disclosure, multiple MPM lists can be configured. The multiple MPM lists may include a first MPM list and a second MPM list. The first MPM list may be referred to as the primary MPM (PMPM) list, and the second MPM list may be referred to as the secondary MPM (SMPM) list.

[0152] The first and second MPM lists can be configured according to predetermined criteria. For example, a generic MPM list containing k candidates can be configured first, the first n candidates in the generic MPM list can be included in the first MPM list, and the remaining m candidates can be included in the second MPM list. In one example, k can be 22, n can be 6, and / or m can be 16. For example, the first candidate in the generic MPM list (the candidate of the first entry) can always be a planar pattern. In another example, where the planar pattern is signaled based on a separate planar flag (or not_planar_flag), k can be 21, n can be 5, and / or m can be 16.

[0153] At least one of the remaining candidates, excluding the planar pattern, can be derived from adjacent blocks.

[0154] Figure 8 An example of adjacent blocks used to derive the MPM list is shown.

[0155] Reference Figure 8 The adjacent block may include at least one of the following: the left adjacent block (L), the bottom left adjacent block (BL), the top adjacent block (A), the top right adjacent block (AR), and / or the bottom left adjacent block (AL) of the current block.

[0156] Furthermore, the remaining candidates may include DIMD-based patterns (DIMD-derived intra-prediction patterns). For example, intra-prediction patterns derived from neighboring blocks and DIMD-based patterns (hereinafter referred to as sorted intra-prediction patterns) can be sorted in ascending order of SAD cost based on SAD cost (hereinafter referred to as sorted intra-prediction patterns), and then added to the general MPM list or the first MPM list. In this case, only up to a predetermined number (p) of sorted intra-prediction patterns can be extracted and added to the general MPM list or the first MPM list. Here, the SAD cost can be calculated based on the reconstruction samples of the template of the current block and the predictors derived by applying the candidate patterns to the template. Among the sorted intra-prediction patterns, directional prediction patterns can be referred to as sorted directional patterns. For example, p can be 5 or 6. Alternatively, p can be 7 or 8. Planar patterns and / or DC patterns can be excluded from the DIMD-based patterns. This also applies below. DIMD-based patterns can be referred to as DIMD-derived patterns.

[0157] For example, sorted prediction patterns, patterns obtained by adding offsets to (extracted) sorted orientation patterns or subtracting offsets from said (extracted) sorted orientation patterns, and predetermined default patterns can be added until all k entries are filled.

[0158] In another example, the sorting process can be omitted for some or all of the DIMD-based patterns. This is because DIMD-based patterns are highly correlated with the current block. Therefore, the sorting process can be omitted for some or all of the DIMD-based patterns, and the sorting process can be preferentially assigned to the front of the MPM list (e.g., a general MPM list or a first MPM list). In this case, planar patterns can be placed first, and some or all of the DIMD-based patterns can be assigned after the planar patterns. Alternatively, when planar patterns are not included in the MPM list, some or all of the DIMD-based patterns can be assigned first. Alternatively, DIMD-based patterns can be compared with intra-prediction patterns derived from neighboring blocks, and overlapping patterns can be preferentially assigned to the front of the MPM list because these patterns are highly correlated with the current block. That is, the pruning process and the classification process can be merged. For example, conventional methods require redundant pruning processes and separate sorting processes to remove redundant intra-prediction patterns derived from neighboring blocks (group 1) and DIMD-based patterns (group 2), while this method can omit the sorting process by prioritizing redundant patterns while performing redundancy checks.

[0159] Figure 9 Intra-prediction modes derived from neighboring blocks and DIMD-based modes are shown.

[0160] Reference Figure 9 If patterns B and C overlap between two groups, then patterns B and C can be assigned to the MPM list with priority. In this case, for example, patterns B and C can be assigned at the beginning of the MPM list, or assigned immediately after the planar patterns.

[0161] In this case, for example, the priority between pattern B and pattern C can be determined based on the priority in the DIMD-based patterns. Since the priority of DIMD-based patterns is already derived based on HoG, the priority can be determined without a separate SAD-based classification process.

[0162] Intra-prediction modes used to encode previous blocks within a specific region can be stored and reused based on history; this can be called history-based intra-prediction modes (HIPMs). HIPMs can be represented using various terms, such as history-based MPMs (HMPMs) or history-based candidate modes. The specific region can include, for example, CTUs, CTU rows, multiple CTU rows, slices, and tiles. Multiple CTU rows can include, for example, the nth CTU row and the (n-1)th CTU row where the current block is located. In this case, the HIPM list can be reordered based on the frequency of the intra-prediction modes. That is, intra-prediction modes with a high occurrence frequency within a specific region have a higher priority than intra-prediction modes with a low occurrence frequency and can be assigned to an earlier position in the list. When assigned to an earlier position in the list, a lower index value can be used to indicate the mode.

[0163] Figure 10 The intra-prediction mode of the previous block within a specific region is shown.

[0164] Reference Figure 10 When a specific pattern (e.g., pattern #50) has the highest frequency, the HIPM list can be configured to give that pattern high priority. This list can be referred to as a buffer. In this case, for example, pattern #28 has the next highest frequency and can therefore be assigned to the position after pattern #50.

[0165] When configuring the HIPM list, the intra-prediction mode of previous blocks within a specific region may be unavailable, for example, when previous blocks are coded based on inter-frame prediction (inter-frame coding). In this case, the intra-prediction mode of the previous blocks can be treated as planar mode or DC mode. Alternatively, the intra-prediction mode of previous blocks can be omitted when configuring the HIPM list.

[0166] The HIPM list can be used to configure the MPM list. For example, the MPM list can be configured based on x candidates from the HIPM list according to priority. That is, the MPM list can include x candidates from the HIPM list according to priority. Alternatively, the candidates for the MPM list can be derived based on candidates for intra-prediction modes derived from neighboring blocks (first group) and DIMD-based modes (second group) and / or candidates from the HIPM list (third group). In this case, a pruning and sorting process can be performed, and at most the top p items can be extracted through the sorting process.

[0167] As described above, a general MPM list, a first MPM list, and / or a second MPM list can exist. The second MPM list can be divided into four groups. In this case, for example, the PMPM flag can be signaled first, and when the PMPM flag is 1, the PMPM index can be signaled. The PMPM flag can indicate whether the intra-prediction mode for the current block exists in the first MPM list. For example, the GMPM flag can be signaled before the PMPM flag. In this case, when the GMPM flag is 1, the PMPM flag and SMPM flag (described below) can be signaled. When the PMPM flag is 0, the SMPM flag can be signaled first, and when the SMPM flag is 1, the group index (SMPM group index) for the second MPM list can be parsed / signed first, and the mode index (SMPM mode index) can be signaled / parsed subsequently. Information about the intra-prediction mode (e.g., GMPM flag, PMPM flag, PMPM index, SMPM flag, SMPM group index, and / or SMPM mode index) can be signaled using CU syntax. When the GMPM flag is 0, the PMPM and SMPM flags can be deduced to be 0 without signaling / coding. The GMPM flag can also be simply referred to as the MPM flag. The GMPM flag can be omitted depending on the situation.

[0168] For example, information about intra-frame prediction can be signaled as follows. Information about intra-frame prediction can include one or more syntax elements. The same applies below.

[0169] [Table 2]

[0170] For example, smpm_group_idx can be binary-coded with a fixed length, and smpm_mode_idx can also be binary-coded with a fixed length.

[0171] In another example, smpm_group_idx can be truncated Rice (or truncated unary) binary. In this case, smpm_mode_idx can be fixed-length binary. Alternatively, the binary method for smpm_mode_idx can be determined based on the smpm_group_idx value. For example, when the smpm_group_idx value is 0 or 1, smpm_mode_idx can be truncated Rice (or truncated unary) binary, while in other cases, it can be fixed-length binary. smpm_mode_idx can be encoded based on a context model, in which case the context model for smpm_mode_idx can be determined differently based on the size of the current block and / or the smpm_group_idx value. For example, the context model can be indicated based on the context index increment (ctxInc), and the ctxInc for the bin of smpm_mode_idx can be determined differently, for example, based on the smpm_group_idx value as shown below. As described below, smpm_group_flag can be used to replace smpm_group_idx.

[0172] [Table 3]

[0173] [Table 4]

[0174] The second MPM list can be divided into two groups. In this case, the number of candidates in the first group can be equal to or different from the number of candidates in the second group. For example, the number of candidates in the second MPM list can be 16, 12, or 8. In this case, the first group can include 4 or 8 candidates, while the second group can include 8 candidates.

[0175] For example, `smpm_group_flag` can be used instead of `smpm_group_idx`. `smpm_group_flag` indicates whether the intra-prediction mode for the current block is included in the second candidate group of the second MPM list. The binary representation method for `smpm_mode_idx` can be determined based on the `smpm_group_flag` value. For example, when the `smpm_group_flag` value indicates the first group (e.g., a value of 1), `smpm_mode_idx` can be truncated Rice (or truncated unary) binary representation, and in other cases, `smpm_mode_idx` can be binary represented by a fixed length.

[0176] In this case, for example, information about intra-frame prediction can be communicated using signals as follows.

[0177] [Table 5]

[0178] You can use smpm_idc instead of smpm_flag and / or smpm_group_flag (or smpm_group_idx).

[0179] In this case, for example, information about intra-frame prediction can be communicated using signals as follows.

[0180] [Table 6]

[0181] `smpm_idc` indicates whether the intra-prediction mode for the current block is included in the second MPM list, and if so, which group the intra-prediction mode belongs to. `smpm_idc` can be truncated Rice (or truncated unary) binary.

[0182] The following table shows the binary representation of smpm_idc.

[0183] [Table 7]

[0184] smpm_mode_idx can also be called smpm_idx. smpm_idx can be truncated into Rice (or truncated unary) binary form.

[0185] The following table shows the binary representation of smpm_idx.

[0186] [Table 8]

[0187] Alternatively, smpm_idx can be binary based on smpm groups. For example, smpm_idx can be binary based on the smpm_idc value. For instance, when the smpm_idc value is 1, smpm_idx can be truncated Rice (or truncated unary) binary, and when the smpm_idc value is 2, smpm_idx can be fixed-length binary. In this case, for example, the number of candidates in the first group can be 4, and the number of candidates in the second group can be 8. Therefore, for the worst case, the maximum number of binary bins can be matched.

[0188] [Table 9]

[0189] [Table 10]

[0190] As mentioned above, considering the best case, the first group in the second MPM list can be binary-coded with variable length because it is highly relevant to the current block, and considering the worst case, the second group in the second MPM list can be binary-coded with fixed length because it is relatively less relevant to the current block.

[0191] Context-based encoding (e.g., CABAC) can be used for smpm_idx, in which case the context index (or context index increment) of bin 0 can be set differently depending on the situation. The context model of smpm_idx can be determined differently based on the size of the current block and / or the value of smpm_idc. For example, the context index (or context index increment) of bin 0 of smpm_idx can be set differently based on smpm_idc (or smpm_group_idx or smpm_group_flag).

[0192] For example, when the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is greater than 1, the context index increment of bin 0 of smpm_idx can be 1. When the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is 1, the context index increment of bin 0 of smpm_idx can be 0, or vice versa. In the table below, smpm_idc can be replaced by smpm_group_idx or smpm_group_flag. In this case, the value of smpm_group_idx or smpm_group_flag as a condition can be set to be 1 less than the value of smpm_idc.

[0193] [Table 11]

[0194] [Table 12]

[0195] [Table 13]

[0196] [Table 14]

[0197] By using the context-based coding described above, the context model can be adaptively assigned to information related to the intra-frame prediction mode, and the entropy coding efficiency can be effectively improved.

[0198] According to embodiments of this disclosure, in addition to existing planar patterns, two or more oriented planar patterns can be added. The oriented planar patterns can include a horizontal planar pattern and a vertical planar pattern.

[0199] Figure 11 Examples of predictions based on horizontal plane patterns and examples of predictions based on vertical plane patterns are shown. Figure 11 In the image, (a) shows an example of prediction based on a horizontal plane pattern, and (b) shows an example of prediction based on a vertical plane pattern.

[0200] Reference Figure 11 In planar horizontal mode, predicted samples are generated using the left reference sample of the target sample in the current block and the top-right reference sample (TR) of the current block. In this case, predicted samples can be generated by performing horizontal linear interpolation using the left reference sample and the top-right reference sample (TR). For example, horizontal linear interpolation can be performed based on the following equation.

[0201] [Equation 1]

[0202] Here, pred(x, y) represents the predicted sample value at the (x, y) coordinate, W represents the width of the current block, and TR represents the x-coordinate of the top-right reference sample. rec(-1, y) can represent the value of the reconstructed reference sample at the (-1, y) coordinate, and rec(TR, -1) can represent the value of the reconstructed reference sample (i.e., the top-right reference sample) at the (TR, -1) coordinate. For reference, the above coordinates can represent the values ​​when the top-left sample position of the current block is (0, 0).

[0203] In planar vertical mode, predicted samples are generated using the top reference sample of the target sample in the current block and the bottom left reference sample of the current block. In this case, predicted samples can be generated by performing vertical linear interpolation using the top reference sample and the bottom left reference sample (BL). For example, vertical linear interpolation can be performed based on the following equation.

[0204] [Equation 2]

[0205] Here, pred(x, y) represents the predicted sample value at coordinates (x, y), H represents the height of the current block, and BL represents the y-coordinate of the lower-left reference sample. rec(x, -1) can represent the value of the reconstructed reference sample at coordinates (x, -1), and rec(-1, BL) can represent the value of the reconstructed reference sample (i.e., the lower-left reference sample) at coordinates (-1, BL). For reference, the above coordinates can represent the values ​​when the upper-left sample position of the current block is (0, 0).

[0206] Oriented planar modes can also include planar diagonal modes.

[0207] Figure 12 An example of prediction based on a planar diagonal pattern is shown.

[0208] Reference Figure 12 In the diagonal planar mode, predicted samples can be generated through linear interpolation of two reference samples in the diagonal direction. For example, predicted samples can be generated through linear interpolation of the upper-right and lower-left diagonal reference samples of the target sample within the current block. In this case, for the target sample located in the lower-right direction within the current block (e.g., P3, P6, P9, P7, P10, P11, P12, P13, P14, or P15), the upper-right and lower-left adjacent reference samples of the current block can be used. For example, TR and BL can be used to predict target samples P3, P6, P9, or P12. In this case, predicted samples can be generated through linear interpolation of TR and BL.

[0209] The availability of directional planar mode can be determined based on high-level syntax and block size. For example, directional planar mode enabling flags (e.g., `dir_planar_enabled_flag`) can be signaled using high-level syntax. Here, high-level syntax can include PPS, SPS, image headers, and slice headers. The `constraint_flag` for directional planar mode can be signaled within general constraint information. For example, general constraint information can include `no_dir_planar_constraint_flag`, which indicates whether directional planar mode is restricted.

[0210] In addition, when the block size meets, for example, the width Oriented planar mode is allowed when height > 64, width <= 128, and height <= 128. Alternatively, when the block size meets the width requirement... Oriented planar mode is allowed when height > 128, width <= 256, and height <= 256. Alternatively, oriented planar mode is allowed only when the current block is not a square. Alternatively, horizontal and vertical planar modes are allowed when the current block is not a square, while diagonal planar mode is allowed when the current block is a square. Alternatively, horizontal planar mode is allowed when the width of the current block is greater than its height, and vertical planar mode is allowed when the height of the current block is greater than its width. Alternatively, horizontal planar mode is allowed when the width of the current block is less than its height, and vertical planar mode is allowed when the height of the current block is less than its width. Alternatively, horizontal planar mode is disallowed when the width of the current block is greater than its height by a certain threshold, and vertical planar mode is disallowed when the height of the current block is greater than its width by a certain threshold. Alternatively, when the width of the current block is less than a certain threshold or less than its height, a planar horizontal mode may be disallowed, and when the height of the current block is less than a certain threshold or less than its width, a planar vertical mode may be disallowed. Therefore, signaling for the selection information (e.g., directional_idx) described below can be omitted depending on the conditions.

[0211] When the Oriented Plane Mode Enable flag allows Oriented Plane Mode and / or when the block size condition is met, information about the Oriented Plane Mode, which will be described below, can be signaled.

[0212] When the current block is not square, directional planar patterns are allowed. For example, when the current block is square, prediction-related information may not include selection information. In this case, selection information can be deduced into specific values ​​without encoding / decoding.

[0213] For example, a horizontal planar mode might not be allowed when the width of the current block is greater than its height. In this case, the candidate modes may not include the horizontal planar mode. That is, selection information can be used to select one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar mode is applied to the current block.

[0214] For example, a planar vertical mode might not be allowed when the width of the current block is greater than its height. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0215] For example, a horizontal planar pattern might not be allowed if the width of the current block is greater than a first threshold or more than its height. In this case, the candidate patterns may not include the horizontal planar pattern. That is, selection information can be used to select one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar pattern is applied to the current block.

[0216] When the width of the current block is greater than a second threshold or more than its height, the planar vertical mode may be disallowed. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be the same as the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0217] For example, a horizontal planar mode might not be allowed when the height of the current block is greater than its width. In this case, the candidate modes may not include the horizontal planar mode. That is, selection information can be used to select one of the candidate modes other than the horizontal planar mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar mode is applied to the current block.

[0218] For example, a planar vertical mode might not be allowed when the height of the current block is greater than its width. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the planar horizontal mode is applied to the current block.

[0219] For example, a horizontal planar pattern might not be allowed if the height of the current block is greater than a first threshold or more than its width. In this case, the candidate patterns may not include the horizontal planar pattern. That is, selection information can be used to select one of the candidate patterns other than the horizontal planar pattern. Alternatively, the signaling of the selection information can be omitted, and it can be implicitly determined that the vertical planar pattern is applied to the current block.

[0220] When the height of the current block is greater than a second threshold or more than its width, the planar vertical mode may be disallowed. In this case, the candidate modes may not include the planar vertical mode. That is, selection information can be used to select one of the candidate modes other than the planar vertical mode. Alternatively, the signaling of the selection information may be omitted, and it may be implicitly determined that the planar horizontal mode is applied to the current block. The first threshold may be the same as the second threshold. Alternatively, the first threshold may be different from the second threshold.

[0221] Oriented planar modes (e.g., horizontal and vertical planar modes) can be signaled, for example, as follows.

[0222] [Table 15]

[0223] When a planar mode is applied to the current block, the horizontal and vertical planar modes can be signaled as additional modes.

[0224] The directional_flag is a flag indicating whether a directional plane mode (e.g., horizontal plane mode or vertical plane mode) is applied to the current block.

[0225] `directional_idx` indicates the selection information for the mode to be applied to the current block from among candidate modes, including planar horizontal and planar vertical modes. `directional_idx` can be referred to as, for example, `planar_hor_flag` or `planar_ver_flag`.

[0226] Flag information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMD-based pattern of the current block, and / or the flag information values ​​of the current block's neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment for the flag information can be derived based on the flag information values ​​of the left neighboring block and the above neighboring block.

[0227] Selection information can be encoded based on a context model. For example, the context model can be determined based on the size of the current block, the DIMD-based pattern of the current block, and / or the selection information values ​​of the current block's neighboring blocks. For example, the context model can be determined based on the context index increment, and the context index increment can be determined based on the size of the current block, the DIMD-based pattern of the current block, and / or the selection information values ​​of the current block's neighboring blocks.

[0228] Furthermore, the if (INTRA_PLANAR) check is a conditional statement used to determine whether the planar pattern is applied to the current block, and can be replaced by if (!not_planar_flag) if the planar pattern is signaled separately.

[0229] [Table 16]

[0230] [Table 17]

[0231] You can replace mpm_flag with pmpm_flag, and you can replace mpm_idx with pmpm_idx.

[0232] You can (for example) truncate Rice (or truncate unary) binaryization of directional_idx.

[0233] [Table 18]

[0234] For example, type 1 could correspond to a horizontal planar mode, type 2 to a vertical planar mode, and type 3 to a diagonal planar mode. However, this example is for illustrative purposes, and the indexing and / or binarization of the directional_idx for the current block can be adaptively performed based on the size of the current block, the DIMD-based mode, etc. For example, when the height of the current block is greater than its width, type 1 could correspond to a horizontal planar mode, type 2 to a vertical planar mode, and type 3 to a diagonal planar mode. When the width of the current block is greater than its height, type 1 could correspond to a vertical planar mode, type 2 to a horizontal planar mode, and type 3 to a diagonal planar mode. When the current block is a square, type 1 could correspond to a diagonal planar mode, type 2 to a horizontal planar mode, and type 3 to a vertical planar mode. Alternatively, when the current block is a square, type 1 could correspond to a diagonal planar mode, type 2 to a vertical planar mode, and type 3 to a horizontal planar mode.

[0235] Instead of the omitted `directional_idx`, the pattern to be applied among candidate patterns, including planar horizontal and planar vertical patterns, can be determined based on TIMD or DIMD. For example, whether to apply the planar vertical or planar horizontal pattern can be implicitly determined based on the HoG derived from DIMD.

[0236] When determining the availability of planar horizontal mode and planar vertical mode by comparing the width and height of the current block as described above, it is possible to determine which mode, planar horizontal mode or planar vertical mode, should be applied in the absence of directional_idx signaling.

[0237] Syntax elements such as dir_planar_idc can be used instead of directional_flag and directional_idx. dir_planar_idc can be called planar_directional_idc or directional_idc.

[0238] [Table 19]

[0239] [Table 20]

[0240] The `dir_planar_idc` parameter indicates whether the current block should be treated as a normal planar, horizontal, or vertical planar. `dir_planar_idc` can be truncated (or unary) binary.

[0241] [Table 21]

[0242] When a directional plane mode (plane horizontal mode or plane vertical mode) is applied to the current block, intra-frame prediction for the current block storage can be determined based on a predetermined method.

[0243] For example, the intra-prediction mode of the current block (which is referenced in the process of deriving the intra-prediction mode for subsequent blocks and generating the MPM list) can be the (normal) planar mode.

[0244] When selecting a transform kernel for the primary / secondary (inverse) transform of the current block, the selection can also be based on the intra-prediction mode of the current block. For example, the transform set including the transform kernel can be determined based on the intra-prediction mode of the current block. In this case, when selecting the transform kernel, the referenced intra-prediction mode can be mapped as follows: if a horizontal planar mode is applied to the current block, it is mapped to a horizontal intra-prediction mode; if a vertical planar mode is applied to the current block, it is mapped to a vertical intra-prediction mode. Furthermore, the scan order for scanning (quantizing) the transform coefficients can be determined based on the intra-prediction mode of the current block. In this case, for example, to determine the scan order, the following can be considered: if a horizontal planar mode is applied to the current block, a horizontal intra-prediction mode is applied; if a vertical planar mode is applied to the current block, a vertical intra-prediction mode is applied. For example, when it is determined that a horizontal intra-prediction mode is applied, the scan order can be horizontal, and when it is determined that a vertical intra-prediction mode is applied, the scan order can be vertical.

[0245] In this case, the problem arises that the current block stores two or more intra-prediction modes, which leads to asymmetric memory buffer allocation in the implementation between cases where a horizontal or vertical planar mode is applied and cases where a horizontal or vertical planar mode is not applied.

[0246] Therefore, a method is needed to reconcile the intra prediction modes stored for reference when deriving intra prediction modes for subsequent blocks with the intra prediction modes stored for selecting the transform kernel for the current block. Thus, a method is needed that stores the prediction modes used to derive intra prediction modes for adjacent blocks as horizontal / vertical modes, or, as usual, uses planar modes as prediction modes for deriving the primary / secondary transform kernels, but changes the transform kernels within the primary / secondary transform set corresponding to the planar modes.

[0247] For example, the intra-prediction mode of the current block (which is referenced in the process of deriving the intra-prediction mode for subsequent blocks and generating the MPM list) can be mapped to a horizontal intra-prediction mode when a planar horizontal mode is applied to the current block, and to a vertical intra-prediction mode when a planar vertical mode is applied to the current block. The following diagram illustrates the mapping when a directional planar mode is applied to the current block.

[0248] [Table 22]

[0249] In other words, in this case, the number of intra-prediction modes that need to be stored for the current block can be reduced to one.

[0250] Alternatively, separate pattern numbers # can be assigned to the horizontal and vertical plane patterns as follows. These pattern numbers # are for illustrative purposes, and other reserved numbers may be assigned.

[0251] [Table 23]

[0252] In this case, the number of intra-prediction modes that need to be stored for the current block can be fixed at 1.

[0253] When referenced for deriving intra-prediction modes for subsequent blocks or for deriving transform kernels, a separate prediction mode mapping process can be performed.

[0254] For example, the storage can include the pattern numbers for horizontal and vertical plane modes, but in practice, these modes can be mapped and utilized as follows.

[0255] [Table 24]

[0256] When a directional planar mode is applied to the current block, the stored mode is a planar mode. However, when deriving the mode referenced for deriving the intra-prediction mode of subsequent blocks and / or for selecting the transform kernel (or transform set), the directional planar selection information (directional_idx or dir_planar_idc) can be checked, and the referenced mode can be adaptively derived based on this.

[0257] For example, as shown in the table below, when applying a directional plane mode (plane horizontal mode or plane vertical mode), the stored mode can be a plane mode.

[0258] [Table 25]

[0259] In this case, for example in the transformation kernel derivation operation, the value of the orientation plane selection information can be referenced, as shown in the following figure.

[0260] Figure 13 The derivation of the transformation kernel for oriented plane modes is shown.

[0261] Reference Figure 13 When `predModeIntra`, which is the intra-prediction mode for the current block, is equal to the intra-planar mode and `directional_flag` is 1, `predModeIntra` can be modified. In this case, for example, when `directional_idx` is 0 or a planar horizontal mode is applied, `predModeIntra` can be mapped to the intra-planar mode. Similarly, when `directional_idx` is 1 or a planar vertical mode is applied, `predModeIntra` can be mapped to the intra-planar mode. And when `directional_idx` is 2 or a planar diagonal mode is applied, `predModeIntra` can be mapped to the intra-planar mode.

[0262] When deriving the transform kernel for the current block, the selection of the transform set and / or the derivation of the transform kernel can be performed using predModeIntra after mapping.

[0263] The aforementioned method can solve the problem of asymmetric memory buffer allocation for directional plane modes, and can coordinate the intra-prediction modes stored for reference when deriving intra-prediction modes for subsequent blocks and the intra-prediction modes stored for selecting transform kernels for the current block based on a single stored intra-prediction mode.

[0264] According to embodiments of this disclosure, TIMD technology can be applied. TIMD can refer to template-based intra-mode derivation or a fusion of template-based intra-mode derivation. TIMD technology can be referred to as a TIMD type. In TIMD, a template including neighboring samples of the current block can be derived, a predictor of the template based on a candidate prediction mode (i.e., a predicted sample of the template) can be derived based on a reference sample of the template, and the optimal intra-mode prediction can be derived by comparing the derived predictor with the reconstructed template (i.e., a reconstructed sample of the template).

[0265] Figure 14 This shows an example of TIMD application when the current block is a square block.

[0266] Reference Figure 14 The template of the current block can have a size L. That is, the template of the current block can include a left template and a top template, the width of the left template can be L, and the height of the top template can be L. A reference sample for the template is derived, and the reference sample for the template includes a left reference sample and a top reference sample. The height of the left reference region containing the left reference sample can be based on the height of the left template, and the width of the top reference region containing the top reference sample can be based on the width of the top template. The width of the left reference region can be equal to the height of the top reference region. The width of the left reference region can be 1, and the height of the top reference region can be 1. When the current block is a square block, the height of the left reference region can be equal to the width of the top reference region. Alternatively, when the current block is a square block, the height of the left reference region can be equal to the width of the top reference region minus 1. Alternatively, when the current block is a square block, the height of the left reference region can be equal to the width of the top reference region plus 1.

[0267] Figure 15 This shows an example of TIMD application when the current block is a non-square block.

[0268] Reference Figure 15The template of the current block includes a left template and a top template, and the sizes of the left template and the top template can be different. For example, the width of the left template can be L1, and the height of the top template can be L2. L2 can be different from L1. The height L2 of the top template and the width L1 of the left template can vary depending on the size of the current block, etc. For example, when the current block is a non-square block with a width greater than its height, L2 can be greater than L1. For example, when the current block is a non-square block with a height greater than its width, L1 can be greater than L2. A reference sample for the template is derived, and the reference sample for the template can be located in the left, upper left, and top regions of the template. The reference sample for the template includes a left reference sample and a top reference sample. The height of the left reference region where the left reference sample is located can be based on the height of the left template, and the width of the top reference region where the top reference sample is located can be based on the width of the top template. The width of the left reference region can be equal to the height of the top reference region. The width of the left reference region can be 1, and the height of the top reference region can be 1. When the current block is a non-square block, the height of the left reference region can be different from the width of the top reference region. The height of the left reference region can be based on the height of the current block (H) and / or the height of the top template (L2). The width of the top reference region can be based on the width of the current block (W) and / or the width of the left template (L1). For example, the height of the left reference region can be equal to 2(H+L2) or 2(H+L2)+1. For example, the width of the top reference region can be equal to 2(W+L1) or 2(W+L1)+1.

[0269] TIMD candidate intra-prediction modes can be determined based on various criteria. For example, TIMD candidate intra-prediction modes can be limited to MPM. Alternatively, TIMD candidate intra-prediction modes can be limited to candidate modes in the MPM (or PMPM) list. In this case, for example, the TIMD flag can be signaled after the MPM flag (or PMPM flag) is signaled.

[0270] In another example, predetermined candidate intra-prediction modes can be defined. These predetermined candidate intra-prediction modes can include some or all of the aforementioned DIMD-derived modes (DIMD-based modes). The predetermined candidate intra-prediction modes can include a default mode. The default mode can include at least one of, for example, a vertical mode, a horizontal mode, a bottom-left diagonal mode, a top-left diagonal mode, or a top-right diagonal mode. In this case, the TIMD operation can be performed before configuring the MPM list. For example, the TIMD flag can be signaled before the MPM flag (or PMPM flag). That is, the TIMD flag can be signaled after the TIMD flag and before the MPM flag (or PMPM flag).

[0271] Based on TIMD technology, n intra-prediction modes with minimum SAD cost or minimum SATD cost can be derived from candidate intra-prediction modes. Here, n can be 1, 2, or larger. For example, when n is 2, two TIM modes can be derived by weight fusion after applying PDPC. The weights can be calculated based on the cost for the n modes.

[0272] The weights for TIMD can be calculated, for example, by the following equation.

[0273] [Equation 3]

[0274] weight1=costMode2 / (costMode1+costMode2)

[0275] [Equation 4]

[0276] weight2 = 1 - weight1

[0277] When deriving weights, division operations can be replaced by using LUTs.

[0278] Under certain conditions, only one intra-prediction mode can be derived as a TIMD-based mode. For example, when the cost of an intra-prediction mode with the minimum SAD cost or minimum SATD cost calculated based on TIMD is less than a threshold, only one intra-prediction mode can be derived. In another example, when the difference between the first SATD cost of a first intra-prediction mode with the minimum SATD cost calculated based on TIMD and the second SATD cost of a second intra-prediction mode with the second minimum SATD cost is greater than a threshold, only one intra-prediction mode can be derived.

[0279] Although examples of cost calculation based on SAD or SATD have been shown above (for illustrative purposes only), the sum of absolute differences (MR SAD) can also be used. In this case, the area of ​​the second template region can be larger than the area of ​​the first template region when calculating cost based on MR SAD. For example, the area of ​​the second template region can be four times the area of ​​the first template region. For example, the value of L2 in the second template region can be twice the value of L2 in the first template region, and the value of L1 in the second template region can be twice the value of L1 in the first template region.

[0280] For a block to which TIMD is applied, a specific intra-frame mode can be stored as an intra-frame prediction mode for the current block. The stored mode can be referenced as a mode for adjacent blocks to be used as a reference when deriving intra-frame prediction modes for subsequent blocks and / or as a mode for selecting transform kernels (or transform sets).

[0281] For example, for a block that has TIMD applied, the first TIMD derivation pattern can be stored as an intra-prediction pattern for the current block.

[0282] In another example, for a block to which TIMD is applied, the first mode in the TIMD derivation mode can be stored as the intra-prediction mode for the current block.

[0283] In another example, a candidate list can be configured using m intra-prediction modes derived from TIMD, and any mode in the candidate list can be stored. Here, m can be equal to n, or m can be 2 or 3. In this case, the first predictor (first prediction sample) of the reconstructed block or the current block generated by TIMD can be compared with the predictors according to the corresponding m intra-prediction modes, and the intra-prediction mode with the smallest SAD / SATD among the m intra-prediction modes can be stored as the intra-prediction mode for the current block.

[0284] In another example, for a block to which TIMD is applied, the first intra-prediction mode derived based on the TIMD technique for the prediction samples used for the current block (i.e., the intra-prediction mode with the highest HoG based on the re-derived Sobel filter-based HoG for the prediction samples (the first intra-prediction mode)) can be stored as the intra-prediction mode for the current block. In this case, the first intra-prediction mode derived based on TIMD for the current block and the first intra-prediction mode derived based on the predictors of the current block can be different.

[0285] TIMD-related signaling may include, for example, the following.

[0286] The following shows an example of signaling that includes prediction-related information for TIMD-related signaling.

[0287] [Table 26]

[0288] According to the above signaling method, it is necessary to additionally notify information indicating whether the regular MPM process should be applied (e.g., regular MPM flag) with a signal, which results in bit loss.

[0289] [Table 27]

[0290] As described above, the presence of a signal indicating whether TIMD technology is applied to the current block's timd_flag can be used before the MPM mode information (e.g., the MPM flag or pmpm_flag), thereby reducing additional information. In this case, as mentioned above, instead of MPM, predetermined candidate intra-prediction modes can be defined and used during the TIMD process.

[0291] TIMD can be used without ISP or MRL. For example, TIMD can be applied without ISP, and TIMD can be applied without MRL. For example, the TIMD flag can be signaled before the ISP flag. In this case, when the TIMD flag is 1, the ISP flag can be implicitly deduced to be 0 (no signaling). For example, the TIMD flag can be signaled before the MRL index. In this case, when the TIMD flag is 1, the MRL index can be implicitly deduced to be 0 (no signaling). In another example, the ISP flag can be signaled before the TIMD flag. In this case, when the ISP flag is 1, the TIMD flag can be implicitly deduced to be 0 (no signaling). In another example, the MRL index can be signaled before the TIMD flag. In this case, when the MRL index is not 0, the TIMD flag can be implicitly deduced to be 0 (no signaling).

[0292] To reduce the computational complexity of TIMD and improve signaling efficiency, a TIMD merging pattern can be used. In this case, the TIMD derivation pattern for the current block can be derived based on previous TIMD encoded blocks. For this purpose, adjacent and non-adjacent spatially adjacent blocks can be scanned. Spatially adjacent blocks can be located in at least one of the following directions: left, top, bottom left, top right, and top left of the current block. Furthermore, temporally adjacent blocks can be scanned for the TIMD merging pattern. Temporally adjacent blocks can include at least one of the following blocks located at the same location in the reference image (column image): bottom right block, center bottom right block, right block, and bottom block. When encoding a scanned block with TIMD or a TIMD merging pattern, TIMD merging candidates, including TIMD information about the scanned block (e.g., prediction pattern, merging flag, and merging weight), can be added to the TIMD merging (candidate) list. The TIMD merging list can include n candidates, where n can be signaled via HLS (such as SPS and PPS) or determined to be a specific value (e.g., 5).

[0293] Whether to apply the TIMD merge mode can be signaled based on a flag (TIMD merge flag) or implicitly determined based on predetermined conditions. Predetermined conditions may include whether the current block size is within a predetermined range. Predetermined conditions may include whether the current block is a square block.

[0294] When the TIMD flag indicates true (1), the TIMD merge flag can be signaled after the TIMD flag is signaled.

[0295] The TIMD merge list can be configured as shown in the table below.

[0296] [Table 28]

[0297] As shown above, the TIMD merging list can include TIMD merging candidates. TIMD merging candidates can include n intra-prediction modes (IPMs) and their weights.

[0298] Candidates in the TIMD merge list can be sorted based on their SAD cost. In this case, a predefined template can be used. For example, the template size is 1, and the template can include at least one of the first left-adjacent sample row and the first top-adjacent sample row of the current block. The first IPM of a TIMD candidate can be used to calculate its SAD cost. Therefore, candidates can be sorted so that a lower index value can be assigned to a candidate with a lower SAD. For example, when two candidates have the same SAD, the scan order can be used. That is, when two candidates have the same SAD, the lower index value can be assigned to the candidate that was added to the list first due to its higher scan order. In another example, when two candidates have the same SAD (e.g., a first SAD), a second SAD can be further calculated using the second IPM of the TIMD candidate. In this case, the second SADs of the two candidates can be compared, and the higher priority can be assigned to the candidate with the lower SAD value. Assigning a higher priority to a particular candidate can mean assigning a lower index value. When the second SADs of two candidates are also the same, a third SAD can be additionally calculated and compared using the third IPMs of the two candidates. Alternatively, when two candidates also have the same second SAD, the higher priority can be assigned to the candidate with the higher scan order. In another example, when two candidates have the same SAD (e.g., the first SAD) and the highest priority, a specific candidate can be indicated by signaling the candidate selection index (or flag) indicating index 0 or index 1 from the encoder to the decoder. Therefore, the best candidate can be selected while minimizing the transmission of side information. The candidate selection index (or flag) can be signaled when the TIMD merging mode is applied and a specific condition is met. This specific condition could be the SAD of the two candidates with the highest priority (i.e., when there are two or more candidates with the lowest SAD).

[0299] TIMD merging can be applied to luma blocks. That is, TIMD merging can be applied when the current block is a luma component block. The application of TIMD merging can be determined based on block size. For example, TIMD merging may not be applied when the current block size is equal to or less than a specific threshold (e.g., a 4×4 block size). Similarly, TIMD merging may not be applied when the current block size is equal to or greater than a specific threshold (e.g., a 36×36 or 64×64 block size). For example, whether to apply TIMD merging can be determined based on whether the current block is non-square. Specifically, for example, TIMD merging may not be applied when the current block is non-square. Alternatively, for example, TIMD merging may not be applied when the difference between the width and height of the current block is greater than a specific value (e.g., 4×32, 32×4, 8×64, 64×8, 4×64, or 64×4).

[0300] Based on the aforementioned TIMD technology, intra-prediction of the current block can be performed based on a template, thereby reducing side information for intra-prediction mode signaling and improving the intra-prediction performance of the current block. Furthermore, intra-prediction modes can be derived based on templates, and these modes can be used to generate the (final) prediction block for the current block through the fusion or weighted summation of prediction blocks, thus improving intra-prediction efficiency. In addition, the intra-prediction modes stored for the current block that has already undergone template-based intra-prediction and / or the intra-prediction modes used to select the transform kernel for the current block can be determined efficiently.

[0301] According to embodiments of this disclosure, intra-frame template matching (TMP) techniques can be applied. Intra-frame template matching can be referred to as intra-frame TMP. Intra-frame TMP can be considered a prediction technique or prediction type.

[0302] Figure 16 An example of intra-frame TMP technology is shown.

[0303] Reference Figure 16 A block can be predicted by searching within a predefined search range of the reconstructed region in the current image for the template most similar to the current template (neighboring reference sample (e.g., L-shaped) template). In this case, a block vector (BV) indicating the location of the matching block (reference block) within the current image can be derived / stored based on the location of the current block.

[0304] The predefined search range can be within the left CTU, the current CTU, the top left CTU, the top CTU, and the top right CTU.

[0305] Intra-frame TMPs are performed in the same manner in both the encoding and decoding devices, and the encoding device can signal whether an intra-frame TMP is used. For example, the encoding device can indicate whether an intra-frame TMP is applied to the current block by signaling an intra-frame TMP flag.

[0306] To perform intra-TMP efficiently, a candidate list for intra-TMP can be configured. The candidate list (intra-TMP candidate list) can include up to n (e.g., 19) template matching block vectors, which can be sorted in ascending order based on the SAD cost for the template.

[0307] Intra-frame TMPs can be available when the height and width of the CU size are less than or equal to 64. The maximum available size can be predefined or signaled from a higher layer.

[0308] Whether to apply intra-frame TMP can be signaled via CU-level flags. For example, when DIMD mode is not applied to the current block, a signal can be used to indicate whether intra-frame TMP should be applied. That is, when dim_flag is 0, intramp_flag can be signaled.

[0309] The block vector (BV) derived through intra-frame TMP can be stored and used for the IBC candidate list for subsequent blocks.

[0310] According to embodiments of this disclosure, Spatial Geometric Partitioning Pattern (SGPM) can also be applied. SGPM can be considered a prediction technique or prediction type.

[0311] According to SGPM, the current block can be divided into two or more partitions (geometric partitions), and different intra-prediction patterns can be applied to the two partitions. For example, a first intra-prediction pattern can be applied to the first partition, a second intra-prediction pattern can be applied to the second partition, and the two predictors derived therefrom can be fused or mixed to generate a single prediction block. The two predictors can include the first prediction partition and the second prediction partition.

[0312] For SGPM, n partition patterns can be used. For example, 26 partition patterns can be used as follows.

[0313] Figure 17 An example of SGPM's partitioning mode is shown.

[0314] Reference Figure 17 The number of partition patterns used in inter-frame prediction geometric partitioning (GPM) can differ from the number used in intra-frame prediction SGPM. For example, as... Figure 13As shown in the boxed area, some of the partition modes in GPM can be selectively used as partition modes in SGPM.

[0315] For SGPM, n partition modes and m intra-frame prediction modes can be combined to configure an SGPM candidate list of length k.

[0316] Figure 18 An example of a candidate SGPM signaling is shown.

[0317] Reference Figure 18 The application of SGPM can be signaled via a flag. The SGPM flag can be called `sgpm_flag` or `cu_sgpm_flag`. The SGPM flag can be sent at the CU level or the CU syntax level.

[0318] For example, when SGPM is applied (when the value of the SGPM flag is 1), the partition mode index, the first intra-prediction mode index (e.g., intra_pred_mode0_idx), and the second intra-prediction mode index (e.g., intra_pred_mode1_idx) can be signaled. Here, at least one of the partition mode index, the first intra-prediction mode index (e.g., intra_pred_mode0_idx), and the second intra-prediction mode index (e.g., intra_pred_mode1_idx) can be omitted. For example, for a specific partition mode index, the first intra-prediction mode index and / or the second intra-prediction mode index can be omitted. In this case, when applying a specific partition mode index, the first intra-prediction mode and / or the second intra-prediction mode can be predefined. For example, the second intra-prediction mode index can be omitted when predetermined conditions are met. The predetermined conditions may include situations where specific information is explicitly signaled, the size of the current block falls within a specific range, or the first intra-frame prediction mode index indicates a non-directional mode (planar mode or DC mode), horizontal mode, vertical mode, or diagonal intra-frame prediction mode. When the second intra-frame prediction mode index is omitted, it may be determined based on the first intra-frame prediction mode index. In this case, the second intra-frame prediction mode may be perpendicular to the first intra-frame prediction mode. Alternatively, the second intra-frame prediction mode may be predetermined as a diagonal intra-frame prediction mode. The diagonal intra-frame prediction mode may include at least one of a lower-left diagonal intra-frame prediction mode, a upper-left diagonal intra-frame prediction mode, or a upper-right diagonal intra-frame prediction mode.

[0319] In another example, when SGPM is applied (when the value of the SGPM flag is 1), the SGPM candidate index can be signaled. In this case, the SGPM candidate index can indicate a combination of partition mode (index), first intra-frame prediction mode (index), and second intra-frame prediction mode (index).

[0320] The SGPM candidate list can be predefined and reordered based on cost.

[0321] A list of SGPM candidates can be derived based on neighboring blocks of the current block. For example, when encoding neighboring blocks of the current block based on SGPM, the SGPM candidates for neighboring blocks can be derived as the first SGPM candidate for the current block. This SGPM candidate can be referred to as a candidate based on neighboring SGPM or a candidate based on intra-block neighboring blocks. The first SGPM candidate can be assigned to the SGPM candidate list for the current block with priority over other candidates. Alternatively, for example, when encoding neighboring blocks of the current block based on GPM, a second SGPM candidate can be derived based on the partitioning pattern of the GPM candidates for neighboring blocks. This SGPM candidate can be referred to as a candidate based on neighboring GPM or a candidate based on inter-block neighboring blocks. A second SGPM candidate can be derived based on a partitioning pattern derived from a combination of a predefined first intra-prediction pattern and a second intra-prediction pattern. The second SGPM candidate can be assigned to the SGPM candidate list for the current block with priority over other candidates. In this case, the first SGPM candidate can have a higher priority than the second SGPM candidate.

[0322] For example, the length k of the SGPM candidate list can be 16. For example, n can be 26, and m can be 3. k can be a predefined value, or it can be set differently based on the size of the current block.

[0323] For example, the value of k can be set differently based on the block size, as follows.

[0324] [Table 29]

[0325] The maximum value of the SGPM candidate index can also be set variably based on the size of the current block. For example, the maximum value of the SGPM candidate index can be set to be equal to k-1. For example, when condition 1 is met, the SGPM candidate index can be truncated to Rice or unary binary, and the maximum length of the empty string of the SGPM candidate index can be equal to k-1.

[0326] As described above, the SGPM candidate list can be reordered based on SAD (cost), in which case a template including neighboring samples can be utilized. Here, SAD can represent the SAD (cost) between the predicted sample and the reconstructed sample of the template. For example, the template size is fixed at 1. Alternatively, the template size can be 2 to 4.

[0327] Figure 19 An example of an SGPM template is shown.

[0328] Reference Figure 19 The template can include an upper template and a left template. The upper template can be located above the current block, and the left template can be located to the left of the current block. The SAD cost can be derived by performing predictions on the templates based on each SGPM candidate, and the SGPM candidate list can be reordered in order of candidates with lower costs. For example, as described above, higher priority can be assigned to the first and second SGPM candidates without reordering.

[0329] You can selectively use either a top template or a left template. For example, you can use signals to indicate the template type information: top template, left template, or left / top template. Alternatively, you can determine the template type based on block size or shape.

[0330] To derive the m intra-prediction modes for SGPM, the same method used for intra-mode derivation to configure the intra-inter-SGPM list can be used. In this case, the horizontal / vertical derivation mode can replace the TIM derivation mode. Alternatively, the m intra-prediction modes for SGPM may include intra-prediction modes from the MPM (PMPM) list. Alternatively, the m intra-prediction modes for SGPM may include at least one of vertical, horizontal, planar, DC, or directional planar modes. Directional planar modes may include planar horizontal and / or planar horizontal modes. Directional planar modes may not be available when SGPM is applied to the current block.

[0331] The SGPM candidate list may include regression-based SGPM candidates. For example, n (e.g., 4) regression-based SGPM candidates may be included in the SGPM candidate list. Regression-based SGPM candidates may be assigned to earlier positions (with lower index values) within the SGPM candidate list, thus taking precedence over general SGPM candidates. For example, regression-based SGPM candidates may be assigned to earlier positions (with lower index values) within the SGPM candidate list, thus employing a higher priority than the first and / or second SGPM candidates. Regression-based SGPM candidates may include two intra-prediction modes from the PMPM list and six block vectors derived from neighboring blocks.

[0332] When selecting a regression-based SGPM candidate, the final prediction block can be generated by mixing the prediction blocks for each partition based on weights. The weights for the regression-based SGPM can be derived based on the integer mixing matrix. The weights (e.g., the integer mixing matrix) can also be derived based on the single-sample row templates above and to the left of the current block.

[0333] When a regression-based SGPM candidate is selected for the current block, the intra-prediction mode derived from DIMD for the current block can be used as an intra-mode for selecting the transform kernel.

[0334] SGPM can be applied to blocks of limited size. For example, SGPM can be applied to blocks that meet the following conditions.

[0335] 4 <= width <= 64, 4 <= height <= 64, width < height 8. Height < Width 8, and / or width Height >= 32

[0336] For SGPM, adaptive blending can be applied. Whether adaptive blending is applied can be signaled via a PPS level flag (e.g., an adaptive blending flag). Adaptive blending can be applied when the PPS level flag is a predetermined value (e.g., 1). When adaptive blending is applied, the blending depth (blending parameters) can be derived, for example, as follows.

[0337] [Table 30]

[0338] When the PPS level flag is 0, a fixed blend with a blending parameter (1 / 4)τ can be applied to all sizes.

[0339] The adaptive blending enable flag can be signaled via high-level syntax (e.g., SPS syntax), and the adaptive blending application flag can be signaled via at least one of PPS, PH, or SH. For example, for images with specific image characteristics (such as screen content encoding (SCC)), in terms of signaling, it may be necessary to use a sequence-level enable flag instead of repeatedly sending image-level flags to perform on / off control.

[0340] Whether SGPM is applied can be signaled based on SGPM flags. For example, an SGPM enable flag (sgpm enable flag) indicating whether SGPM is available can be signaled via high-level syntax (e.g., SPS syntax), and when the SGPM enable flag (sgpm enable flag) is 1, the SGPM flag can be signaled. In another example, the SGPM enable flag indicating whether SPGM is available can be signaled via high-level syntax (e.g., SPS syntax), and when the SGPM enable flag is 1, an SGPM application flag indicating whether SPGM is applied can be signaled via high-level syntax (e.g., PPS syntax), and when the SGPM application flag (sgpm application flag) is 1, the SGPM flag can be signaled. In this case, the SGPM flag can be signaled via CU syntax.

[0341] When the SGPM flag is 1, the index information (e.g., the SGPM index) of one of the k SGPM candidates can be indicated by a signal. The SGPM index can be referred to as the intra-frame SGPM index. In this case, the index information can be encoded based on fixed-length (FL) binary or truncated binary (TB) binary. The following shows an example of indicating the SGPM index using a signal.

[0342] [Table 31]

[0343] When the SGPM enable flag or SGPM application flag is 1, the SGPM flag (sgpm_flag) can be signaled. When the value of the SGPM flag is 1, the SGPM index (sgpm_index or sgpm_idx) can be signaled.

[0344] SGPM indexes can be encoded based on context. Alternatively, SGPM indexes can be encoded based on bypass mode.

[0345] For example, the context model can be indicated based on the context index increment (ctxInc), and the ctxInc (sgpm_index or sgpm_idx) of the bin of the SGPM index can be determined as follows:

[0346] [Table 32]

[0347] An SGPM index can have four bins, and each bin can be encoded / decoded based on bypass encoding; this is for illustrative purposes only. Alternatively, the first bin of the SGPM index can be encoded based on context, and at least one of the remaining bins of the SGPM index can be encoded based on a bypass mode.

[0348] The k candidates can be divided into two groups, and the SGPM transpose flag and / or SGPM mode information can be signaled.

[0349] For example, when the SGPM transpose flag is 1, i) the partition angle and / or ii) the prediction modes A / B can be inverted. That is, the candidates indicated by the SGPM index information can differ depending on the value of the SGPM transpose flag. In this case, the first candidate indicating SGPM mode information when the SGPM transpose flag is 0 and the second candidate when the SGPM transpose flag is 1 can be different from each other. For the first and second candidates, i) the partition angle and / or ii) the intra-frame prediction modes A / B can be inverted. When the partition angle is inverted, it can be flipped based on the horizontal / vertical direction or based on the diagonal direction. The inversion of prediction modes can indicate that the prediction mode A of the first partition and the prediction mode B of the second partition are inverted relative to each other.

[0350] Figure 20 The differences between candidates based on the SGPM transpose flag are shown.

[0351] Reference Figure 20 i) The partition angle and / or ii) the intra-prediction modes A / B indicated by the SGPM mode (or SGPM index) can be changed according to the value of the SGPM transpose flag. For example, the partition angle can remain the same, but the intra-prediction modes assigned to the two partitions can be interchanged. In other words, when the value of the SGPM transpose flag is 0, the first intra-prediction mode can be assigned to the first partition, and the second intra-prediction mode can be assigned to the second partition; while when the value of the SGPM transpose flag is 1, the first intra-prediction mode can be assigned to the second partition, and the second intra-prediction mode can be assigned to the first partition. The first partition can be the partition assigned partition index 0, and the second partition can be the partition assigned partition index 1. For example, the first partition can be the partition covering the lower left or upper left sample of the current block. In another example, the intra-prediction modes assigned to the two partitions can remain unchanged, while the partition angle can be changed. In yet another example, not only the partition angle but also the intra-prediction modes assigned to the two partitions can be changed.

[0352] Specifically, for example, when the partition mode of the first candidate indication indicated by the SGPM pattern (SGPM index) information is #0 when the SGPM transpose flag is 0, the prediction mode A of the first partition is the first prediction mode (e.g., horizontal prediction mode), and the prediction mode B of the second partition is the second prediction mode (e.g., vertical prediction mode). When the SGPM transpose flag is 1, the partition mode of the second candidate indication indicated by the SGPM pattern (SGPM index) information can be #36, and / or the prediction mode A of the first partition can be the second prediction mode, and the prediction mode B of the second partition can be the first prediction mode.

[0353] The following shows the signaling information related to SGPM.

[0354] [Table 33]

[0355] The SGPM transpose flag (sgpm_transposed_flag) indicates whether the SGPM (candidate) has been transposed. The SGPM mode information (sgpm_mode) indicates the SGPM. SGPM mode information can also be referred to as SGPM index information. The SGPM transpose flag can be called the transpose flag or the intra-frame SGPM transpose flag, and the SGPM can be referred to as the intra-frame SGPM.

[0356] The SGPM transpose flag can be encoded based on FL binaryization, and the SGPM mode information can be encoded based on FL binaryization or truncated binary (TB) binaryization.

[0357] The SGPM flag (sgpm_flag) can be encoded based on context information (or context model). In this case, the context information can be set differently based on the block size. For example, the context model can be indicated based on the context information, and the context information can be based on context index increments. For example, different context index increments can be assigned based on a comparison between the block's width and height, and the context information (or context model) can be set differently based on this.

[0358] For example, the ctxInc of the bin of the SGPM flag can be determined as follows.

[0359] [Table 34]

[0360] Additionally, for example, both the SGPM transpose flag and the SGPM mode information can be encoded based on the bypass mode. In this case, the ctxInc of each bin of the SGPM transpose flag and SGPM mode information can be determined, for example, as follows.

[0361] [Table 35]

[0362] In another example, the SGPM transpose prefix flag can be encoded based on context information, and the SGPM mode information can be encoded based on the bypass mode. In this case, the ctxInc for each bin of the SGPM transpose flag and SGPM mode information can be determined, for example, as follows.

[0363] [Table 36]

[0364] According to the above method, the SGPM transpose flag can be encoded based on context, thereby achieving coding gain compared to the case of bypass coding based on the regular SGPM index.

[0365] In yet another example, the SGPM transpose flag and SGPM mode information can be encoded based on contextual information.

[0366] As described above, the SGPM enable flag can be signaled at the SPS level, and when the SGPM enable flag is 1, the SGPM application flag and / or the SGPM flag can be signaled. When the SGPM enable flag is 0, the signaling of the SGPM application flag and / or the SGPM flag can be omitted; in this case, the values ​​of SGPM, the SGPM application flag, and / or the SGPM flag can be inferred as 0. For example, instead of signaling flags at the picture level (PPS level) or block level for each instance to indicate whether SGPM is applied, SGPM on / off can be performed once using available flags at the sequence level (SPS level), depending on the image type and characteristics.

[0367] When SGPM is applied to the current block, it is necessary to determine which mode is stored as the intra-prediction mode in order to derive the transform kernel for the current block and the intra-prediction mode for adjacent blocks.

[0368] For example, the first mode derived by applying DIMD to samples predicted by SGPM can be stored as an intra-prediction mode for deriving the transform kernel. In this case, while high accuracy is achieved, complexity and latency issues may arise; therefore, a simplified approach needs to be considered.

[0369] In another example, between two intra-prediction modes derived via SGPM, the intra-prediction mode of the second partition located on the right or bottom can be stored as the intra-prediction mode for the current block. This is because the intra-prediction mode of the second partition between the first and second partitions of the current block can actually be highly correlated with the next block.

[0370] In another example, depending on the SGPM partitioning pattern, the intra-prediction pattern of the first or second partition can be variably stored as the intra-prediction pattern for the current block. In this case, the intra-prediction pattern of the partition with the larger area can be used. Depending on the SGPM partitioning pattern, the proportion of the current block occupied by the first and second partitions changes, and the intra-prediction pattern of the partition occupying the higher proportion of the current block can be stored as the intra-prediction pattern for the current block.

[0371] In another example, when the intra prediction mode of the first partition is defined as the first intra prediction mode and the intra prediction mode of the second partition is defined as the second intra prediction mode, block cell prediction for the current block can be performed based on the first and second intra prediction modes used for SGPM. Then, the block predicted according to SGPM can be compared with the block predicted according to the first intra prediction mode, and the block predicted according to SGPM can be compared with the block predicted according to the second intra prediction mode, thereby storing the intra prediction mode with the lowest SAD (cost) or SATD (cost) as the intra prediction mode for the current block. That is, the SAD for the current block (for prediction of the current block) can be calculated relative to the two intra prediction modes derived by SGPM, and the intra prediction mode with the lower SAD can be considered the intra prediction mode for the corresponding block.

[0372] In yet another example, the first derivation mode according to the existing DIMD technique described above can be stored as an intra-frame mode for the current block. The first derivation mode according to the existing DIMD technique can refer to a first derivation mode derived based on the HoG of the template for the current block.

[0373] When SGPM is applied, at least one partition in a partition can be predicted based on block vectors (e.g., intraTMP).

[0374] Figure 21 This shows an example of intraTMP being applied to a partition of the current block where SGPM is applied.

[0375] Reference Figure 21 The first intra-frame prediction mode can be applied to the first partition of the current block to which SGPM is applied, and the intraTMP can be applied to the second partition. In this case, the first predicted partition can be derived by predicting the first partition based on the first intra-frame prediction, and the second predicted partition can be derived by predicting the second partition based on the block vector using the intraTMP. The predicted block for the current block can be derived based on the first and second predicted partitions.

[0376] For example, intraTMP can be applied to one or both of the first and second partitions. However, for example, to reduce complexity, intraTMP can be applied only to the second partition, since the first partition has high correlation with its left spatial neighbor and its upper spatial neighbor.

[0377] Even when the current slice is an intra-slice, intraTMP can be used in SGPM. Since existing inter-frame GPM is not applied to intra-slices, SGPM applied to intra-slices requires performing prediction based on block vectors by applying intraTMP to the partition.

[0378] In other words, signals can be used to indicate whether intraTMP is applied in SGPM and / or specific information about the applied partition. This specific information can be referred to as the SGPM intraTMP application flag or sgpm_intraTMP_flag. Specific information can be signaled even when the current slice is an intra-frame slice.

[0379] An enable flag indicating whether intraTMP is available in SGPM can be signaled using high-level syntax (e.g., SPS or PPS). When the enable flag value is 0, the signaling for the SGPM intraTMP application flag or sgpm_intraTMP_flag can be omitted.

[0380] Alternatively, an enable flag indicating whether a block vector is available in at least one partition of an SGPM block or a partition of an SGPM block can be signaled using high-level syntax (e.g., SPS or PPS). When the value of the enable flag is 0, the signaling of the SGPM intraTMP application flag or sgpm_intraTMP_flag can be omitted.

[0381] Based on the aforementioned SGPM technique, intra-prediction can be performed based on the geometric partitions of the current block, thereby improving the intra-prediction performance for the current block. Furthermore, by efficiently signaling the intra-prediction mode and partition shape for each partition and controlling SGPM on / off via a high-level syntax, additional information can be reduced. Additionally, prediction partitions can be generated using different intra-prediction modes or block vectors on a partition basis, and the (final) prediction block for the current block can be generated by fusing or mixing prediction partitions, further improving intra-prediction performance. Moreover, the intra-prediction mode stored for the current block where intra-prediction has been performed based on partitions and / or the intra-prediction mode used to select the transform kernel for the current block can be determined efficiently.

[0382] As mentioned above, various intra-prediction modes can be used for intra-prediction of the current block. For example, intra-prediction modes can be derived based on decoder-side intra-mode derivation (DIMD) techniques. DIMD can be referred to as the DIMD (prediction) type.

[0383] Figure 22 An example of template-based HoG computation in DIMD is shown.

[0384] Reference Figure 22 In DIMD, a histogram of gradients (HoG) can be computed using a template that includes neighboring samples of the current block. For example, the HoG can be computed based on a template with n (e.g., 3) neighboring sample rows using horizontal / vertical Sobel filters. In this case, both horizontal and vertical Sobel filters can be applied to compute the HoG. When the templates are located in different CTUs, the Sobel filter may not be applied to the upper CTU boundary, or the DIMD technique may not be applied to the current block.

[0385] Different filters can be determined for HoG computation based on the block size. For example, when the block size is 4×4, 4×8, or 8×4, a 2×2 kernel filter can be used instead of a 3×3 Sobel filter for HoG computation.

[0386] An adaptive number of (neighboring) reference samples can be used for HoG calculations in DIMD. In this case, the number of sample rows in the DIMD template can be four or more.

[0387] Among candidate intra-prediction modes, the n intra-prediction modes with the highest histogram values ​​can be selected / extracted through HoG calculation. In this case, the n intra-prediction modes can be used to derive the prediction block for the current block. Furthermore, in this case, the prediction block can be derived using n intra-prediction modes and non-directional modes (e.g., planar or DC modes). In this case, predictors can be derived for the corresponding intra-prediction modes, and the prediction block can be derived by performing a weighted sum / weighted average of the predictors. That is, through prediction fusion, the predictors of the n (e.g., 5) selected / extracted intra-prediction modes can be fused with the predictors of the planar mode.

[0388] For example, n can be 5. In this case, the 5 derived patterns with the highest HoG can be derived. The derived patterns can be called DIMD-based patterns or DIMD-derived patterns. For example, the number n can be determined based on different block sizes. For example, when the block's W When H is 128 or greater, n can be 7; otherwise, n can be 5.

[0389] For HoG calculations in DIMD, specific block vectors can be used. For example, block vectors can be derived based on the motion vectors or block vectors of neighboring blocks. A reference region at the shifted position can be derived using specific block vectors based on the current block's position, and the HoG can be calculated using (reconstructed) reference samples within the reference region. In this case, the template can be located within the reference region, or it can be positioned above / to the left of a neighboring reference region. In this case, the size of the reference region can be the same as the size of the current block. Alternatively, the size of the reference region can differ from the size of the current block. For example, the width of the reference region can be half the width of the current block, and / or the height of the reference region can be half the height of the current block. In this case, the size of the template can also vary within the range of the width and height of the reference region.

[0390] The n intra-frame modes derived from DIMD can be referred to as DIMD-derived modes or DIMD-based modes. DIMD-derived modes can be included as candidates in the MPM list (e.g., the PMPM list).

[0391] When a DIMD technique (or type) is applied to the current block, the first intra-mode derived from DIMD can be stored as the intra-mode for the target block and can be referenced when configuring the MPM list for subsequent blocks.

[0392] When a DIMD technique (or type) is applied to the current block, the first intra-frame mode derived from DIMD can be referred to as the mode used to select the transform kernel (or transform set).

[0393] For weighted and / or weighted average or predictive fusion, weights between predictors are required. According to embodiments of this disclosure, lookup tables (LUTs) can be used to derive these weights. While existing full HoG calculations can be used to compute dimdMode... i However, Habove and Hleft can be calculated separately to calculate dimdMode. i Is it highly dependent on the top template or the left template, that is, dimMode? i Is it position-dependent?

[0394] When dimMode i When it is not position-dependent, the weight (wDimd) i (wPlanar) can be determined based on the HoG size, as in traditional methods.

[0395] When dimMode i When the problem is position-dependent, sample-based mixing can be applied. In this case, weights can be applied differently based on the samples.

[0396] When the upper HoG (e.g., Habove) is equal to or greater than twice the left HoG (e.g., Hleft), the weights can be based on the following equation.

[0397] [Equation 5]

[0398] here, It can represent the difference between the upper HoG and the left HoG, or the absolute value of that difference.

[0399] When the left HoG (e.g., Hleft) is equal to or greater than twice the upper HoG (e.g., Habove), the weights can be based on the following equation.

[0400] [Equation 6]

[0401] here, It can represent the difference between the left HoG and the top HoG, or the absolute value of that difference.

[0402] For DIMD, an additional W column can be used when the upper-right adjacent reference sample is available, and an additional H row can be used when the lower-left adjacent reference sample is available. In other words, the template region including adjacent (reconstructed) samples used for HOG calculation can be based on the availability of adjacent (reconstructed) samples.

[0403] When deriving n intra-frame (prediction) modes in DIMD, n intra-frame mode predictors and planar predictors can be combined. However, when deriving only one intra-frame mode in DIMD, only the intra-frame mode can be used. Alternatively, even when deriving only one intra-frame mode in DIMD, the intra-frame mode can be combined with a planar mode predictor. For example, when the HoG value of the intra-frame mode with the highest HoG in DIMD is equal to or greater than a threshold, only one intra-frame mode can be derived. In another example, when the first HoG value of the intra-frame mode with the highest HoG in DIMD is a specific value or greater than the second HoG value of the intra-frame mode with the second highest HoG, only one intra-frame mode can be derived. Furthermore, among the modes derived in DIMD, n intra-frame prediction modes with HoG equal to or greater than a specific threshold can be selected.

[0404] For a block to which DIMD is applied, the pattern derived from the first DIMD can be stored as the intra-prediction pattern for the current block. The stored pattern can be referenced as the patterns of neighboring blocks referenced for deriving the intra-prediction pattern for subsequent blocks and / or as the pattern for selecting the transform kernel (or transform set).

[0405] There may be situations where the pattern derived by the first DIMD may differ from the actual prediction pattern of the current block (or the best prediction pattern with the highest correlation to the current block). Therefore, in addition to the first intra-prediction pattern derived by DIMD, it is necessary to investigate patterns that can be stored as intra-prediction patterns for the current block. For example, when the HoG of the pattern derived by the first DIMD is greater than a certain threshold, the first pattern can be stored as an intra-prediction pattern for the current block; otherwise, a plane or DC pattern can be stored as an intra-prediction pattern for the current block. Alternatively, by comparing the first predictor (first prediction sample) of the current block generated by DIMD with the second predictor (second prediction sample) generated by the pattern derived by the first DIMD, when the difference between the two is within a certain SAD / SATD, the pattern derived by the first DIMD can be stored as an intra-prediction pattern for the current block; otherwise, a plane or DC pattern can be stored as an intra-prediction pattern for the current block.

[0406] In another example, a candidate list can be configured using m intra-prediction modes derived from DIMD, and any mode in the candidate list can be stored. Here, m can be equal to n, or m can be 2 or 3. In this case, the first predictor (first prediction sample) of the current block generated by DIMD can be compared with the predictors based on the corresponding m intra-prediction modes, and the intra-prediction mode with the lowest SAD / SATD among the m intra-prediction modes can be stored as the intra-prediction mode for the current block.

[0407] Alternatively, for the first predictor (first prediction sample) derived by the DIMD technique, the first intra-prediction mode derived by the DIMD technique (i.e., the intra-prediction mode with the highest HoG based on the HoG of the Sobel filter re-derived for the prediction sample (the first intra-prediction mode)) can be stored as the intra-prediction mode for the current block. In this case, the first intra-prediction mode derived from the template of the current block can be different from the first intra-prediction mode derived from the predictor of the current block.

[0408] For DIMD, DIMD-related information can be communicated using signals. DIMD-related information may include at least one of the following: DIMD enable flag, DIMD application flag, and / or DIMD flag.

[0409] The DIMD enable flag (dimd_enable_flag) can be signaled via high-level syntax (e.g., SPS). When the value of the DIMD enable flag is 1, the DIMD apply flag (dimd_applied_flag) and / or the DIMD flag (dimd_flag) can be signaled.

[0410] The DIMD application flag can be signaled at the PPS, Picture Header (PH), Slice Header (SH), or CTU level. Even when DIMD is available, it can be enabled / disabled at the picture, slice, or CTU level via the DIMD application flag. The DIMD flag can be signaled at the CU, PU, ​​or TU level. The DIMD flag can be signaled before the MPM flag (or PMPM flag).

[0411] DIMD-related information can be communicated using signals, for example, as shown below.

[0412] The DIMD flag (dimd_flag) can be signaled before the PMPM flag (pmpm_flag). In this case, when the value of the DIMD flag is 0, the PMPM flag can be signaled, and when the value of the DIMD flag is 1, the signaling of the PMPM flag can be omitted, as can be represented, for example, in the table below.

[0413] [Table 37]

[0414] The DIMD flag (dimd_flag) can be signaled before the reference line index (intra_luma_ref_idx) (in the parsing order). When multiple adjacent reference sample lines are used for intra-prediction of the current block, the reference line index is information indicating one or more of the adjacent reference sample lines. The reference line index can be signaled when the DIMD flag is not 1. When the DIMD flag is not 1 and the reference sample line index indicates 0, the PMPM flag can be signaled, which can be expressed, for example, as shown in the table below.

[0415] [Table 38]

[0416] The DIMD flag (dimd_flag) can be signaled after the reference row index (intra_luma_ref_idx) (in the parsing order). The DIMD flag can be signaled when the reference row index is 0. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample row index indicates 0, as can be expressed, for example, as shown in the table below.

[0417] [Table 39]

[0418] `intra_luma_ref_idx` represents the index of the adjacent reference sample line of the current block. A `intra_luma_ref_idx` value of 0 indicates that the reference sample line of the current block is adjacent to the top or left boundary of the current block.

[0419] When the signaling for pmpm_flag is omitted and intra_luma_ref_idx is not 0, the value of pmpm_flag can be inferred to be 1.

[0420] According to the aforementioned implementation, an intra-prediction mode for the current block can be derived based on a template, thereby reducing side information in the intra-prediction mode signaling while deriving the optimal prediction block for the current block. Furthermore, the intra-prediction mode stored for the current block where intra-prediction is performed based on a template and / or the intra-prediction mode used to select the transform kernel for the current block can be determined efficiently. Additionally, the intra-prediction mode derived based on DIMD can be used as a representative mode for a specific prediction technique or inter-frame prediction block, thereby improving correlation with neighboring blocks in intra-prediction.

[0421] Figure 23 A video / image encoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 23 The method disclosed in the article can be derived from Figure 2 The encoding device disclosed in the document executes the code. Specifically, for example, Figure 23 S2300 to S2330 can be executed by the predictor 220 of the encoding device 200, and Figure 23 S2340 can be executed by the entropy encoder 240 of the encoding device 200. Figure 23 The methods disclosed herein may include the aforementioned embodiments.

[0422] Reference Figure 23 The encoding device determines a filter for the template of the current block (S2300). The template can be set differently based on the size or position of the current block. For example, when the current block is a block of a first size, the filter can be a first filter, and when the current block is a block of a second size, the filter can be a second filter. The second filter can differ from the first filter in at least one aspect of shape or type. For example, the first filter can be a 3×3 filter, and the second filter can be a 2×2 filter, a 2×3 filter, or a 3×2 filter. For example, the first filter can be a 3×3 Sobel filter, and the second filter can be a 2×2 kernel filter.

[0423] The template size can be set differently based on the size of the current block. For example, when the current block has a first size (e.g., 16x16 or larger), the template size can be smaller than the template size when the current block has a second size (e.g., 8x8 or smaller). For example, the first size can be larger than the second size in at least one of the width or height. This is because when the block is partitioned into larger sizes, color variations tend to be small.

[0424] The template can include a left template and a top template. Based on the CTU, chunk, or slice, at least one of the left and top templates may be unavailable. For example, when the top template does not belong to the current coding tree unit (CTU) to which the current block belongs, the filter may not be applied to the top template. In this case, for example, the filter can be applied to the left template to compute the HoG. In another example, the top template can be generated by performing a padding or replacement process on it, and the filter can be applied to both the left and top templates. In this case, the distortion of the HoG value can be reduced by decreasing the dependence on the left template.

[0425] The template can be located near the current block. Alternatively, the template can be located outside the current block. For example, a block vector for the current block can be derived, and the template can be located at a position shifted from the current block's position based on the block vector. In this example, the size of the template can be equal to the size of the current block. In another example, the width of the template can be half the width of the current block, and / or the height of the template can be half the height of the current block.

[0426] The coding device derives n intra-prediction modes for the current block (S2310). The coding device applies a filter to a template to derive a gradient histogram (HoG) for the candidate intra-prediction modes, and derives the first n intra-prediction modes among the candidate intra-prediction modes into n intra-prediction modes based on the HoG.

[0427] The value of n can be set differently based on the size of the current block. For example, when the width of the current block... When the height is 128 or greater, n can be 7; otherwise, n can be 5.

[0428] The encoding device generates a prediction block for the current block (S2320). The encoding device can generate the prediction block for the current block based on n intra-prediction modes and non-directional modes. The n intra-prediction modes can include directional intra-prediction modes. The non-directional modes can be planar modes or DC modes.

[0429] n predictors can be derived from n intra-frame prediction modes, and a single predictor can be derived from a non-directional prediction mode. A prediction block can be derived from a weighted sum or weighted average of n predictors and a single predictor.

[0430] For example, n can be 1. In this case, only one intra-prediction mode can be derived when the HoG value of the intra-prediction mode with the highest HoG is equal to or greater than a threshold. In another example, only one intra-prediction mode can be derived when the first HoG value of the intra-prediction mode with the highest HoG is a specific value or greater than the second HoG value of the intra-prediction mode with the second highest HoG. Additionally, among the modes derived by DIMD, n intra-prediction modes with HoG equal to or greater than a specific threshold can be selected.

[0431] For example, the intra prediction mode stored for the current block can be the first of n intra prediction modes. The stored intra prediction modes can be referenced to derive intra prediction modes for subsequent blocks or to derive the transform kernel (transform set) associated with the current block.

[0432] For example, when the HoG of the first intra-prediction mode out of n intra-prediction modes is greater than a specific threshold, the first intra-prediction mode can be set as the intra-prediction mode for the current block storage. When the HoG of the first intra-prediction mode is less than or equal to the specific threshold, the plane mode or DC mode can be set as the intra-prediction mode for the current block storage.

[0433] For example, n intra-prediction patterns can be referred to as DIMD-derived patterns or DIMD-based patterns. There may be a situation where the first DIMD-derived pattern is different from the actual intra-prediction pattern for the current block (or the best prediction pattern with the highest correlation to the current block). Therefore, in addition to the first intra-prediction pattern derived by DIMD, patterns that can be stored as intra-prediction patterns for the current block can also be derived. For example, when the HoG of the first DIMD-derived pattern is greater than a certain threshold, the first pattern can be stored as an intra-prediction pattern for the current block; otherwise, a planar or DC pattern can be stored as an intra-prediction pattern for the current block. Alternatively, by comparing the first predictor (first prediction sample) of the current block generated by DIMD with the second predictor (second prediction sample) generated by the first DIMD-derived pattern, the first DIMD-derived pattern can be stored as an intra-prediction pattern for the current block when the difference between them is within a certain SAD / SATD; otherwise, a planar or DC pattern can be stored as an intra-prediction pattern for the current block.

[0434] In another example, a candidate list can be configured using m intra-prediction modes derived from DIMD, and any mode in the candidate list can be stored. Here, m can be equal to n, or m can be 2 or 3. In this case, the first predictor (first prediction sample) of the current block generated by DIMD can be compared with the predictors based on the corresponding m intra-prediction modes, and the intra-prediction mode with the lowest SAD / SATD among the m intra-prediction modes can be stored as the intra-prediction mode for the current block.

[0435] Alternatively, for the first predictor derived by DIMD (the first predicted sample), the first intra-prediction mode derived by DIMD (i.e., the intra-prediction mode with the highest HoG based on the re-derived HoG of the Sobel filter for the predicted sample, (the first intra-prediction mode)) can be stored as the intra-prediction mode for the current block. In this case, the first intra-prediction mode derived from the template of the current block can be different from the first intra-prediction mode derived from the predictor of the current block.

[0436] The encoding device generates prediction-related information (S2330). The prediction-related information may include the various information described above. The prediction-related information may include information disclosed in the table above. The prediction-related information may include information related to DIMD.

[0437] For example, prediction-related information may include DIMD flags indicating whether decoder-side motion vector derivation (DIMD) is applied to the current block.

[0438] It can indicate that the value based on the DIMD flag information is 1, and generate a prediction block for the current block based on n intra-frame prediction modes and planar modes.

[0439] Prediction-related information may include the DIMD enable flag and / or the DIMD apply flag. The DIMD enable flag (e.g., the dimd_enabled flag) can be signaled using high-level syntax (e.g., SPS). When the value of the DIMD enable flag is 1, the DIMD apply flag (e.g., dimd_applied_flag) and / or the DIMD flag (e.g., dimd_flag) can be signaled.

[0440] The DIMD application flag can be signaled at the PPS, Picture Header (PH), Slice Header (SH), or CTU level. Even when DIMD is available, it can be enabled / disabled via the DIMD application flag at the picture, slice, or CTU level. The DIMD flag can be signaled at the CU, PU, ​​or TU level. The DIMD flag can be signaled before the MPM flag (or PMPM flag).

[0441] For example, prediction-related information may include a Most Probable Pattern (MPM) flag, and in signaling order, the DIMD flag may precede the MPM flag. The MPM flag can be signaled based on the DIMD flag being 0.

[0442] For example, prediction-related information may include reference sample row index information indicating the index of adjacent reference sample rows, and the MPM flag may be signaled when the value of the DIMD flag is 0 and the value of the reference sample row index is 0.

[0443] For example, prediction-related information may include reference sample row index information indicating the index of adjacent reference sample rows, and the DIMD flag may be signaled when the value of the index of the reference sample row is 0.

[0444] The encoding device encodes image information, including prediction-related information (S2340). The image information may be referred to as video information.

[0445] According to embodiments of this disclosure, image information may include various types of information. For example, image information may include information disclosed in at least one of the tables described above.

[0446] Image information can include residual information. Residual information is information about the residual samples. Residual information can include information about the quantized transform coefficients for the residual samples.

[0447] Encoded image information can be output as a bitstream. The bitstream can be sent to a decoding device via a network or storage medium. For example, image data including a bitstream can be sent to the decoding device by a sending device (or transmitter). In this case, the image data including the bitstream can be sent to the decoding device via a streaming server.

[0448] Furthermore, as mentioned above, the encoding device can generate a reconstructed image (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This allows the encoding device to derive the same prediction results as the decoding device, thereby improving encoding efficiency. Therefore, the encoding device can store the reconstructed image (or reconstructed samples and reconstructed blocks) in memory and use it as a reference image for inter-frame prediction. As mentioned above, the in-loop filtering process can be further applied to the reconstructed image.

[0449] According to the aforementioned implementation, an intra-prediction mode for the current block can be derived based on a template, thereby reducing side information in the intra-prediction mode signaling while deriving the optimal prediction block for the current block. Furthermore, the intra-prediction mode stored for the current block where intra-prediction is performed based on a template and / or the intra-prediction mode used to select the transform kernel for the current block can be determined efficiently. Additionally, the intra-prediction mode derived based on DIMD can be used as a representative mode for a specific prediction technique or inter-frame prediction block, thereby improving correlation with neighboring blocks in intra-prediction.

[0450] Figure 24 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 24 The method disclosed in the article can be derived from Figure 3 The decoding device disclosed in the document performs the operation. Specifically, for example, Figure 24 The S2400 can be executed by the entropy decoder 310 of the decoding device 300. Figure 24 The methods disclosed herein may include the aforementioned embodiments.

[0451] Reference Figure 24 The decoding device determines a filter for the template of the current block (S2400). The template can be set differently based on the size or position of the current block. For example, when the current block is a block of a first size, the filter can be a first filter, and when the current block is a block of a second size, the filter can be a second filter. The second filter can differ from the first filter in at least one aspect of shape or type. For example, the first filter can be a 3×3 filter, and the second filter can be a 2×2 filter, a 2×3 filter, or a 3×2 filter. For example, the first filter can be a 3×3 Sobel filter, and the second filter can be a 2×2 kernel filter.

[0452] The template size can be set differently based on the size of the current block. For example, when the current block has a first size (e.g., 16x16 or larger), the template size can be smaller than the template size when the current block has a second size (e.g., 8x8 or smaller). For example, the first size can be larger than the second size in at least one of the width or height. This is because when the block is partitioned into larger sizes, color variations tend to be small.

[0453] The template can include a left template and a top template. Based on the CTU, chunk, or slice, at least one of the left and top templates may be unavailable. For example, when the top template does not belong to the current coding tree unit (CTU) to which the current block belongs, the filter may not be applied to the top template. In this case, for example, the filter can be applied to the left template to compute the HoG. In another example, the top template can be generated by performing a padding or replacement process on it, and the filter can be applied to both the left and top templates. In this case, the distortion of the HoG value can be reduced by decreasing the dependence on the left template.

[0454] The template can be located near the current block. Alternatively, the template can be located outside the current block. For example, a block vector for the current block can be derived, and the template can be located at a position shifted from the current block's position based on the block vector. In this example, the size of the template can be equal to the size of the current block. In another example, the width of the template can be half the width of the current block, and / or the height of the template can be half the height of the current block.

[0455] The decoding device derives n intra-prediction modes for the current block (S2410). The decoding device applies a filter to a template to derive a gradient histogram (HoG) for the candidate intra-prediction modes, and derives the first n intra-prediction modes among the candidate intra-prediction modes into n intra-prediction modes based on the HoG.

[0456] The value of n can be set differently based on the size of the current block. For example, when the width of the current block... When the height is 128 or greater, n can be 7; otherwise, n can be 5.

[0457] The decoding device generates a prediction block for the current block (S2420). The encoding device can generate a prediction block for the current block based on n intra-frame prediction modes and non-directional modes. The n intra-frame prediction modes can include directional intra-frame prediction modes. The non-directional modes can be planar modes or DC modes.

[0458] n predictors can be derived from n intra-frame prediction modes, and a single predictor can be derived from a non-directional prediction mode. A prediction block can be derived from a weighted sum or weighted average of n predictors and a single predictor.

[0459] For example, n can be 1. In this case, only one intra-prediction mode can be derived when the HoG value of the intra-prediction mode with the highest HoG is equal to or greater than a threshold. In another example, only one intra-prediction mode can be derived when the first HoG value of the intra-prediction mode with the highest HoG is a specific value or greater than the second HoG value of the intra-prediction mode with the second highest HoG. Additionally, among the modes derived by DIMD, n intra-prediction modes with HoG equal to or greater than a specific threshold can be selected.

[0460] For example, the intra prediction mode stored for the current block can be the first of n intra prediction modes. The stored intra prediction modes can be referenced to derive intra prediction modes for subsequent blocks or to derive the transform kernel (transform set) associated with the current block.

[0461] For example, when the HoG of the first intra-prediction mode out of n intra-prediction modes is greater than a specific threshold, the first intra-prediction mode can be set as the intra-prediction mode for the current block storage. When the HoG of the first intra-prediction mode is less than or equal to the specific threshold, the plane mode or DC mode can be set as the intra-prediction mode for the current block storage.

[0462] For example, n intra-prediction patterns can be referred to as DIMD-derived patterns or DIMD-based patterns. There may be a situation where the first DIMD-derived pattern is different from the actual intra-prediction pattern for the current block (or the best prediction pattern with the highest correlation to the current block). Therefore, in addition to the first intra-prediction pattern derived by DIMD, patterns that can be stored as intra-prediction patterns for the current block can also be derived. For example, when the HoG of the first DIMD-derived pattern is greater than a certain threshold, the first pattern can be stored as an intra-prediction pattern for the current block; otherwise, a planar or DC pattern can be stored as an intra-prediction pattern for the current block. Alternatively, by comparing the first predictor (first prediction sample) of the current block generated by DIMD with the second predictor (second prediction sample) generated by the first DIMD-derived pattern, the first DIMD-derived pattern can be stored as an intra-prediction pattern for the current block when the difference between them is within a certain SAD / SATD; otherwise, a planar or DC pattern can be stored as an intra-prediction pattern for the current block.

[0463] In another example, a candidate list can be configured using m intra-prediction modes derived from DIMD, and any mode in the candidate list can be stored. Here, m can be equal to n, or m can be 2 or 3. In this case, the first predictor (first prediction sample) of the current block generated by DIMD can be compared with the predictors based on the corresponding m intra-prediction modes, and the intra-prediction mode with the lowest SAD / SATD among the m intra-prediction modes can be stored as the intra-prediction mode for the current block.

[0464] Alternatively, for the first predictor derived by DIMD (the first predicted sample), the first intra-prediction mode derived by DIMD (i.e., the intra-prediction mode with the highest HoG based on the re-derived HoG of the Sobel filter for the predicted sample, (the first intra-prediction mode)) can be stored as the intra-prediction mode for the current block. In this case, the first intra-prediction mode derived from the template of the current block can be different from the first intra-prediction mode derived from the predictor of the current block.

[0465] Although not shown, the decoding device can obtain prediction-related information from the bitstream. The decoding device can obtain image information, including prediction-related information, from the bitstream. The image information may further include residual information as described above.

[0466] Forecast-related information may include the information disclosed in the table above. Forecast-related information may include information related to DIMD.

[0467] For example, prediction-related information may include DIMD flags indicating whether decoder-side motion vector derivation (DIMD) is applied to the current block.

[0468] It can indicate that the value based on the DIMD flag information is 1, and generate a prediction block for the current block based on n intra-frame prediction modes and planar modes.

[0469] Prediction-related information may include the DIMD enable flag and / or the DIMD apply flag. The DIMD enable flag (e.g., the dimd_enabled flag) can be signaled using high-level syntax (e.g., SPS). When the value of the DIMD enable flag is 1, the DIMD apply flag (e.g., dimd_applied_flag) and / or the DIMD flag (e.g., dimd_flag) can be signaled.

[0470] The DIMD application flag can be signaled at the PPS, Picture Header (PH), Slice Header (SH), or CTU level. Even when DIMD is available, it can be enabled / disabled via the DIMD application flag at the picture, slice, or CTU level. The DIMD flag can be signaled at the CU, PU, ​​or TU level. The DIMD flag can be signaled before the MPM flag (or PMPM flag).

[0471] For example, prediction-related information may include a Most Probable Pattern (MPM) flag, and in signaling order, the DIMD flag may precede the MPM flag. The MPM flag can be signaled based on the DIMD flag being 0.

[0472] For example, prediction-related information may include reference sample row index information indicating the index of adjacent reference sample rows, and the MPM flag may be signaled when the value of the DIMD flag is 0 and the value of the reference sample row index is 0.

[0473] For example, prediction-related information may include reference sample row index information indicating the index of adjacent reference sample rows, and the DIMD flag may be signaled when the value of the index of the reference sample row is 0.

[0474] As mentioned above, if necessary, a prediction sample filtering process can also be performed on all or part of the prediction blocks (prediction samples) of the current block.

[0475] The decoding device can generate reconstructed samples based on the predicted samples for the current block. For example, the decoding device can generate reconstructed samples for the current block based on the residual samples and predicted samples for the current block. Residual samples for the current block can be generated based on the received residual information. Additionally, for example, the decoding device can generate a reconstructed image including the reconstructed samples. As described above, the in-loop filtering process can be further applied to the reconstructed image.

[0476] According to the aforementioned implementation, an intra-prediction mode for the current block can be derived based on a template, thereby reducing side information in the intra-prediction mode signaling while deriving the optimal prediction block for the current block. Furthermore, the intra-prediction mode stored for the current block where intra-prediction is performed based on a template and / or the intra-prediction mode used to select the transform kernel for the current block can be determined efficiently. Additionally, the intra-prediction mode derived based on DIMD can be used as a representative mode for a specific prediction technique or inter-frame prediction block, thereby improving correlation with neighboring blocks in intra-prediction.

[0477] Although the method is described as a series of steps or blocks based on the flowchart in the foregoing embodiments, the embodiments are not limited to the order of the steps, and a step may occur in a different order or simultaneously with another step described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps may be deleted without affecting the scope of the embodiments of this disclosure.

[0478] The aforementioned methods according to embodiments of the present disclosure can be implemented in software, and the encoding and / or decoding devices according to the present disclosure can be included in apparatus for performing image processing, such as TVs, computers, smartphones, set-top boxes, and display devices.

[0479] The embodiments described above can be implemented in the form of a recording medium including computer-executable (program) instructions, such as program modules executed by a computer. These modules can be stored in memory and executed by a processor. The memory can reside internally or externally to the processor and can be connected to the processor via various known means. The computer-readable medium can be any available medium accessible to a computer and can include volatile and non-volatile media, as well as removable and non-removable media. Furthermore, the computer-readable medium can include both computer storage media and communication media. Computer storage media can include volatile and non-volatile media, as well as removable and non-removable media, implemented using any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically include computer-readable instructions, data structures, program modules, other data in modulated data signals such as carrier waves, or other transmission mechanisms, and include any information delivery medium.

[0480] Furthermore, the embodiments of this disclosure described above can be implemented as a computer program (or computer program product) including computer-executable instructions. The computer program may include programmable machine instructions processed by a processor and may be implemented using a high-level programming language, an object-oriented programming language, assembly language, or machine language. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical media, or solid-state drive (SSD)).

[0481] Therefore, when the above-described computer program is executed by a computing device, embodiments of the present disclosure can be implemented. The computing device may include at least some of a processor, memory, storage devices, high-speed interfaces connected to the memory and high-speed expansion ports, and low-speed interfaces connected to low-speed buses and storage devices. These components may be interconnected via various buses and may be mounted on a common motherboard or otherwise suitably mounted.

[0482] The processor can process instructions within a computing device. These instructions may include instructions stored in memory or a storage device to display graphical information for providing a graphical user interface (GUI) on an external input / output device, such as a display connected to a high-speed interface. In another embodiment, multiple processors and / or multiple buses may be suitably used with multiple memories and memory types. Furthermore, the processor can be implemented as a chipset comprising multiple independent analog and / or digital processor chips.

[0483] Memory stores information within a computing device. For example, memory may include volatile memory cells or a set of volatile memory cells. In another example, memory may include non-volatile memory cells or a set of non-volatile memory cells. Memory may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0484] Storage devices can provide high-capacity storage space for computing devices. Storage devices can be computer-readable media or components that include computer-readable media. For example, storage devices can include devices within a storage area network (SAN) or other components, and can be floppy disk devices, hard disk devices, optical disk devices, magnetic tape devices, flash memory, other similar semiconductor memory devices, or device arrays.

[0485] The network can be implemented as a wired network (such as a local area network (LAN), a wide area network (WAN), or a value-added network (VAN)) or various types of wireless networks (such as mobile radio communication networks or satellite communication networks).

[0486] Although this disclosure has been described with reference to embodiments shown in the accompanying drawings, these embodiments are merely exemplary. Those skilled in the art will understand that various modifications and variations of the embodiments are possible. That is, the scope of this disclosure is not limited to the described embodiments, and various modifications and changes made by those skilled in the art based on the basic concepts defined in the appended claims also fall within the scope of the claims. Therefore, the true technical scope of this disclosure should be determined by the technical spirit of the appended claims.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising the following steps: Determine the filter to be applied to the template of the current block; Based on the template and the filter, derive n intra-frame prediction modes for the current block; as well as Generate a prediction block for the current block based on the n intra-frame prediction modes and non-directional modes. Among them, the n intra-frame prediction modes include directional intra-frame prediction modes, and The non-directional mode is either a planar mode or a DC mode.

2. The image decoding method according to claim 1, wherein, When the current block is a block of a first size, the filter is a first filter. When the current block is a block of the second size, the filter is the second filter, and The second filter differs from the first filter in at least one aspect, either in shape or type.

3. The image decoding method according to claim 2, wherein, The first filter is a 3×3 filter, and the second filter is a 2×2 filter.

4. The image decoding method according to claim 2, wherein, The first filter is a 3×3 Sobel filter, and the second filter is a 2×2 kernel filter.

5. The image decoding method according to claim 1, wherein, The size of the template is set differently based on the size of the current block.

6. The image decoding method according to claim 1, wherein, The template includes a left template and a top template, and the filter is not applied to the top template when the top template does not belong to the current coding tree unit (CTU) to which the current block belongs.

7. The image decoding method according to claim 1, further comprising the following steps: Receive prediction-related information from the bitstream. The prediction-related information includes indicators of whether the decoder-side motion vector derivation DIMD is applied to the DIMD flag of the current block, and Based on the value of the DIMD flag being 1, the prediction block for the current block is generated based on the n intra-frame prediction modes and the plane mode.

8. The image decoding method according to claim 7, wherein, The prediction-related information includes the DIMD enable flag. When the value of the DIMD enable flag is 1, the DIMD flag is signaled, and The DIMD enable flag is signaled via a sequence parameter set.

9. The image decoding method according to claim 7, wherein, The prediction-related information includes the most probable pattern (MPM) flag, and According to the signaling sequence, the DIMD flag precedes the MPM flag.

10. The image decoding method according to claim 9, wherein, The MPM flag is signaled when the value of the DIMD flag is 0.

11. The image decoding method according to claim 9, wherein, The prediction-related information includes reference sample row index information indicating the index of adjacent reference sample rows, and The MPM flag is signaled when the value of the DIMD flag is 0 and the value of the index of the reference sample row is 0.

12. The image decoding method according to claim 7, wherein, The prediction-related information includes reference sample row index information indicating the index of adjacent reference sample rows, and When the value of the index of the reference sample row is 0, the DIMD flag is notified by a signal.

13. The image decoding method according to claim 1, wherein, The steps of deriving the n intra-frame prediction modes for the current block based on the template and the filter include: The gradient histogram (HoG) for the candidate intra-prediction mode is derived by applying the filter to the template; and Based on the HoG, the first n intra-prediction modes among the candidate intra-prediction modes are derived as the n intra-prediction modes.

14. The image decoding method according to claim 1, wherein, Based on the n intra-frame prediction modes, n predictors are derived. A predictor is derived based on the aforementioned non-directional prediction mode, and The prediction block is derived based on the weighted sum or weighted average of the n predictors and the one predictor.

15. The image decoding method according to claim 14, wherein, n is set differently based on the size of the current block.

16. The image decoding method according to claim 1, wherein, Derive the block vector for the current block. The template is located at a position shifted from the position of the current block based on the block vector, and The size of the template is equal to the size of the current block.

17. The image decoding method according to claim 1, wherein, The intra-prediction mode for the current block storage is the first intra-prediction mode among the n intra-prediction modes.

18. The image decoding method according to claim 1, wherein, When the HoG of the first intra-prediction mode among the n intra-prediction modes is greater than a specific threshold, the first intra-prediction mode is set to the intra-prediction mode for the current block storage, and When the HoG of the first intra-prediction mode is less than or equal to the specific threshold, the plane mode or the DC mode is set to the intra-prediction mode for the current block storage.

19. An image encoding method performed by an encoding device, the image encoding method comprising the following steps: Determine the filter to be applied to the template of the current block; Based on the template and the filter, n intra-frame prediction modes for the current block are derived; Based on the n intra-frame prediction modes and non-directional modes, a prediction block is generated for the current block; Generate prediction-related information; as well as The image information, including the prediction-related information, is encoded. Among them, the n intra-frame prediction modes include directional intra-frame prediction modes, and The non-directional mode is either a planar mode or a DC mode.

20. A method for transmitting image data, the method comprising the following steps: Obtaining a bitstream generated by an image coding method, the image coding method comprising the following steps: determining a filter for a template applied to a current block; deriving n intra-frame prediction modes for the current block based on the template and the filter; generating a prediction block for the current block based on the n intra-frame prediction modes and non-directional modes; generating prediction-related information; and encoding image information including the prediction-related information; and Send image data including the bitstream. Among them, the n intra-frame prediction modes include directional intra-frame prediction modes, and The non-directional mode is either a planar mode or a DC mode.

Images