Image encoding method based on intra prediction and apparatus therefor

Abstract

An image decoding method according to an embodiment of the present disclosure includes the steps of: obtaining prediction-related information through a bitstream; deriving a most probable mode (MPM) list for a current block; deriving an intra prediction mode of the current block based on the MPM list and the prediction-related information; and generating a prediction sample for the current block based on the derived intra prediction mode, wherein the MPM list includes: a first MPM list, a second MPM list, and a third MPM list, the first MPM list does not include a candidate included in the second MPM list, and the third MPM list includes the candidate included in the second MPM list.

Description

Technical Field

[0001] This disclosure relates to an image / video encoding method and an image / video encoding device. Background Technology

[0002] Image / video encoding is used in a variety of applications, such as digital storage media, television broadcasting, video streaming services, and real-time communications. Recently, the demand for high-resolution and high-quality images / videos has been increasing across various fields.

[0003] As images / videos achieve higher resolution and quality, their data size increases, leading to a relative increase in the amount of information or bits transmitted. Therefore, the costs of transmission and storage rise when using conventional media such as wired and wireless broadband lines to transmit image data or when using existing storage media to store image / video data.

[0004] Recently, there has been an increasing interest in and demand for immersive media such as virtual reality (VR), augmented reality (AR), mixed reality (MR) content, and holograms. Furthermore, there is a growing number of attempts to use immersive media to deliver immersive experiences in gaming, education, healthcare, real estate, and marketing.

[0005] Therefore, efficient image / video compression technology is needed to efficiently compress, send, store, and play high-resolution and high-quality image / video information with various characteristics. Summary of the Invention

[0006] Technical solution

[0007] According to embodiments of this disclosure, a method and apparatus for enhancing video / image coding efficiency are provided.

[0008] According to embodiments of this disclosure, a video / image coding method and a video / image coding apparatus based on intra-frame prediction are provided.

[0009] According to embodiments of this disclosure, an image decoding method performed by a decoding device is provided. The method includes: obtaining prediction-related information via a bitstream; deriving a prediction type for a current block based on the prediction-related information; deriving a template for the current block based on the prediction type; deriving n intra-frame prediction modes for the current block based on the template for the current block; and generating a prediction block for the current block based on the n intra-frame prediction modes, wherein the size of the template is determined based on the size of the current block.

[0010] According to embodiments of this disclosure, an image encoding method performed by an encoding device is provided. The method includes: determining a prediction type for a current block; deriving a template for the current block based on the prediction type; deriving n intra-frame prediction modes for the current block based on the template; generating a prediction block for the current block based on the n intra-frame prediction modes; generating prediction-related information based on the determined prediction type; and encoding image information including the prediction-related information, wherein the size of the template is determined based on the size of the current block.

[0011] According to embodiments of this disclosure, a decoding apparatus for image decoding is provided. The decoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: obtaining prediction-related information via a bitstream; deriving a prediction type for a current block based on the prediction-related information; deriving a template for the current block based on the prediction type; deriving n intra-frame prediction modes for the current block based on the template for the current block; and generating a prediction block for the current block based on the n intra-frame prediction modes, wherein the size of the template is determined based on the size of the current block.

[0012] According to embodiments of this disclosure, an encoding apparatus for image encoding is provided. The encoding apparatus includes a memory and at least one processor connected to the memory, wherein the at least one processor is configured to perform the following operations: determining a prediction type for a current block; deriving a template for the current block based on the prediction type; deriving n intra-frame prediction modes for the current block based on the template; generating a prediction block for the current block based on the n intra-frame prediction modes; generating prediction-related information based on the determined prediction type; and encoding image information including the prediction-related information, wherein the size of the template is determined based on the size of the current block.

[0013] According to embodiments of the present disclosure, a method for transmitting video / image data is provided, the video / image data comprising a bitstream generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0014] According to embodiments of the present disclosure, an apparatus for transmitting video / image data is provided, the video / image data comprising a bitstream generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0015] According to embodiments of the present disclosure, a computer-readable storage medium is provided that stores a program for performing a method according to at least one embodiment of the present disclosure.

[0016] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded video / image information generated by a video / image encoding method according to at least one embodiment of the present disclosure.

[0017] According to embodiments of the present disclosure, a computer-readable digital storage medium is provided that stores encoded information or encoded video / image information, the encoded information or encoded video / image information causing a decoding device to perform a video / image decoding method according to at least one embodiment of the present disclosure.

[0018] Invention Effects

[0019] According to the embodiments of this disclosure, the overall video / image compression efficiency can be improved.

[0020] According to embodiments of this disclosure, prediction performance for the current block can be enhanced.

[0021] According to embodiments of this disclosure, an efficient intra-prediction mode candidate list can be configured for intra-prediction mode signaling. Attached Figure Description

[0022] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0023] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied.

[0024] Figure 3 This diagram schematically illustrates the configuration of a video / image decoding device to which embodiments of the present disclosure can be applied.

[0025] Figure 4 The intra-frame prediction process is illustrated.

[0026] Figure 5 An example of a video / image coding method based on intra-frame prediction is shown.

[0027] Figure 6 An example of a video / image decoding method based on intra-frame prediction is shown.

[0028] Figure 7 An example of a directional intra-frame prediction mode is shown.

[0029] Figure 8 An example of template-based HoG computation in DIMD is shown.

[0030] Figure 9 An example of adjacent blocks used to derive the MPM list is shown.

[0031] Figure 10 Intra-prediction modes derived from neighboring blocks and DIMD-based modes are shown.

[0032] Figure 11 The intra-prediction mode of the previous block within a specific region is shown.

[0033] Figure 12 A video / image encoding method according to an embodiment of the present disclosure is illustrated schematically.

[0034] Figure 13 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Detailed Implementation

[0035] Because this disclosure can have various variations and implementations, specific embodiments are illustrated in the accompanying drawings and will be described in detail. However, it should be understood that the embodiments of this disclosure are not intended to be limited to a particular embodiment. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the technical spirit of this disclosure. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise. As used herein, the term “and / or” includes any one and all combinations of two or more associated listed items. As used herein, the terms “comprising,” “including,” and “having” specify the presence of stated features, numbers, operations, elements, components, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, numbers, operations, elements, components, and / or combinations thereof. In this disclosure, the use of the term “may” in connection with examples or embodiments (e.g., regarding what an example or embodiment may include or implement) indicates the existence of at least one example or embodiment that includes or implements such a feature, but all examples are not limited thereto, and the corresponding feature or configuration may be omitted.

[0036] For ease of explanation of the different features, each component in the accompanying drawings described in this disclosure is shown independently, but this does not mean that each component is implemented as a separate piece of hardware or separate piece of software. For example, two or more of these components may be combined to form a single component, or a single component may be divided into multiple components. Embodiments in which components are integrated and / or separated are also included within the scope of this disclosure without departing from its spirit.

[0037] In this disclosure, “A or B” may mean “A only”, “B only”, or “both A and B”. In other words, “A or B” in this disclosure may be interpreted as “A and / or B”. For example, “A, B or C” in this disclosure may mean “A only”, “B only”, “C only”, or “any one and all combinations of A, B and C”.

[0038] The forward slash ( / ) or comma used in this article can mean "and / or". For example, "A / B" can mean "A and / or B". Therefore, "A / B" can mean "A only", "B only", or "both A and B". For example, "A, B, C" can mean "A, B, or C".

[0039] In this disclosure, "at least one of A and B" may mean "only A", "only B" or "both A and B". Furthermore, the expression "at least one of A or B" or "at least one of A and / or B" may be interpreted in the same way as "at least one of A and B".

[0040] In this disclosure, "at least one of A, B, and C" may mean "only A", "only B", "only C", or "any one and all combinations of A, B, and C". Furthermore, "at least one of A, B, or C" or "at least one of A, B, and / or C" may mean "at least one of A, B, and C".

[0041] The parentheses used in this disclosure can indicate "for example". Specifically, when indicated as "prediction (intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction". In other words, "prediction" in this disclosure is not limited to "intra-frame prediction", and "intra-frame prediction" can be cited as an example of "prediction". Furthermore, even when indicated as "prediction (i.e., intra-frame prediction)", "intra-frame prediction" can be cited as an example of "prediction".

[0042] In this disclosure, the technical features explained individually in a single figure may be implemented independently or simultaneously.

[0043] This disclosure relates to video / image coding. For example, the methods / implementations described in this disclosure can be applied to methods disclosed in enhanced compression models or the H.267 standard. Additionally, the methods / implementations disclosed herein can be applied to methods disclosed in the AOMedia Video2 (AV2) standard or next-generation video / image coding standards (e.g., H.268 and H.269).

[0044] In this disclosure, encoding may include encoding and / or decoding. In this disclosure, image encoding may be used interchangeably with video encoding.

[0045] In this disclosure, video can refer to a collection of images over time. A frame typically refers to a unit representing a single image at a specific moment, while a slice / tile refers to a unit that forms part of a frame during encoding. A slice / tile may include one or more coding tree units (CTUs). A single frame may include one or more slices / tiles. A tile may represent a rectangular area of ​​a CTU within a specific tile row and column of a frame.

[0046] A single frame can be divided into two or more sub-frames. A sub-frame can be a rectangular area of ​​one or more slices of the frame.

[0047] A pixel or cell can refer to the smallest unit that makes up a picture (or image). A "sample" can be used as the term corresponding to a pixel. A sample can typically represent a pixel or a pixel value, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0048] A unit can represent a basic unit of image processing. A unit may include a specific region of the image and at least one of the information associated with that region. A single unit may include a luminance block and two chrominance (e.g., Cb and cr) blocks. In some cases, the term "unit" may be used interchangeably with terms such as "block" or "region". Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients in M ​​columns and N rows.

[0049] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Furthermore, throughout the drawings, the same reference numerals may be used to denote similar elements, and redundant descriptions of similar elements may be omitted.

[0050] Figure 1 An example of a video / image coding system to which embodiments of the present disclosure may be applied is illustrated schematically.

[0051] Reference Figure 1 A video / image encoding system may include a first device (encoding device) and a second device (decoding device). The first device may deliver encoded video / image information or data to the second device in the form of a file or stream via a digital storage medium or network.

[0052] A video / image encoding system may also include a video / image acquisition device and a video / image renderer. The video / image acquisition device may be included in the encoding device, or it may be configured as a separate device or an external component. The video / image renderer may be included in the decoding device, or it may be configured as a separate device or an external component.

[0053] The first device may include a transmitter as an internal component, or a transmitter as a separate device or an external component.

[0054] The second device may include a receiver as an internal component, or as a receiver as a separate device or an external component.

[0055] An encoding device can be called an encoder, and a decoding device can be called a decoder. A transmitter can be included in the encoding device. A receiver can be included in the decoding device. A renderer can include a display, and the display can be configured as a separate device or an external component.

[0056] Decoding and encoding devices using embodiments of this disclosure can be included in multimedia broadcasting transmitters / receivers, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video conferencing devices, real-time communication devices such as video communication, mobile streaming devices, storage media, portable cameras, video-on-demand (VoD) service providers, over-the-top (OTT) video devices, Internet streaming service providers, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, video telephony devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicle terminals), aircraft terminals, and ship terminals), and medical video devices, and can be used to process video signals or data signals. For example, over-the-top (OTT) video devices can include game consoles, Blu-ray players, Internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0057] A video / image acquisition device can acquire video / image sources. The video / image acquisition device can acquire video / images through processes of capturing, compositing, or generating video / images. The video / image acquisition device may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras and a video / image archive including previously captured video / images. The video / image generation device may include, for example, a portable camcorder, a computer, a tablet PC, and a smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, in which case the process of generating relevant data can replace the video / image capture process. The video / image source can perform a video / image preprocessing process to input the optimized video / image into the encoder.

[0058] Encoding devices can encode input video / images. For compression and encoding efficiency, encoding devices can perform a series of processes such as prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output as a bitstream.

[0059] A transmitter can send encoded images / image information or data, output as a bitstream, to a receiver in a receiving device via a digital storage medium or network, either as a file or a stream. The encoded images / image information or data output as a bitstream can be sent to the receiver via a streaming server. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can receive / extract the bitstream and send the received bitstream to a decoding device.

[0060] During the sending or receiving of bitstreams, the streaming server may temporarily store the bitstreams. The streaming server sends multimedia data to the user's device via a web server based on user requests, and the web server acts as an intermediary for notifying the user of available services. When a user requests a desired service from the web server, the web server forwards the request to the streaming server, and the streaming server sends the multimedia data to the user. The content streaming system may include a separate control server, in which case the control server controls the commands / responses between devices in the content streaming system.

[0061] A streaming server can receive content from media storage devices and / or encoding devices. For example, when receiving content from an encoding device, the content can be received in real time. In this case, the streaming server can store the bitstream for a certain period of time to provide a smooth streaming service.

[0062] Decoding devices can decode video / images by performing a series of processes corresponding to the operations of encoding devices, such as dequantization, inverse transform, and prediction.

[0063] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.

[0064] Figure 2 This diagram schematically illustrates the configuration of a video / image encoding apparatus to which embodiments of the present disclosure may be applied. In the following, the encoding apparatus may include image encoding apparatus and / or video encoding apparatus.

[0065] Reference Figure 2The encoding device 200 may include an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor and an intra-frame predictor. The residual processor 230 may include a transformer 232, a quantizer 233, a dequantizer 234, and an inverse transformer 235. The residual processor 230 may also include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstruction block generator. According to embodiments, the image segmenter 210, predictor 220, residual processor 230, entropy encoder 240, adder 250, and filter 260 may be configured as at least one hardware component (e.g., an encoder chipset or processor). The memory 270 may include a decoded picture buffer (DPB) or may be configured as a digital storage medium. The hardware component may also include the memory 270 as an internal / external component.

[0066] Image segmenter 210 can segment an input image (or picture or frame) input to encoding device 200 into one or more processing units. For example, a processing unit may be referred to as a coding unit (CU). In this case, coding units can be recursively segmented from coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree-binary-tritree (QTBTTT) structure. For example, a single coding unit can be segmented into multiple coding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, a quadtree structure can be applied first, and a binary tree structure and / or a ternary structure can be applied later. Alternatively, a binary tree structure can be applied first. The encoding process according to this disclosure can be performed based on the final coding unit that is no longer segmented. In this case, the maximum coding unit can be used as the final coding unit based on the encoding efficiency according to the image characteristics, or if necessary, the coding unit can be recursively segmented into deeper coding units, thereby using the coding unit with the optimal size as the final coding unit. Here, the encoding process may include prediction, transformation, and reconstruction processes, which will be described below. In another example, the processing unit may also include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit can be divided or segmented from the final encoding unit described above. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving the residual signal from the transform coefficients.

[0067] Depending on the context, the term "unit" may be used interchangeably with the terms "area" or "region". Generally, an M×N block can represent an array of samples or transform coefficients arranged in M ​​rows and N columns. Samples can typically represent pixels or pixel values, and can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component. "Sample" can be used as a term corresponding to a pixel or cell in a single frame (or image).

[0068] Encoding device 200 generates a residual signal (residual signal, residual block, or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the predictor from the input image signal (original block or original sample array), and sends the generated residual signal to converter 232. In this case, as shown, the component in encoder 200 used to subtract the prediction signal (prediction block or prediction sample array) from the input image signal (original block or original sample array) can be referred to as subtractor 231. The predictor can perform prediction on the processing target block (hereinafter referred to as the current block) and can generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The predictor can generate various information about the prediction, such as prediction mode information, and can send the generated information to entropy encoder 240, as described below in the description of each prediction mode. The information about the prediction can be encoded by entropy encoder 240 and output as a bitstream.

[0069] An intra-frame predictor can refer to samples within the current frame to predict the current block. The referenced samples can be located as neighboring (adjacent to) the current block, or, depending on the prediction mode, as distant from the current block. In intra-frame prediction, prediction modes can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC modes and planar modes. Depending on the granularity of the prediction direction, directional modes can include, for example, 33 or 65 directional prediction modes. However, this example is for illustration only, and more or fewer directional prediction modes can be used depending on the configuration. The intra-frame predictor can determine the prediction mode applied to the current block based on the prediction modes applied to neighboring blocks.

[0070] Inter-frame predictors can derive predicted blocks for the current block based on reference blocks (reference sample arrays) specified by motion vectors on a reference frame. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted based on the correlation between motion information between neighboring blocks and the current block, at the block, sub-block, or sample level. Motion information can include motion vectors and reference frame indices. Motion information can also include inter-frame prediction direction (L0 prediction, L1 prediction, and Bi prediction) information. In inter-frame prediction, neighboring blocks can include spatially adjacent blocks existing within the current frame and temporally adjacent blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally adjacent block can be the same or different. A temporally adjacent block can be referred to as a co-located reference block or a co-located CU (colCU), and a reference frame including a temporally adjacent block can also be referred to as a co-located frame (colPic). For example, the inter-frame predictor can configure a motion information candidate list based on neighboring blocks and can generate information indicating candidates for deriving the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip and merge modes, the inter-frame predictor can use motion information about neighboring blocks as motion information about the current block. In skip mode, unlike merge mode, residual signals may not be sent. In motion vector prediction (MVP) mode, the motion vector of the current block can be indicated by using motion vectors from neighboring blocks as motion vector predictors and signaling the motion vector difference.

[0071] Predictor 220 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also simultaneous intra-frame prediction and inter-frame prediction, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for predicting blocks. The IBC prediction mode or palette mode can be used for, for example, screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame based on the block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0072] The predicted signal generated by predictor 220 can be used to generate a reconstructed signal or a residual signal. Transformer 232 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graphical-Based Transform (GBT), or Conditional Nonlinear Transform (CNT).

[0073] Quantizer 233 can quantize the transform coefficients and send them to entropy encoder 240, which can encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal as a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 233 can rearrange the block-form quantized transform coefficients into a one-dimensional vector based on the coefficient scan order, and can generate information about the transform coefficients based on the one-dimensional vector form of the quantized transform coefficients. Entropy encoder 240 can perform various encoding methods, such as exponential Golomb, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). Entropy encoder 240 can encode information required for video / image reconstruction (e.g., values ​​of syntax elements) in addition to the quantized transform coefficients, either together or separately. Encoded information (e.g., encoded video / image information) can be sent or stored as a bitstream based on the Network Abstraction Layer (NAL). The video / image information can also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. In this disclosure, information and / or syntax elements transmitted from the encoding device / notified by signal to the decoding device may be included in the video / image information. The video / image information may be encoded by the aforementioned encoding process and included in a bitstream. The bitstream may be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, and SSD. A transmitter (not shown) and / or memory (not shown) for transmitting and / or storing signals output from the entropy encoder 240 may be configured as internal / external components of the encoding device 200, or the transmitter may be included in the entropy encoder 240.

[0074] The quantized transform coefficients output from quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients using dequantizer 234 and inverse transform unit 235. Adder 250 can add the reconstructed residual signal to the prediction signal output from the predictor to generate a reconstructed signal (reconstructed frame, reconstructed block, or reconstructed sample array). When no residual is available for processing the target block (e.g., when a skip mode is applied), the prediction block can be used as a reconstructed block. Adder 250 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current frame, or for inter-frame prediction of the next frame after filtering, as described below.

[0075] Luminance Mapping and Chroma Scaling (LMCS) can be applied to image encoding and / or reconstruction processing.

[0076] Filter 260 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 260 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and the modified reconstructed image can be stored in memory 270, specifically in the DPB of memory 270. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, and bilateral filtering. Filter 260 can generate filtering-related information and can send the generated information to entropy encoder 240. The filtering-related information can be encoded by entropy encoder 240 and output as a bitstream.

[0077] The modified reconstructed frame sent to memory 270 can be used as a reference frame in the inter-frame predictor. When inter-frame prediction is applied via the modified reconstructed frame, the encoding device can avoid prediction mismatch between the encoding device 200 and the decoding device, and can improve encoding efficiency.

[0078] The DPB of memory 270 can store modified reconstructed images for use as reference images in the inter-frame predictor. Memory 270 can store motion information about blocks in the current frame from which motion information is derived (or encoded) and / or about blocks in already reconstructed frames. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 270 can store reconstructed samples of reconstructed blocks in the current frame and can send these reconstructed samples to the intra-frame predictor.

[0079] Figure 3This diagram schematically illustrates the configuration of a video / image decoding apparatus to which embodiments of the present disclosure may be applied. Hereinafter, the decoding apparatus may include an image decoding apparatus and / or a video decoding apparatus.

[0080] Reference Figure 3 The decoding device 300 may include an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor and an intra-frame predictor. The residual processor 320 may include a dequantizer 321 and an inverse transformer 322. According to an embodiment, the entropy decoder 310, residual processor 320, predictor 330, adder 340, and filter 350 may be configured as a single hardware component (e.g., a decoder chipset or processor). The memory 360 may include a decoded picture buffer (DPB) and may be configured as a digital storage medium. The hardware component may also include the memory 360 as an internal / external component.

[0081] When the input includes a bitstream containing video / image information, the decoding device 300 can determine the appropriate response based on the input data. Figure 2 The encoding device processes video / image information to reconstruct the image. For example, the decoding device 300 can deduce units / blocks based on block segmentation information obtained from the bitstream. The decoding device 300 can perform decoding using processing units applied to the encoding device. Therefore, the processing unit used for decoding can be, for example, an encoding unit, and the encoding unit can be segmented from the encoding tree unit or the maximum encoding unit according to a quadtree structure, binary tree structure, and / or ternary tree structure. One or more transform units can be derived from the encoding unit. The reconstructed image signal decoded and output by the decoding device 300 can be reproduced via a reproduction device.

[0082] Decoding device 300 can receive signals output from encoding device in the form of a bitstream, and the received signals can be decoded by entropy decoder 310. For example, entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). Furthermore, the video / image information may also include general constraint information. Decoding device can further decode the picture based on the information about the parameter sets and / or general constraint information. In this disclosure, the information and / or syntax elements transmitted / received by signals, as described below, can be decoded through a decoding process and can be obtained from the bitstream. For example, entropy decoder 310 can decode the information in the bitstream based on an encoding method (such as exponential Golomb coding, CAVLC, or CABAC) and can output the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients for the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream. It can determine a context model using information about the target syntax element and decoding information about adjacent and target blocks, or information about symbols / bins decoded in previous stages. It can generate symbols corresponding to the values ​​of each syntax element by predicting the occurrence probability of bins based on the determined context model and performing arithmetic decoding on the bins. Here, after determining the context model, the CABAC entropy decoding method can update the context model using information about the decoded symbols / bins for the next symbol / bin. Prediction-related information from the information decoded by the entropy decoder 310 can be provided to the predictor 330, and the residual values ​​obtained through entropy decoding in the entropy decoder 310 (i.e., quantized transform coefficients and related parameter information) can be input to the residual processor 320. The residual processor 320 can derive residual signals (residual blocks, residual samples, or residual sample arrays). Filter-related information from the information decoded by the entropy decoder 310 can be provided to the filter 350. A receiver (not shown) for receiving signals output from the encoding device can also be configured as an internal / external element of the decoding device 300, or the receiver can be a component of the entropy decoder 310. The decoding device according to this disclosure can be referred to as a video / image / picture decoding device and can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoder 310, and the sample decoder may include at least one of a dequantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, and a predictor 330.

[0083] Dequantizer 321 can dequantize the quantized transform coefficients to output transform coefficients. Dequantizer 321 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the reordering can be performed based on the coefficient scan order performed in the encoding device. Dequantizer 321 can use quantization parameters (e.g., quantization step size information) to perform dequantization on the quantized transform coefficients and obtain the transform coefficients.

[0084] The inverse transformer 322 performs an inverse transformation on the transformation coefficients to obtain the residual signal (residual block or residual sample array).

[0085] The predictor can perform predictions for the current block and generate prediction blocks that include prediction samples of the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on prediction-related information output from the entropy decoder 310, and determine a specific intra-frame / inter-frame prediction mode.

[0086] Predictor 330 can generate a prediction signal based on various prediction methods described below. For example, the predictor can apply not only intra-frame prediction or inter-frame prediction to the prediction of a block, but also simultaneous intra-frame prediction and inter-frame prediction, which can be referred to as combined inter-frame and intra-frame prediction (CIIP). Furthermore, the predictor can be based on an intra-block copy (IBC) prediction mode or a palette mode for predicting blocks. The IBC prediction mode or palette mode can be used for, for example, screen content coding (SCC). IBC essentially performs prediction within the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame based on the block vector. That is, IBC can utilize at least one of the inter-frame prediction techniques described in this disclosure.

[0087] An intra-frame predictor can refer to samples within the current frame to predict the current block. The referenced samples can be located as neighboring (adjacent to) the current block, or, depending on the prediction mode, as distant from the current block. In intra-frame prediction, the prediction mode can include multiple non-directional modes and multiple directional modes. The intra-frame predictor can determine the prediction mode applied to the current block based on the prediction modes applied to neighboring blocks.

[0088] An inter-frame predictor can deduce a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. Here, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, and Bi prediction) information. In inter-frame prediction, neighboring blocks may include spatially adjacent blocks existing within the current frame and temporally adjacent blocks existing in the reference frame. For example, inter-frame predictor 332 can configure a motion information candidate list based on neighboring blocks and can deduce the motion vector and / or reference frame index of the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and information about the prediction may include information indicating the inter-frame prediction mode for the current block.

[0089] Adder 340 can add the obtained residual signal to the prediction signal (prediction block or prediction sample array) output from predictor 330 to generate a reconstruction signal (reconstructed image, reconstruction block, or reconstruction sample array). When there is no residual for the processing target block, such as when a skip mode is applied, the prediction block can be used as a reconstruction block.

[0090] Adder 340 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next processing target block in the current frame, or it can be output after being filtered as described below, or used for inter-frame prediction of the next frame.

[0091] Luminance mapping and chroma scaling (LMCS) can be applied during the image decoding process.

[0092] Filter 350 can improve subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 350 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image, and can send the modified reconstructed image to memory 360, specifically to the DPB in memory 360. Various filtering methods can include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, and bilateral filtering.

[0093] The (modified) reconstructed frame stored in the DPB of memory 360 can be used as a reference frame in the inter-frame predictor. Memory 360 can store motion information about blocks from which motion information in the current frame is derived (or decoded) and / or about blocks in already reconstructed frames. The stored motion information can be sent to the inter-frame predictor to be used as motion information about spatially adjacent blocks or about temporally adjacent blocks. Memory 360 can store reconstructed samples of reconstructed blocks in the current frame and can send the reconstructed samples to the intra-frame predictor.

[0094] The embodiments described in this specification for the filter 260 and predictor 210 of the encoding device 200 can also be applied equivalently or accordingly to the filter 350 and predictor 330 of the decoding device 300.

[0095] As described above, prediction is performed during video encoding to improve compression efficiency. This prediction generates prediction blocks that include prediction samples for the current block, which is the target block for encoding. The prediction blocks consist of prediction samples in the spatial domain (or pixel domain). The prediction blocks are derived identically in both the encoding and decoding devices, and the encoding device can enhance image encoding efficiency by signaling information about the residuals between the original block and the prediction blocks (residual information) instead of the original sample values ​​of the original block to the decoding device. The decoding device can derive residual blocks including residual samples based on the residual information, generate reconstruction blocks including reconstructed samples by combining the residual blocks and the prediction blocks, and generate a reconstructed image including the reconstruction blocks.

[0096] Residual information can be generated through transformation and quantization processes. For example, an encoding device can derive a residual block between the original block and the prediction block, perform a transformation process on the residual samples (residual sample array) included in the residual block to derive transform coefficients, perform a quantization process on the transform coefficients to derive quantized transform coefficients, and signal the relevant residual information to the decoding device (via bitstream). Residual information may include information such as the values ​​and locations of the quantized transform coefficients, transform technique, transform kernel, and quantization parameters. The decoding device can perform dequantization / inverse transform processes based on the residual information to derive residual samples (or residual blocks). The decoding device can generate a reconstructed frame based on the prediction block and the residual block. The encoding device can perform dequantization / inverse transform on the quantized transform coefficients to derive residual blocks for reference in inter-frame prediction of subsequent frames, and can generate a reconstructed frame based on the residual blocks.

[0097] In this disclosure, at least one of quantization / dequantization and / or transformation / inverse transformation may be omitted. When quantization / dequantization is omitted, the transformation coefficients of the quantization may be referred to as transformation coefficients. When transformation / inverse transformation is omitted, the transformation coefficients may be referred to as coefficients or residual coefficients, or, for the sake of consistency in the expression, may still be referred to as transformation coefficients.

[0098] Furthermore, in this disclosure, the quantized transform coefficients and the transform coefficients can be referred to as transform coefficients and scaled transform coefficients, respectively. In this case, the residual information can include information about the transform coefficients, and this information can be signaled via a residual coding syntax. The transform coefficients can be derived based on the residual information (or the information about the transform coefficients), and the scaled transform coefficients can be derived by the inverse transform (scaling) of the transform coefficients. The residual samples can be derived based on the inverse transform (scaling) of the scaled transform coefficients. These details can be equivalently applied to or described in other parts of this disclosure.

[0099] Intra-frame prediction can refer to generating prediction samples for the current block based on reference samples within the frame to which the current block belongs (hereinafter, the current frame). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. Neighboring reference samples for the current block may include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, neighboring reference samples for the current block may include multiple rows of upper neighboring samples and multiple columns of left neighboring samples.

[0100] Some neighboring reference samples for the current block may not have been decoded yet, or may be unavailable. In this case, the decoder can configure the neighboring reference samples to be used for prediction by filling in or replacing unavailable samples with available samples.

[0101] When deriving neighboring reference samples, the predicted samples for the current block can be derived based on the neighboring reference samples and intra-prediction mode / type information. Here, the intra-prediction mode can indicate one of a non-directional prediction mode and a directional prediction mode representing the spatial correlation used for intra-prediction. The directional prediction mode can be referred to as the angular prediction mode, and the non-directional prediction mode can be referred to as the non-angular prediction mode. The intra-prediction type can indicate various prediction types used to perform intra-prediction. Intra-prediction types can include, for example, multiple reference lines (MRL), intra-segmentation (ISP), position-dependent intra-prediction (PDPC), matrix-weighted intra-prediction or matrix-based intra-prediction (MIP), cross-component linear model (CCLM), multi-model linear model (MMLM), decoder-side intra-mode derivation (DIMD), fusion of chroma intra-prediction modes, intra-template matching, fusion of template-based intra-mode derivation (TIMD), intra-prediction fusion, cross-component convolutional model (CCCM), cross-component prediction (CCP), and spatial geometric segmentation mode (SGPM). Depending on the circumstances, intra-prediction can be performed using intra-prediction modes and / or intra-prediction types.

[0102] Specifically, the intra-frame prediction process may include an intra-frame prediction mode / type determination operation, a reference sample derivation operation, and a prediction sample derivation operation based on the intra-frame prediction mode / type. Furthermore, post-filtering operations for the derived prediction samples may be performed as needed.

[0103] Figure 4 The intra-frame prediction process is illustrated.

[0104] Reference Figure 4 As described above, the intra-frame prediction process may include operations to determine the intra-frame prediction mode / type, operations to derive reference samples, and operations to perform intra-frame prediction (generate prediction samples). As described above, the intra-frame prediction process can be performed in both the encoding and decoding devices.

[0105] The encoding device determines the intra-frame prediction mode / type (S400). As described above, the encoding device may include an encoding device and / or a decoding device.

[0106] The encoding device can determine the intra prediction mode / type applicable to the current block from the various intra prediction modes / types described in this disclosure, and can generate prediction-related information. The prediction-related information may include intra prediction mode information indicating the intra prediction mode applied to the current block and / or intra prediction type information indicating the intra prediction type applied to the current block. The decoding device can determine the intra prediction mode / type applicable to the current block based on the prediction-related information.

[0107] For example, when applying intra-prediction, the intra-prediction mode to be applied to the current block can be determined based on the intra-prediction modes of neighboring blocks. For instance, the coding device can select one of the most probable mode (MPM) candidates from a list of most probable modes derived from the intra-prediction modes and / or additional candidate modes of the current block's neighboring blocks (e.g., left and / or top neighboring blocks) based on received index information, or it can select one of the remaining intra-prediction modes not included in the MPM candidates based on MPM residual information (remaining intra-prediction mode information). The MPM list can be configured to include or exclude planar modes as candidates.

[0108] The encoding device can be configured with a list of most probable modes (MPMs) for the current block. The MPM list can also be referred to as the MPM candidate list. Here, MPM can refer to a mode used in intra-frame predictive mode coding to improve coding efficiency by considering the similarity between the current block and neighboring blocks.

[0109] The encoding device can perform prediction based on various intra-prediction modes and determine the optimal intra-prediction mode based on rate distortion optimization (RDO). In this case, the encoding device can determine the optimal intra-prediction mode using MPM candidates configured in the MPM list, or it can use any intra-prediction mode other than those in the MPM list. Specifically, for example, when the intra-prediction type of the current block is not the normal intra-prediction type but a specific type (e.g., DIMD, TIMD, MRL, or ISP), the encoding device can determine the optimal intra-prediction mode by only considering the MPM candidates as intra-prediction mode candidates for the current block. That is, in this case, the intra-prediction mode for the current block can be determined only from the MPM candidates, and the MPM flag can be omitted from encoding / signaling. In this case, the decoding device can estimate the MPM flag as 1 without receiving the MPM flag separately.

[0110] Typically, when the intra-prediction mode for the current block is not a planar mode but one of the MPM candidates in the MPM list, the encoding device generates an MPM index (mpm idx) indicating one of the MPM candidates. When the intra-prediction mode for the current block is not even in the MPM list, the encoding device generates MPM remainder information (remaining intra-prediction mode information) indicating the same mode as the intra-prediction mode for the current block among the remaining intra-prediction modes (and planar modes) not included in the MPM list. MPM remainder information may include, for example, the intra_luma_mpm_remainder syntax element.

[0111] The decoding device obtains intra-prediction mode information from the bitstream. As described above, the intra-prediction mode information may include at least one of an MPM flag, an MPM index, and remaining MPM information (remaining intra-prediction mode information). The decoding device may configure an MPM list. The MPM list is configured to be the same as the MPM list configured by the encoding device. That is, the MPM list may include intra-prediction modes of adjacent blocks, or it may also include specific intra-prediction modes according to a predetermined method.

[0112] The decoding device can determine the intra-prediction mode for the current block based on the MPM list and intra-prediction mode information. For example, when the value of the MPM flag is 1, the decoding device can deduce the intra-prediction mode for the current block from the MPM candidates indicated by the MPM index in the MPM list.

[0113] In another example, when the value of the MPM flag is 0, the decoding device can deduce the intra-prediction mode indicated by the remaining intra-prediction mode information (which may be called the MPM remaining information) in the remaining intra-prediction modes as the intra-prediction mode for the current block.

[0114] The encoding device derives reference samples for the current block (S410). The reference samples may include adjacent reference samples of the current block. The adjacent reference samples of the current block may include H+W samples located to the left of the W×H current block, W+H samples located above the current block, and at least one sample adjacent to the upper left of the current block. Alternatively, the adjacent reference samples of the current block may include multiple row-adjacent samples and multiple column-adjacent samples.

[0115] The encoding device performs intra-frame prediction on the current block to derive prediction samples (S430). The encoding device can derive prediction samples based on the intra-frame prediction mode / type and reference samples. The encoding device can derive reference samples from the reference samples of the current block according to the intra-frame prediction mode of the current block, and can derive prediction samples of the current block based on the reference samples.

[0116] Intra-prediction-based coding processes can schematically include, for example, the following.

[0117] Figure 5 An example of a video / image coding method based on intra-frame prediction is shown.

[0118] Reference Figure 5The process can be as follows: S500 can be executed by the predictor of the encoding device, S505 can be executed by the residual processor of the encoding device, and S510 or S515 can be executed by the entropy encoder of the encoding device. Specifically, prediction-related information can be derived by the predictor and encoded by the entropy encoder. Residual information can be derived by the residual processor and encoded by the entropy encoder. Residual information is information about residual samples. Residual information may include information about the transform coefficients quantized for the residual samples. As mentioned above, transform coefficients can be derived from residual samples by the transformer of the encoding device, and quantized transform coefficients can be derived from transform coefficients by the quantizer. Information about quantized transform coefficients can be encoded in the entropy encoder through a residual encoding process.

[0119] The encoding device performs intra-prediction for the current block (S500). The encoding device derives the intra-prediction mode / type for the current block, derives reference samples for the current block, and generates prediction samples within the current block based on the intra-prediction mode / type and the reference samples. The processes for determining the intra-prediction mode / type, deriving adjacent reference samples, and generating prediction samples can be performed simultaneously, or one process can be performed before the other. The encoding device can determine the mode / type applicable to the current block from multiple intra-prediction modes / types. The encoding device can compare the RD costs for the intra-prediction modes / types and determine the optimal intra-prediction mode / type for the current block.

[0120] Encoding devices can perform a predictive sample filtering process. Predictive sample filtering can also be called post-filtering. Some or all of the predicted samples can be filtered through this process. In some cases, the predictive sample filtering process can be omitted.

[0121] The encoding device generates residual samples for the current block based on the predicted samples (S505). The encoding device can compare the predicted samples with the original samples of the current block based on the phase and derive the residual samples.

[0122] The encoding device can encode image / video information, including information about intra-frame prediction (prediction-related information) and / or information about residual samples (residual information) (S510 or S515). Prediction-related information may include information about the intra-frame prediction mode and information about the intra-frame prediction type. The encoding device can output the encoded image / video information as a bitstream. The output bitstream can be sent to the decoding device via a storage medium or network.

[0123] Residual information can include residual coding syntax elements. The coding device can derive the quantization transform coefficients by transforming / quantizing the residual samples. The residual information can include information about the quantization transform coefficients.

[0124] As described above, the encoding device can generate a reconstructed frame (including reconstructed samples and reconstructed blocks). To do this, the encoding device can derive (modified) residual samples by performing dequantization / inverse transform on the quantized transform coefficients. The reason for transforming / quantizing the residual samples and then performing dequantization / inverse transform on them is, as mentioned above, to derive the same residual samples as those derived in the decoding device. The encoding device can generate a reconstructed block that includes reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. A reconstructed frame for the current frame can be generated based on the reconstructed blocks. As mentioned above, in-loop filtering processes, etc., can be further applied to the reconstructed frame.

[0125] Decoding devices can perform operations corresponding to those performed by encoding devices. Intra-frame prediction-based video / image decoding processes may include, for example, the following.

[0126] Figure 6 An example of a video / image decoding method based on intra-frame prediction is shown.

[0127] Reference Figure 6 S600 can be executed by the entropy decoder of the decoding device, S610 can be executed by the predictor of the decoding device, S615 can be executed by the residual processor of the decoding device, and S620 can be executed by the adder or reconstructor of the decoding device.

[0128] Specifically, the decoding device obtains image / video information from the bitstream (S600). The image / video information may include prediction-related information and / or residual information.

[0129] The decoding device performs intra-frame prediction based on prediction-related information (S610). The decoding device can derive an intra-frame prediction mode / type for the current block based on the prediction-related information, derive a reference sample for the current block, and generate prediction samples within the current block based on the intra-frame prediction mode / type and the reference sample. In this case, the decoding device can perform a prediction sample filtering process. Prediction sample filtering can be referred to as post-filtering. Some or all of the prediction samples can be filtered by the prediction sample filtering process. In some cases, the prediction sample filtering process can be omitted.

[0130] The decoding device performs residual processing based on the residual information (S615). The decoding device can derive residual samples for the current block based on the residual information. Specifically, the dequantizer of the residual processor can derive the transform coefficients by dequantizing the quantized transform coefficients derived based on the residual information, and the inverse transformer of the residual processor can derive residual samples for the current block by performing an inverse transform on the transform coefficients.

[0131] The decoding device generates reconstructed blocks / frames (S620). The decoding device can generate reconstructed samples for the current block based on predicted samples and / or residual samples, and can derive reconstructed blocks including the reconstructed samples. A reconstructed frame for the current frame can be generated based on the reconstructed blocks. As described above, in-loop filtering processes, etc., can be further applied to the reconstructed frame.

[0132] Information about the intra-prediction mode and / or the intra-prediction type can be encoded / decoded using the binarylization and encoding methods described in this disclosure. For example, information about the intra-prediction mode and / or the intra-prediction type can be binarylized using fixed-length binarylization, truncated Rice binarylization, or truncated unary binarylization. For example, information about the intra-prediction mode and / or the intra-prediction type can be encoded / decoded using entropy coding (e.g., CABAC or CAVLC).

[0133] Figure 7 An example of a directional intra-frame prediction mode is shown. Figure 7 An example can be shown that includes 65 directional intra-frame prediction modes.

[0134] Reference Figure 7 Directional intra-prediction modes can include modes #2 through #65. In this case, mode #50 could represent a vertical intra-prediction mode, and mode #18 could represent a horizontal intra-prediction mode. However, this example is for illustration, and the number and type of candidate intra-prediction modes can vary. One or more non-directional intra-prediction modes can be considered; for example, mode #0 could represent an intra-plane prediction mode (plane mode), and mode #1 could represent an intra-DC prediction mode (DC mode).

[0135] Intra-frame prediction modes can be assigned mode numbers, as shown in the table below, for example.

[0136] [Table 1]

[0137] According to this disclosure, various intra-prediction modes can be used for intra-prediction of the current block. For example, intra-prediction modes can be derived based on decoder-side intra-mode derivation (DIMD) techniques. DIMD can be referred to as DIMD type.

[0138] Figure 8 An example of template-based HoG computation in DIMD is shown.

[0139] Reference Figure 8In DIMD, a histogram of gradients (HoG) can be computed using a template that includes neighboring samples of the current block. For example, the HoG can be computed using horizontal / vertical Sobel filters based on a template with n (e.g., 3) neighboring sample rows. In this case, both horizontal and vertical Sobel filters can be applied to compute the HoG. When the templates are located in different CTUs, the Sobel filter may not be applied to the upper CTU boundary, or the DIMD technique may not be applied to the current block.

[0140] Different filters can be used for HoG computation based on the block size. For example, when the block size is 4×4, 4×8, or 8×4, a 2×2 kernel filter can be used instead of a 3×3 Sobel filter for HoG computation.

[0141] An adaptive number of (neighboring) reference samples can be used for HoG calculations in DIMD. In this case, the number of sample rows in the DIMD template can be four or more.

[0142] Among candidate intra-prediction modes, n intra-prediction modes with the highest histogram values ​​can be selected / extracted through HoG computation. In this case, the n intra-prediction modes can be used to derive the prediction block for the current block. Furthermore, in this case, n intra-prediction modes and non-directional modes (e.g., planar or DC modes) can be used to derive the prediction block. In this case, predictors can be derived for the corresponding intra-prediction modes, and the prediction block can be derived by performing a weighted sum / weighted average of the predictors. That is, through prediction fusion, the predictors of the n (e.g., 5) selected / extracted intra-prediction modes can be fused with the predictor of the planar mode.

[0143] For example, n can be 5. In this case, five derived patterns with the highest HoG can be derived. The derived patterns can be called DIMD-based patterns or DIMD-derived patterns. For example, the number of n can be determined differently based on the block size. For example, when the block's W... When H is 128 or greater, n can be 7; otherwise, n can be 5.

[0144] For HoG calculations in DIMD, specific block vectors can be used. For example, block vectors can be derived based on the motion vectors or block vectors of neighboring blocks. A reference region at the shifted position can be derived using specific block vectors based on the current block's position, and the HoG can be calculated using (reconstructed) reference samples within the reference region. In this case, the template can be located within the reference region, or it can be positioned above / to the left of a neighboring reference region. The size of the reference region can be the same as the current block's size. Alternatively, the size of the reference region can differ from the current block's size. For example, the width of the reference region can be half the width of the current block, and / or the height of the reference region can be half the height of the current block. In this case, the template size can also vary within the range of the width and height of the reference region.

[0145] The n intra-frame modes derived through DIMD can be referred to as DIMD-derived modes or DIMD-based modes. DIMD-derived modes can be included as candidates in the MPM list (e.g., the PMPM list).

[0146] When a DIMD technique (or type) is applied to the current block, the first intra-mode derived from DIMD can be stored as the intra-mode for the target block and can be referenced when configuring the MPM list for subsequent blocks.

[0147] When a DIMD technique (or type) is applied to the current block, the first intra-frame mode derived from DIMD can be referred to as the mode used to select the transform kernel (or transform set).

[0148] For weighted and / or weighted average or predictive fusion, weights between predictors are required. According to embodiments of this disclosure, lookup tables (LUTs) can be used to derive these weights. While existing full HoG calculations can be used to compute dimdMode... i However, Habove and Hleft can be calculated separately to calculate dimdMode. i Is it highly dependent on the top template or the left template, i.e., dimdMode? i Is it position-dependent?

[0149] When dimMode i When it is not position-dependent, the weight (wDimd) i (wPlanar) can be determined based on the HoG size, as in traditional methods.

[0150] When dimMode i When the problem is position-dependent, sample-based mixing can be applied. In this case, weights can be applied differently based on the samples.

[0151] When the upper HoG (e.g., Habove) is equal to or greater than twice the left HoG (e.g., Hleft), the weights can be based on the following equation.

[0152] [Equation 1]

[0153] here, It can represent the difference between the upper HoG and the left HoG, or the absolute value of that difference.

[0154] When the left HoG (e.g., Hleft) is equal to or greater than twice the upper HoG (e.g., Habove), the weights can be based on the following equation.

[0155] [Equation 2]

[0156] here, It can represent the difference between the left HoG and the upper HoG, or the absolute value of that difference.

[0157] For DIMD, an additional W column can be used when the upper-right adjacent reference sample is available, and an additional H row can be used when the lower-left adjacent reference sample is available. That is, the template region including adjacent (reconstructed) samples used for HOG calculation can be based on the availability of adjacent (reconstructed) samples.

[0158] When deriving n intra-frame (prediction) modes in DIMD, n intra-frame mode predictors and planar predictors can be combined. However, when deriving only one intra-frame mode in DIMD, only the intra-frame mode can be used. Alternatively, even when deriving only one intra-frame mode in DIMD, the intra-frame mode can be combined with a planar mode predictor. For example, when the HoG value of the intra-frame mode with the highest HoG in DIMD is equal to or greater than a threshold, only one intra-frame mode can be derived. In another example, when the first HoG value of the intra-frame mode with the highest HoG in DIMD is a specific value or greater than the second HoG value of the intra-frame mode with the second highest HoG, only one intra-frame mode can be derived. Furthermore, among the modes derived in DIMD, n intra-frame prediction modes with HoG equal to or greater than a specific threshold can be selected.

[0159] For a block that has applied DIMD, the derivation pattern of the first DIMD can be stored as the intra-prediction pattern for the current block. The stored pattern can be referenced as the patterns of neighboring blocks referenced for deriving the intra-prediction pattern for subsequent blocks and / or as the pattern for selecting the transform kernel (or transform set).

[0160] There may be situations where the pattern derived by the first DIMD may differ from the actual prediction pattern of the current block (or the best prediction pattern with the highest correlation to the current block). Therefore, in addition to the first intra-prediction pattern derived by DIMD, it is necessary to investigate patterns that can be stored as intra-prediction patterns for the current block. For example, when the HoG of the pattern derived by the first DIMD is greater than a certain threshold, the first pattern can be stored as an intra-prediction pattern for the current block; otherwise, a plane or DC pattern can be stored as an intra-prediction pattern for the current block. Alternatively, by comparing the first predictor (first prediction sample) of the current block generated by DIMD with the second predictor (second prediction sample) generated by the pattern derived by the first DIMD, when the difference between the two is within a certain SAD / SATD, the pattern derived by the first DIMD can be stored as an intra-prediction pattern for the current block; otherwise, a plane or DC pattern can be stored as an intra-prediction pattern for the current block.

[0161] In another example, a candidate list can be configured using m intra-prediction modes derived from DIMD, and any mode in the candidate list can be stored. Here, m can be equal to n, or m can be 2 or 3. In this case, the first predictor (first prediction sample) of the current block generated by DIMD can be compared with the predictors based on the corresponding m intra-prediction modes, and the intra-prediction mode with the lowest SAD / SATD among the m intra-prediction modes can be stored as the intra-prediction mode for the current block.

[0162] Alternatively, for the first predictor (first prediction sample) derived by the DIMD technique, the first intra-prediction mode derived by the DIMD technique (i.e., the intra-prediction mode with the highest HoG based on the Sobel filter re-derived for the prediction sample (the first intra-prediction mode)) can be stored as the intra-prediction mode for the current block. In this case, the first intra-prediction mode derived from the template of the current block can be different from the first intra-prediction mode derived from the predictor of the current block.

[0163] For DIMD, DIMD-related information can be communicated using signals. DIMD-related information may include at least one of the following: DIMD enable flag, DIMD application flag, and / or DIMD flag.

[0164] The DIMD enable flag (dimd_enable_flag) can be signaled via high-level syntax (e.g., SPS). When the DIMD enable flag is 1, the DIMD apply flag (dimd_applied_flag) and / or the DIMD flag (dimd_flag) can be signaled.

[0165] The DIMD application flag can be signaled at the PPS, PH, SH, or CTU levels. Even when DIMD is available, it can be enabled / disabled at the PH, SH, or CTU levels via the DIMD application flag. The DIMD flag can be signaled at the CU, PU, ​​or TU levels. The DIMD flag can be signaled before the MPM flag (or PMPM flag).

[0166] DIMD-related information can be communicated using signals, for example, as shown below.

[0167] The DIMD flag (dimd_flag) can be signaled before the PMPM flag (pmpm_flag). In this case, when the value of the DIMD flag is 0, the PMPM flag can be signaled, and when the value of the DIMD flag is 1, the signaling of the PMPM flag can be omitted, as can be represented, for example, in the table below.

[0168] [Table 2]

[0169] The DIMD flag (dimd_flag) can be signaled before the reference line index (intra_luma_ref_idx) (in the parsing order). When multiple adjacent reference sample lines are used for intra-prediction of the current block, the reference line index indicates one or more of these adjacent reference sample lines. The reference line index can be signaled when the DIMD flag is not 1. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample line index indicates 0, as can be expressed, for example, as shown in the table below.

[0170] [Table 3]

[0171] The DIMD flag (dimd_flag) can be signaled after the reference row index (intra_luma_ref_idx) (in parsing order). The DIMD flag can be signaled when the reference row index is 0. The PMPM flag can be signaled when the DIMD flag is not 1 and the reference sample row index indicates 0, as can be expressed, for example, as shown in the table below.

[0172] [Table 4]

[0173] As described above, the MPM list can be configured to efficiently signal the intra-prediction mode for the current block. In the method according to embodiments of this disclosure, multiple MPM lists can be configured. The multiple MPM lists may include a first MPM list and a second MPM list. The first MPM list may be referred to as the primary MPM (PMPM) list, and the second MPM list may be referred to as the secondary MPM (SMPM) list.

[0174] The first and second MPM lists can be configured according to predetermined criteria. For example, a generic MPM list containing k candidates can be configured first, the first n candidates in the generic MPM list can be included in the first MPM list, and the remaining m candidates can be included in the second MPM list. In one example, k can be 22, n can be 6, and / or m can be 16. For example, the first candidate in the generic MPM list (the candidate of the first entry) can always be a planar pattern. In another example, where the planar pattern is signaled based on a separate planar flag (or not_planar_flag), k can be 21, n can be 5, and / or m can be 16.

[0175] At least one of the remaining candidates, excluding the planar pattern, can be derived from adjacent blocks.

[0176] Figure 9 An example of adjacent blocks used to derive the MPM list is shown.

[0177] Reference Figure 9 The adjacent block may include at least one of the following: the left adjacent block (L), the lower left adjacent block (BL), the upper adjacent block (A), the upper right adjacent block (AR), and / or the lower left adjacent block (AL) of the current block.

[0178] Furthermore, the remaining candidates may include DIMD-based patterns (DIMD-derived intra-prediction patterns). For example, intra-prediction patterns derived from neighboring blocks and DIMD-based patterns (hereinafter referred to as sorted intra-prediction patterns) can be sorted in ascending order of SAD cost based on SAD cost (hereinafter referred to as sorted intra-prediction patterns), and then added to the general MPM list or the first MPM list. In this case, only up to a predetermined number (p) of sorted intra-prediction patterns can be extracted and added to the general MPM list or the first MPM list. Here, the SAD cost can be calculated based on the reconstructed samples of the template of the current block and the predictors derived by applying the candidate patterns to the template. Among the sorted intra-prediction patterns, directional prediction patterns can be referred to as sorted directional patterns. For example, p can be 5 or 6. Alternatively, p can be 7 or 8. Planar patterns and / or DC patterns can be excluded from the DIMD-based patterns. This also applies below. DIMD-based patterns can be referred to as DIMD-derived patterns.

[0179] For example, sorted prediction patterns, patterns obtained by adding offsets to (extracted) sorted orientation patterns or subtracting offsets from said (extracted) sorted orientation patterns, and predetermined default patterns can be added until all k entries are filled.

[0180] In another example, the sorting process can be omitted for some or all of the DIMD-based patterns. This is because DIMD-based patterns are highly correlated with the current block. Therefore, for some or all of the DIMD-based patterns, the sorting process can be omitted, and the sorting process can be preferentially assigned to the front of the MPM list (e.g., a general MPM list or a first MPM list). In this case, planar patterns can be placed first, and some or all of the DIMD-based patterns can be assigned after the planar patterns. Alternatively, when planar patterns are not included in the MPM list, some or all of the DIMD-based patterns can be assigned first. Alternatively, DIMD-based patterns can be compared with intra-prediction patterns derived from neighboring blocks, and overlapping patterns can be preferentially assigned to the front of the MPM list because these patterns are highly correlated with the current block. That is, the pruning process and the classification process can be merged. For example, conventional methods require redundant pruning processes and separate sorting processes to remove redundant intra-prediction patterns derived from neighboring blocks (group 1) and DIMD-based patterns (group 2), while this method can omit the sorting process by prioritizing redundant patterns while performing redundancy checks.

[0181] Figure 10 Intra-prediction modes derived from neighboring blocks and DIMD-based modes are shown.

[0182] Reference Figure 10 If patterns B and C overlap between two groups, then patterns B and C can be assigned to the MPM list with priority. In this case, for example, patterns B and C can be assigned at the beginning of the MPM list, or assigned immediately after the planar patterns.

[0183] In this case, for example, the priority between pattern B and pattern C can be determined based on the priority in the DIMD-based patterns. Since the priority of DIMD-based patterns is already derived based on HoG, the priority can be determined without a separate SAD-based classification process.

[0184] Intra-prediction modes used to encode previous blocks within a specific region can be stored and reused based on history; this can be called history-based intra-prediction modes (HIPMs). HIPMs can be represented using various terms, such as history-based MPMs (HMPMs) or history-based candidate modes. The specific region can include, for example, CTUs, CTU rows, multiple CTU rows, slices, and tiles. Multiple CTU rows can include, for example, the nth CTU row and the (n-1)th CTU row where the current block is located. In this case, the HIPM list can be reordered based on the frequency of the intra-prediction modes. That is, intra-prediction modes with a high occurrence frequency within a specific region have a higher priority than intra-prediction modes with a low occurrence frequency and can be assigned to an earlier position in the list. When assigned to an earlier position in the list, a lower index value can be used to indicate the mode.

[0185] Figure 11 The intra-prediction mode of the previous block within a specific region is shown.

[0186] Reference Figure 11 When a specific pattern (e.g., pattern #50) has the highest frequency, the HIPM list can be configured to give that pattern high priority. This list can be referred to as a buffer. In this case, for example, pattern #28 has the next highest frequency and can therefore be assigned to the position after pattern #50.

[0187] When configuring the HIPM list, the intra-prediction mode of previous blocks within a specific region may be unavailable, for example, when previous blocks are coded based on inter-frame prediction (inter-frame coding). In this case, the intra-prediction mode of the previous blocks can be treated as planar mode or DC mode. Alternatively, the intra-prediction mode of previous blocks can be omitted when configuring the HIPM list.

[0188] The HIPM list can be used to configure the MPM list. For example, the MPM list can be configured based on x candidates from the HIPM list according to priority. That is, the MPM list can include x candidates from the HIPM list according to priority. Alternatively, the candidates for the MPM list can be derived based on candidates for intra-prediction modes derived from neighboring blocks (first group) and DIMD-based modes (second group) and / or candidates from the HIPM list (third group). In this case, a pruning and sorting process can be performed, and at most the top p items can be extracted through the sorting process.

[0189] As described above, a general MPM list, a first MPM list, and / or a second MPM list can exist. The second MPM list can be divided into four groups. In this case, for example, the PMPM flag can be signaled first, and when the PMPM flag is 1, the PMPM index can be signaled. The PMPM flag can indicate whether the intra-prediction mode for the current block exists in the first MPM list. For example, the GMPM flag can be signaled before the PMPM flag. In this case, when the GMPM flag is 1, the PMPM flag and SMPM flag (described below) can be signaled. When the PMPM flag is 0, the SMPM flag can be signaled first, and when the SMPM flag is 1, the group index (SMPM group index) for the second MPM list can be parsed / signed first, and the mode index (SMPM mode index) can be signaled / parsed subsequently. Information about the intra-prediction mode (e.g., GMPM flag, PMPM flag, PMPM index, SMPM flag, SMPM group index, and / or SMPM mode index) can be signaled using CU syntax. When the GMPM flag is 0, the PMPM and SMPM flags can be deduced to be 0 without signaling / coding. The GMPM flag can also be simply referred to as the MPM flag. The GMPM flag can be omitted depending on the situation.

[0190] For example, information about intra-frame prediction can be signaled as follows. This information can include one or more syntax elements. The same applies below.

[0191] [Table 5]

[0192] For example, smpm_group_idx can be binary-coded with a fixed length, and smpm_mode_idx can also be binary-coded with a fixed length.

[0193] In another example, smpm_group_idx can be truncated to rice (or truncated unary) binary. In this case, smpm_mode_idx can be fixed-length binary. Alternatively, the binary method for smpm_mode_idx can be determined based on the smpm_group_idx value. For example, when the smpm_group_idx value is 0 or 1, smpm_mode_idx can be truncated to rice (or truncated unary) binary, while in other cases, it can be fixed-length binary. smpm_mode_idx can be encoded based on a context model, in which case the context model for smpm_mode_idx can be determined differently based on the size of the current block and / or the smpm_group_idx value. For example, the context model can be indicated based on the context index increment (ctxInc), and the ctxInc for the bin of smpm_mode_idx can be determined differently, for example, based on the smpm_group_idx value as shown below. As described below, smpm_group_flag can be used to replace smpm_group_idx.

[0194] [Table 6]

[0195] [Table 7]

[0196] The second MPM list can be divided into two groups. In this case, the number of candidates in the first group can be equal to or different from the number of candidates in the second group. For example, the number of candidates in the second MPM list can be 16, 12, or 8. In this case, the first group can include 4 or 8 candidates, while the second group can include 8 candidates.

[0197] For example, `smpm_group_flag` can be used instead of `smpm_group_idx`. `smpm_group_flag` indicates whether the intra-prediction mode for the current block is included in the second candidate group of the second MPM list. The binary representation method for `smpm_mode_idx` can be determined based on the `smpm_group_flag` value. For example, when the `smpm_group_flag` value indicates the first group (e.g., a value of 1), `smpm_mode_idx` can be truncated to Rice (or truncated unary) binary representation, and in other cases, `smpm_mode_idx` can be fixed-length binary representation.

[0198] In this case, for example, information about intra-frame prediction can be communicated using signals as follows.

[0199] [Table 8]

[0200] You can use smpm_idc instead of smpm_flag and / or smpm_group_flag (or smpm_group_idx).

[0201] In this case, for example, information about intra-frame prediction can be communicated using signals as follows.

[0202] [Table 9]

[0203] `smpm_idc` indicates whether the intra-prediction mode for the current block is included in the second MPM list and, if included, which group the intra-prediction mode belongs to. `smpm_idc` can be truncated to Rice (or truncated unary) binary form.

[0204] The following table shows the binary representation of smpm_idc.

[0205] [Table 10]

[0206] smpm_mode_idx can be referred to as smpm_idx. Smpm_idx can be truncated into a Rice (or truncated unary) binary.

[0207] The following table shows the binary representation of smpm_idx.

[0208] [Table 11]

[0209] Alternatively, smpm_idx can be binary based on smpm groups. For example, smpm_idx can be binary based on the smpm_idc value. For instance, when smpm_idc is 1, smpm_idx can be truncated rice (or truncated unary) binary, and when smpm_idc is 2, smpm_idx can be fixed-length binary. In this case, for example, the number of candidates in the first group can be 4, and the number of candidates in the second group can be 8. Therefore, in the worst case, the maximum number of binary bins can be matched.

[0210] [Table 12]

[0211] [Table 13]

[0212] As mentioned above, considering the best case, the first group in the second MPM list can be binary-coded with variable length because it is highly relevant to the current block, and considering the worst case, the second group in the second MPM list can be binary-coded with fixed length because it is relatively less relevant to the current block.

[0213] Context-based encoding (e.g., CABAC) can be used for smpm_idx, in which case the context index (or context index increment) of bin 0 can be set differently depending on the situation. The context model of smpm_idx can be determined differently based on the current block size and / or the smpm_idc value. For example, the context index (or context index increment) of bin 0 of smpm_idx can be set differently based on smpm_idc (or smpm_group_idx or smpm_group_flag).

[0214] For example, when the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is greater than 1, the context index increment of bin 0 of smpm_idx can be 1. When the value of smpm_idc (or smpm_group_idx or smpm_group_flag) is 1, the context index increment of bin 0 of smpm_idx can be 0, or vice versa. In the table below, smpm_idc can be replaced by smpm_group_idx or smpm_group_flag. In this case, the value of smpm_group_idx or smpm_group_flag as a condition can be set to be 1 less than the value of smpm_idc.

[0215] [Table 14]

[0216] [Table 15]

[0217] [Table 16]

[0218] [Table 17]

[0219] Through the above context-based coding, the context model can be adaptively assigned to information related to the intra-frame prediction mode, and the entropy coding efficiency can be effectively increased.

[0220] Figure 12 A video / image encoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 12 The method disclosed in the article can be derived from Figure 2 The encoding device disclosed in the document is used to execute it. Specifically, for example, Figure 12 S1200 to S1220 can be performed by the predictor 220 of the encoding device 200, and Figure 12 S1230 can be executed by the entropy encoder 240 of the encoding device 200. Figure 12 The methods disclosed herein may include the aforementioned embodiments.

[0221] Reference Figure 12 The encoding device derives the MPM list for the current block (S1200).

[0222] The MPM list may include intra-prediction mode candidates for the current block. The MPM list may include a first MPM list, a second MPM list, and a third MPM list. In this case, the second MPM list and the first MPM list may not overlap, and the second MPM list and the third MPM list may overlap. For example, candidates included in the second MPM list may not be included in the first MPM list, and candidates included in the second MPM list may be included in the third MPM list. The first MPM list may be referred to as the PMPM list. The second MPM list may be referred to as the SMPM list. The third MPM list may be referred to as the GMPM list.

[0223] The MPM list can be derived based on a first intra-prediction mode group and a second intra-prediction mode group. The first intra-prediction mode group may include the intra-prediction modes of neighboring blocks of the current block, and the second intra-prediction mode group may include the intra-prediction mode for the current block based on decoder-side intra-mode derivation (DIMD).

[0224] Intra-prediction modes in the second intra-prediction mode group can have a higher priority than intra-prediction modes in the first intra-prediction mode group and can be assigned to the front of the MPM list.

[0225] The overlap pattern between the intra-prediction modes of the first intra-prediction mode group and the intra-prediction modes of the second intra-prediction mode group can be derived. In this case, the overlap pattern can have a higher priority than the non-overlapping pattern and can be assigned to the front of the MPM list.

[0226] The overlap pattern between the intra-prediction modes of the first intra-prediction mode group and the intra-prediction modes of the second intra-prediction mode group can be derived. When multiple overlap patterns exist, the priority among the overlap patterns can be based on the priority among the DIMD-based intra-prediction modes.

[0227] The MPM list can be further derived based on the third intra-prediction mode group. The third intra-prediction mode group can include history-based intra-prediction modes in the history-based intra-prediction mode (HIPM) buffer.

[0228] The HIPM buffer can include intra-prediction modes of blocks previously decoded in a specific region where the current block resides and which are not adjacent to the current block. Intra-prediction modes in the HIPM buffer can be reordered based on their frequency of occurrence within a specific region. The intra-prediction mode of an unavailable block within a specific region can be set to be equal to the DIMD-based intra-prediction mode or a predetermined intra-prediction mode for that specific block. The intra-prediction mode of a specific block encoded based on inter-frame prediction or intra-block copy (IBC) prediction within a specific region can be set to be equal to the DIMD-based intra-prediction mode or a predetermined intra-prediction mode for that specific block. The predetermined intra-prediction mode can be a planar mode or a DC mode. The HIPM buffer can be initialized on a CTU line basis. The HIPM buffer can also be initialized on a CTU, multiple CTU lines, slice, or tile basis.

[0229] The coding device derives the intra-prediction mode for the current block (S1210). The coding device can derive the intra-prediction mode applicable to the current block from the intra-prediction modes based on the RD cost.

[0230] The encoding device generates prediction-related information (S1220). The encoding device can generate prediction-related information based on the derived intra-prediction mode and MPM list.

[0231] For example, prediction-related information may include at least one of the following: PMPM flag, PMPM index, SMPM flag, SMPM group index, or SMPM mode index. In this case, the PMPM flag may indicate whether the intra-prediction mode of the current block exists in a first MPM list, the PMPM index may be an index indicating a candidate in the first MPM list, the SMPM flag may indicate whether the intra-prediction mode of the current block exists in a second MPM list, the SMPM group index may be an index indicating an SMPM group in the second MPM list, and the SMPM mode index may be an index indicating a candidate in the SMPM group. When the value of the SMPM flag is 1, the SMPM group index and SMPM mode index can be signaled.

[0232] The binary method for SMPM pattern indexes can be determined differently based on the values ​​of the SMPM group indexes.

[0233] The SMPM pattern index can be encoded based on the context model, and the context model for the SMPM pattern index can be determined based on the value of the SMPM group index.

[0234] The SMPM group index can indicate either a first SMPM group or a second SMPM group, and the number of candidates in the first SMPM group can differ from the number of candidates in the second SMPM group. For example, the number of candidates in the second SMPM group can be twice the number of candidates in the first SMPM group.

[0235] In another example, prediction-related information may include at least one of the following: PMPM flag, PMPM index, SMPM flag, SMPM group flag, or SMPM mode index. The PMPM flag may indicate whether the intra-prediction mode of the current block exists in a first MPM list; the PMPM index may be an index indicating a candidate in the first MPM list; the SMPM flag may indicate whether the intra-prediction mode of the current block exists in a second MPM list; the SMPM group flag may be an index indicating an SMPM group in the second MPM list; and the SMPM mode index may be an index indicating a candidate in the SMPM group. When the value of the SMPM flag is 1, the SMPM group flag and SMPM mode index can be signaled. As mentioned above, at least one of the SMPM flag and SMPM group flag can be replaced by SMPM indication (smpm_idc) information.

[0236] The binary representation method for the SMPM pattern index can be determined differently based on the value of the SMPM group flag. For example, if the SMPM group flag value is 1, the SMPM pattern index can be binary represented using a truncated unary code; if the SMPM group flag value is 0, the SMPM pattern index can be binary represented using a fixed length.

[0237] The SMPM pattern index can be encoded based on a context model, and the context model for the SMPM pattern index can be determined based on the value of the SMPM group flag.

[0238] The encoding device encodes image information, including prediction-related information (S1230). For example, the prediction-related information may include the CU syntax for the current block. The image information may be referred to as video information.

[0239] According to embodiments of this disclosure, image information may include various information segments. For example, image information may include information disclosed in at least one of the tables above.

[0240] Image information can include residual information. Residual information is information about the residual samples. Residual information can include information about the transform coefficients used for quantization of the residual samples.

[0241] Encoded image information can be output as a bitstream. The bitstream can be sent to a decoding device via a network or storage medium. For example, image data including a bitstream can be sent to the decoding device by a sending device (or transmitter). In this case, image data including a bitstream can be sent to the decoding device via a streaming server.

[0242] Furthermore, as mentioned above, the encoding device can generate reconstructed frames (including reconstructed samples and reconstructed blocks) based on reference samples and residual samples. This allows the encoding device to derive the same prediction results as the decoding device, thereby improving encoding efficiency. Therefore, the encoding device can store the reconstructed frames (or reconstructed samples and reconstructed blocks) in memory and use the reconstructed frames (or reconstructed samples or reconstructed blocks) as reference frames for inter-frame prediction. As mentioned above, the in-loop filtering process can be further applied to the reconstructed frames.

[0243] According to the aforementioned implementation, the prediction mode applied to the current block can be efficiently indicated, and the amount of side information data can be reduced. Furthermore, by configuring the MPM list considering not only the correlation with neighboring blocks but also the correlation with neighboring reconstructed samples, the probability of the intra-frame prediction mode of the current block being configured as an upper candidate in the MPM list can be increased, thereby improving the signaling efficiency of the intra-frame prediction mode.

[0244] Figure 13 A video / image decoding method according to an embodiment of the present disclosure is illustrated schematically. Figure 13 The method disclosed in the article can be derived from Figure 3 The decoding device disclosed in the document is used to perform this. Specifically, for example, Figure 13 S1300 can be executed by the entropy decoder 310 of the decoding device 300, and S1310 to S1330 can be executed by the predictor 330 of the decoding device 300. Figure 13 The methods disclosed herein may include the aforementioned embodiments.

[0245] Reference Figure 13 The decoding device obtains prediction-related information through the bitstream (S1300). The decoding device can obtain image information including prediction-related information through the bitstream. The image information may also include residual information as described above.

[0246] For example, prediction-related information may include at least one of the following: PMPM flag, PMPM index, SMPM flag, SMPM group index, or SMPM mode index. In this case, the PMPM flag may indicate whether the intra-prediction mode of the current block exists in a first MPM list, the PMPM index may be an index indicating a candidate in the first MPM list, the SMPM flag may indicate whether the intra-prediction mode of the current block exists in a second MPM list, the SMPM group index may be an index indicating an SMPM group in the second MPM list, and the SMPM mode index may be an index indicating a candidate in the SMPM group. When the value of the SMPM flag is 1, the SMPM group index and SMPM mode index can be signaled.

[0247] The binary method for SMPM pattern indexes can be determined differently based on the values ​​of the SMPM group indexes.

[0248] The SMPM pattern index can be decoded based on the context model, and the context model for the SMPM pattern index can be determined based on the value of the SMPM group index.

[0249] The SMPM group index can indicate either a first SMPM group or a second SMPM group, and the number of candidates in the first SMPM group can differ from the number of candidates in the second SMPM group. For example, the number of candidates in the second SMPM group can be twice the number of candidates in the first SMPM group.

[0250] In another example, prediction-related information may include at least one of the following: PMPM flag, PMPM index, SMPM flag, SMPM group flag, or SMPM mode index. The PMPM flag may indicate whether the intra-prediction mode of the current block exists in a first MPM list; the PMPM index may be an index indicating a candidate in the first MPM list; the SMPM flag may indicate whether the intra-prediction mode of the current block exists in a second MPM list; the SMPM group flag may be an index indicating an SMPM group in the second MPM list; and the SMPM mode index may be an index indicating a candidate in the SMPM group. When the value of the SMPM flag is 1, the SMPM group flag and SMPM mode index can be signaled. As mentioned above, at least one of the SMPM flag and SMPM group flag can be replaced by SMPM indication (smpm_idc) information.

[0251] The binary representation method for the SMPM pattern index can be determined differently based on the value of the SMPM group flag. For example, if the SMPM group flag value is 1, the SMPM pattern index can be binary represented using a truncated unary code; if the SMPM group flag value is 0, the SMPM pattern index can be binary represented using a fixed length.

[0252] The SMPM pattern index can be decoded based on the context model, and the context model for the SMPM pattern index can be determined based on the value of the SMPM group flag.

[0253] The decoding device derives the MPM list for the current block (S1310).

[0254] The MPM list may include intra-prediction mode candidates for the current block. The MPM list may include a first MPM list, a second MPM list, and a third MPM list. In this case, the second MPM list and the first MPM list may not overlap, and the second MPM list and the third MPM list may overlap. For example, candidates included in the second MPM list may not be included in the first MPM list, and candidates included in the second MPM list may be included in the third MPM list. The first MPM list may be referred to as the PMPM list. The second MPM list may be referred to as the SMPM list. The third MPM list may be referred to as the GMPM list.

[0255] The MPM list can be derived based on a first intra-prediction mode group and a second intra-prediction mode group. The first intra-prediction mode group may include the intra-prediction modes of neighboring blocks of the current block, and the second intra-prediction mode group may include the intra-prediction mode for the current block based on decoder-side intra-mode derivation (DIMD).

[0256] Intra-prediction modes in the second intra-prediction mode group can have a higher priority than intra-prediction modes in the first intra-prediction mode group and can be assigned to the front of the MPM list.

[0257] The overlap pattern between the intra-prediction modes of the first intra-prediction mode group and the intra-prediction modes of the second intra-prediction mode group can be derived. In this case, the overlap pattern can have a higher priority than the non-overlapping pattern and can be assigned to the front of the MPM list.

[0258] The overlap pattern between the intra-prediction modes of the first intra-prediction mode group and the intra-prediction modes of the second intra-prediction mode group can be derived. When multiple overlap patterns exist, the priority among the overlap patterns can be based on the priority among the DIMD-based intra-prediction modes.

[0259] The MPM list can be further derived based on the third intra-prediction mode group. The third intra-prediction mode group can include history-based intra-prediction modes in the history-based intra-prediction mode (HIPM) buffer.

[0260] The HIPM buffer can include intra-prediction modes of blocks previously decoded in a specific region where the current block resides and which are not adjacent to the current block. Intra-prediction modes in the HIPM buffer can be reordered based on their frequency of occurrence within a specific region. The intra-prediction mode of unavailable blocks within a specific region can be set to be equal to the DIMD-based intra-prediction mode or a predetermined intra-prediction mode for that specific block. The intra-prediction mode of a specific block encoded based on inter-frame prediction or intra-block copy (IBC) prediction within a specific region can be set to be equal to the DIMD-based intra-prediction mode or a predetermined intra-prediction mode for that specific block. The predetermined intra-prediction mode can be a planar mode or a DC mode. The HIPM buffer can be initialized on a CTU line basis. The HIPM buffer can also be initialized on a CTU, multiple CTU lines, slice, or tile basis.

[0261] The decoding device derives the intra-prediction mode for the current block (S1320). The decoding device can derive the intra-prediction mode for the current block based on the MPM list and prediction-related information.

[0262] The decoding device generates prediction samples based on the derived intra-prediction mode (S1330). For example, the decoding device may generate prediction samples for the current block based on the derived intra-prediction mode and adjacent reference samples. In this case, as described above, a prediction sample filtering process may be further performed on all or some of the prediction samples for the current block, depending on the circumstances.

[0263] The decoding device can generate reconstructed samples based on predicted samples for the current block. For example, the decoding device can generate reconstructed samples for the current block based on residual samples and predicted samples for the current block. Residual samples for the current block can be generated based on received residual information. Additionally, for example, the decoding device can generate a reconstructed image that includes the reconstructed samples. As described above, the in-loop filtering process can be further applied to the reconstructed image.

[0264] According to the aforementioned implementation, the prediction mode applied to the current block can be efficiently indicated, and the amount of side information data can be reduced. Furthermore, by configuring the MPM list considering not only the correlation with neighboring blocks but also the correlation with neighboring reconstructed samples, the probability of the intra-prediction mode of the current block being configured as a top candidate in the MPM list can be increased, thereby improving the signaling efficiency of the intra-prediction mode.

[0265] Although the method is described as a series of steps or blocks based on the flowchart in the foregoing embodiments, the embodiments are not limited to the order of the steps, and a step may occur in a different order or simultaneously with another step described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps may be deleted without affecting the scope of the embodiments of this disclosure.

[0266] The aforementioned methods according to embodiments of the present disclosure can be implemented in software, and the encoding and / or decoding devices according to the present disclosure can be included in apparatus for performing image processing, such as TVs, computers, smartphones, set-top boxes, and display devices.

[0267] The embodiments described above can be implemented in the form of a recording medium including computer-executable (program) instructions, such as program modules executed by a computer. These modules can be stored in memory and executed by a processor. The memory can reside internally or externally to the processor and can be connected to the processor via various known means. The computer-readable medium can be any available medium accessible to a computer and can include volatile and non-volatile media, as well as removable and non-removable media. Furthermore, the computer-readable medium can include both computer storage media and communication media. Computer storage media can include volatile and non-volatile media, as well as removable and non-removable media, implemented using any method or technique for storing information such as computer-readable instructions, data structures, program modules, or other data. Communication media typically include computer-readable instructions, data structures, program modules, other data in modulated data signals such as carrier waves, or other transmission mechanisms, and include any information delivery medium.

[0268] Furthermore, the embodiments of this disclosure described above can be implemented as a computer program (or computer program product) including computer-executable instructions. The computer program may include programmable machine instructions processed by a processor and may be implemented using a high-level programming language, an object-oriented programming language, assembly language, or machine language. Additionally, the computer program may be recorded on a tangible computer-readable recording medium (e.g., memory, hard disk, magnetic / optical media, or solid-state drive (SSD)).

[0269] Therefore, when the above-described computer program is executed by a computing device, embodiments of the present disclosure can be implemented. The computing device may include at least some of a processor, memory, storage devices, high-speed interfaces connected to the memory and high-speed expansion ports, and low-speed interfaces connected to low-speed buses and storage devices. These components may be interconnected via various buses and may be mounted on a common motherboard or otherwise suitably mounted.

[0270] The processor can process instructions within a computing device. These instructions may include instructions stored in memory or a storage device to display graphical information for providing a graphical user interface (GUI) on an external input / output device, such as a display connected to a high-speed interface. In another embodiment, multiple processors and / or multiple buses may be suitably used with multiple memories and memory types. Furthermore, the processor can be implemented as a chipset comprising multiple independent analog and / or digital processor chips.

[0271] The memory stores information within a computing device. For example, the memory may include volatile memory cells or a set of volatile memory cells. In another example, the memory may include non-volatile memory cells or a set of non-volatile memory cells. The memory may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0272] Storage devices can provide high-capacity storage space for computing devices. Storage devices can be computer-readable media or components that include computer-readable media. For example, storage devices can include devices within a storage area network (SAN) or other components, and can be floppy disk devices, hard disk devices, optical disk devices, magnetic tape devices, flash memory, other similar semiconductor memory devices, or device arrays.

[0273] The network can be implemented as a wired network (such as a local area network (LAN), a wide area network (WAN), or a value-added network (VAN)) or various types of wireless networks (such as mobile radio communication networks or satellite communication networks).

[0274] Although this disclosure has been described with reference to embodiments shown in the accompanying drawings, these embodiments are merely exemplary. Those skilled in the art will understand that various modifications and variations of the embodiments are possible. That is, the scope of this disclosure is not limited to the described embodiments, and various modifications and changes made by those skilled in the art based on the basic concepts defined in the appended claims also fall within the scope of the claims. Therefore, the true technical scope of this disclosure should be determined by the technical spirit of the appended claims.

Claims

1. An image decoding method performed by a decoding device, the image decoding method comprising: Obtain prediction-related information through bitstream; Derive the most likely pattern (MPM) list for the current block; The intra-frame prediction mode of the current block is derived based on the MPM list and the prediction-related information. as well as Based on the derived intra-frame prediction mode, prediction samples are generated for the current block. The MPM list includes a first MPM list, a second MPM list, and a third MPM list. Candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

2. The image decoding method according to claim 1, wherein, The MPM list is derived based on the first intra-frame prediction mode group and the second intra-frame prediction mode group. The first intra-prediction mode group includes the intra-prediction modes of the adjacent blocks of the current block, and The second intra-prediction mode group includes intra-prediction modes based on decoder-side intra-mode derivation (DIMD) for the current block.

3. The image decoding method according to claim 2, wherein, The intra-prediction modes of the second intra-prediction mode group have a higher priority than the intra-prediction modes of the first intra-prediction mode group and are assigned to the front of the MPM list.

4. The image decoding method according to claim 2, further comprising: Derive the overlap pattern between the intra-prediction modes of the first intra-prediction mode group and the intra-prediction modes of the second intra-prediction mode group. The overlapping mode has a higher priority than the non-overlapping mode and is assigned to the front of the MPM list.

5. The image decoding method according to claim 2, further comprising: Derive the overlap pattern between the intra-prediction modes of the first intra-prediction mode group and the intra-prediction modes of the second intra-prediction mode group. Wherein, when there are multiple overlapping modes, the priority of the overlapping modes is based on the priority of the DIMD-based intra-prediction modes.

6. The image decoding method according to claim 1, wherein, The MPM list is further derived based on the third intra-frame prediction mode group, and The third intra-prediction mode group includes the history-based intra-prediction mode in the history-based intra-prediction mode HIPM buffer.

7. The image decoding method according to claim 6, wherein, The HIPM buffer includes intra-prediction modes for blocks that were previously decoded in a specific region where the current block is located and are not adjacent to the current block.

8. The image decoding method according to claim 7, wherein, The intra-prediction modes in the HIPM buffer are reordered based on the frequency of occurrence of the intra-prediction modes within the specific region.

9. The image decoding method according to claim 7, wherein, The intra-prediction mode of a specific block encoded based on inter-frame prediction or intra-block copy IBC prediction within the specific region is set to be equal to the DIMD-based intra-prediction mode or a predetermined intra-prediction mode for the specific block.

10. The image decoding method according to claim 6, wherein, The HIPM buffer is initialized in units of CTU lines.

11. The image decoding method according to claim 1, wherein, The prediction-related information includes at least one of the following: PMPM flag, PMPM index, SMPM flag, SMPM group index, or SMPM pattern index. The PMPM flag indicates whether the intra-prediction mode of the current block exists in the first MPM list. The PMPM index is an index used to indicate candidates in the first MPM list. The SMPM flag indicates whether the intra-prediction mode of the current block exists in the second MPM list. The SMPM group index is an index used to indicate the SMPM group in the second MPM list. The SMPM pattern index is an index used to indicate candidates in the SMPM group, and When the value of the SMPM flag is 1, the SMPM group index and the SMPM pattern index are signaled.

12. The image decoding method according to claim 11, wherein, The binary method for the SMPM pattern index is determined differently based on the value of the SMPM group index.

13. The image decoding method according to claim 11, wherein, The SMPM pattern index is decoded based on the context model, and The context model for the SMPM pattern index is determined based on the value of the SMPM group index.

14. The image decoding method according to claim 11, wherein, The SMPM group index indicates either the first SMPM group or the second SMPM group, and The number of candidates in the first SMPM group is different from the number of candidates in the second SMPM group.

15. The image decoding method according to claim 1, wherein, The prediction-related information includes at least one of the following: PMPM flag, PMPM index, SMPM flag, SMPM group flag, or SMPM pattern index. The PMPM flag indicates whether the intra-prediction mode of the current block exists in the first MPM list. The PMPM index is an index used to indicate candidates in the first MPM list. The SMPM flag indicates whether the intra-prediction mode of the current block exists in the second MPM list. The SMPM group flag is an index used to indicate the SMPM group in the second MPM list. The SMPM pattern index is an index used to indicate candidates in the SMPM group, and When the value of the SMPM flag is 1, the SMPM group flag and the SMPM mode index are signaled.

16. The image decoding method according to claim 15, wherein, The binary method for the SMPM pattern index is determined differently based on the value of the SMPM group flag.

17. The image decoding method according to claim 16, wherein, When the value of the SMPM group flag is 1, the SMPM pattern index is binary-coded based on the truncated unary code, and... When the value of the SMPM group flag is 0, the SMPM pattern index is binary-coded based on a fixed length.

18. The image decoding method according to claim 15, wherein, The SMPM mode index is decoded based on a context model, and The context model for the SMPM pattern index is determined based on the value of the SMPM group flag.

19. An image encoding method performed by an encoding device, the image encoding method comprising: Derive the most likely pattern (MPM) list for the current block; Derive the intra-prediction mode of the current block; Based on the MPM list, generate prediction-related information to indicate the intra-prediction mode of the current block; as well as The image information, including the prediction-related information, is encoded. The MPM list includes a first MPM list, a second MPM list, and a third MPM list. Candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

20. A method for transmitting image data, the method comprising: Obtain a bitstream generated by an image encoding method, the image encoding method comprising: deriving a most probable mode (MPM) list for a current block; deriving an intra-prediction mode (IMM) for the current block; generating prediction-related information based on the MPM list to indicate the IMM for the current block; and encoding image information including the prediction-related information; and Send image data including the bitstream. The MPM list includes a first MPM list, a second MPM list, and a third MPM list. Candidates included in the second MPM list are not included in the first MPM list, and candidates included in the second MPM list are included in the third MPM list.

Images