Matrix-based intra-prediction device and method

Abstract

The present invention provides a method and apparatus for improving the efficiency of video coding. [Solution] The video decoding method according to this document includes the steps of: receiving flag information indicating whether or not matrix-based intraprediction (MIP) is used for the current block; receiving matrix-based intraprediction (MIP) mode information based on the flag information; generating an intraprediction sample for the current block based on the MIP mode information; and generating a reconstruction sample for the current block based on the intraprediction sample, wherein the binstring of syntax elements for the MIP mode information is binary-evolved using a truncated binary evolution scheme.

Description

[Technical Field]

[0001] This document relates to video coding technology, specifically to video coding technology for matrix-based intra-prediction devices and methods. [Background technology]

[0002] In recent years, demand for high-resolution, high-quality video, such as 4K or 8K or higher UHD (Ultra High Definition) video, has been increasing in various fields. As video data becomes higher resolution and higher quality, the amount of information or bits transmitted increases relative to existing video data. Therefore, when transmitting video data using existing wired or wireless broadband lines, or storing video data using existing storage media, transmission and storage costs increase.

[0003] Furthermore, in recent years, interest in and demand for immersive media such as VR (Virtual Reality), AR (Artificial Reality), and holograms have increased, and there has been a rise in broadcasting of video content with different visual characteristics from real-world footage, such as game footage.

[0004] Consequently, highly efficient video compression technology is required to effectively compress, transmit, store, and play back high-resolution, high-quality video information possessing the various characteristics described above. [Overview of the project] [Problems that the invention aims to solve]

[0005] The technical objective of this document is to provide a method and apparatus for improving the efficiency of video coding.

[0006] Another technical challenge addressed in this document is to provide an efficient intra-prediction method and apparatus.

[0007] Another technical challenge of this document is to provide a video coding method and apparatus for matrix-based intra-prediction.

[0008] Another technical challenge of this document is to provide a video coding method and apparatus for coding mode information for matrix-based intra-prediction. [Means for solving the problem]

[0009] According to one embodiment of this document, a video decoding method is provided that is performed by a decoding device. The method includes the steps of: receiving flag information indicating whether or not matrix-based intraprediction (MIP) is used for the current block; receiving matrix-based intraprediction (MIP) mode information based on the flag information; generating an intraprediction sample for the current block based on the MIP mode information; and generating a reconstructed sample for the current block based on the intraprediction sample, wherein the binstring of syntax elements for the MIP mode information can be binary-evolved using a truncated binary evolution scheme.

[0010] The maximum length of the bin string of the syntax element for the MIP mode information can be set to a different value depending on the size of the current block.

[0011] The maximum length is set to one of three values ​​depending on the size of the current block, and the maximum length may be largest when the width and height of the current block are 4.

[0012] Such MIP mode information can be decoded using a bypass method.

[0013] The MIP mode information may be index information indicating the MIP mode applied to the current block.

[0014] The step of generating the intra prediction sample may include: a step of downsampling reference samples adjacent to the current block to derive a reduced boundary sample; a step of deriving a reduced prediction sample based on the multiplication of the reduced boundary sample and the MIP matrix; and a step of upsamping the reduced prediction sample to generate the intra prediction sample for the current block.

[0015] Here, the reduced boundary sample may be downsampled by averaging the reference samples, and the intra prediction sample may be upsampled by linear interpolation of the reduced prediction sample.

[0016] The MIP matrix can be derived based on the size of the current block and the index information.

[0017] The MIP matrix can be selected from any one of three matrix sets classified according to the size of the current block, and each of the three matrix sets can include a plurality of MIP matrices.

[0018] According to one embodiment of this document, a video encoding method is provided that is performed by an encoding device. The method includes the steps of: determining whether or not matrix-based intraprediction (MIP) is applied to the current block; if the MIP is applied to the current block, deriving intraprediction samples of the current block based on the MIP; deriving residual samples for the current block based on the intraprediction samples; and encoding information for the residual samples and information for the MIP, wherein the information for the MIP includes matrix-based intraprediction (MIP) mode information, and the binstring of syntax elements for the MIP mode information can be binary-evolved using a truncated binary evolution scheme.

[0019] According to yet another embodiment of this document, a digital storage medium can be provided which video data containing encoded video information and a bitstream generated by a video encoding method performed by an encoding device is stored.

[0020] According to yet another embodiment of this document, a digital storage medium can be provided which stores video data including encoded video information and a bitstream that triggers the video decoding method to be performed by a decoding device. [Effects of the Invention]

[0021] This document can have a variety of effects. For example, according to one embodiment of this document, the overall video compression efficiency can be improved. Alternatively, according to one embodiment of this document, the overall coding efficiency can be improved by reducing the complexity of implementation and improving prediction performance through efficient intra-prediction. Or, according to one embodiment of this document, the coding efficiency can be improved by efficiently coding the index information that indicates matrix-based intra-prediction.

[0022] The effects that can be obtained through a specific example in this document are not limited to those listed above. For example, there may be a variety of technical effects that a person having ordinary skill in the related art can understand or derive from this document. Thus, the specific effects of this document are not limited to those explicitly stated herein, but may include a variety of effects that can be understood or derive from the technical features of this document. [Brief explanation of the drawing]

[0023] [Figure 1] This figure schematically illustrates the configuration of a video / image encoding device that can be applied to the embodiments described herein.

[0024] [Figure 2] This figure schematically illustrates the configuration of a video / image decoding device that can be applied to the embodiments described herein.

[0025] [Figure 3] This example illustrates CABAC (context-adaptive binary arithmetic coding) for encoding syntax elements.

[0026] [Figure 4] This document presents an example of a video encoding method based on a schematic intra-prediction to which the embodiments described herein can be applied.

[0027] [Figure 5] A schematic diagram of the intra-prediction unit within the encoding device is shown.

[0028] [Figure 6] This document presents an example of a video decoding method based on a schematic intra-prediction to which the embodiments described herein can be applied.

[0029] [Figure 7] A schematic diagram of the intra-prediction unit within the decoding device is shown.

[0030] [Figure 8] This document shows an example of an intra-prediction mode to which the embodiments described herein can be applied.

[0031] [Figure 9] This diagram illustrates the procedure for generating a prediction sample based on MIP for one example.

[0032] [Figure 10] This diagram shows the MIP procedure for a 4x4 block.

[0033] [Figure 11] This diagram shows the MIP procedure for an 8x8 block.

[0034] [Figure 12] This diagram shows the MIP procedure for an 8x4 block.

[0035] [Figure 13] This diagram shows the MIP procedure for a 16x16 block.

[0036] [Figure 14] This diagram illustrates the boundary averaging procedure within the MIP procedure.

[0037] [Figure 15] This diagram illustrates linear interpolation, a part of the MIP procedure.

[0038] [Figure 16] This is a diagram illustrating the MIP technique as an example in this document.

[0039] [Figure 17]This flowchart schematically illustrates a decoding method that can be performed by a decoding device according to one embodiment of this document.

[0040] [Figure 18] This flowchart schematically illustrates an encoding method that can be performed by an encoding device according to one embodiment of this document.

[0041] [Figure 19] An example of a video / image coding system that can be applied to the embodiments described herein is schematically shown.

[0042] [Figure 20] Examples of content streaming systems to which the embodiments disclosed in this document can be applied are shown below. [Modes for carrying out the invention]

[0043] This document may be modified in various ways and may have various embodiments, but it attempts to illustrate and describe in detail specific embodiments with drawings. However, this does not mean that this document is intended to limit itself to any particular embodiment. The terminology used in this document is used solely to describe specific embodiments and is not intended to limit the technical ideas contained herein. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this document, terms such as "includes" or "has" are intended to indicate the presence of features, figures, steps, actions, components, parts, or combinations thereof described in the document, and should be understood not to preemptively exclude the possibility of the presence or addition of one or more different features, figures, steps, actions, components, parts, or combinations thereof.

[0044] On the other hand, each configuration shown in the diagrams described in this document is shown independently for the convenience of explaining its distinct characteristic functions, and does not mean that each configuration is embodied in separate hardware or separate software. For example, two or more configurations may be combined to form one configuration, and one configuration may be divided into multiple configurations. Embodiments in which each configuration is integrated and / or separated are also included in the scope of this document as long as they do not deviate from the essence of this document.

[0045] In this document, "A or B" may mean "just A," "just B," or "both A and B." In other words, in this document, "A or B" may be interpreted as "A and / or B." For example, in this document, "A, B or C" may mean "just A," "just B," "just C," or "any combination of A, B and C."

[0046] The slashes ( / ) and commas used in this document can mean "and / or". For example, "A / B" can mean "A and / or B". Thus, "A / B" can mean "just A", "just B", or "both A and B". For example, "A, B, C" can mean "A, B or C".

[0047] In this document, "at least one of A and B" may mean "just A," "just B," or "both A and B." Furthermore, in this document, the expressions "at least one of A or B" and "at least one of A and / or B" may be interpreted similarly to "at least one of A and B."

[0048] Furthermore, in this document, "at least one of A, B and C" may mean "just A," "just B," "just C," or "any combination of A, B and C." In addition, "at least one of A, B or C" or "at least one of A, B and / or C" may mean "at least one of A, B and C."

[0049] Furthermore, parentheses used in this document may mean "for example." Specifically, when "prediction (intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction." In other words, "prediction" in this document is not limited to "intra prediction," and "intra prediction" may be proposed as an example of "prediction." Also, when "prediction (i.e., intra prediction)" is displayed, "intra prediction" may be proposed as an example of "prediction."

[0050] Technical features described individually within a single drawing in this document may be embodied individually or simultaneously.

[0051] This document relates to video / image coding. For example, the methods / embodiments disclosed herein can be applied to methods disclosed in the VVC (versatile video coding) standard, EVC (essential video coding) standard, AV1 (AOMedia Video 1) standard, AVS2 (2nd generation of audio video coding standard), or next-generation video / image coding standards (e.g., H.267 or H.268).

[0052] This document presents various embodiments relating to video / image coding, and unless otherwise noted, these embodiments may be performed in combination with each other.

[0053] In this document, "video" can mean a collection of images over time. "Picture" generally refers to a unit representing a single image at a specific time point in time, and "slice" or "tile" is a unit that constitutes part of a picture in coding. A slice or tile can contain one or more coding tree units (CTUs). A single picture can consist of one or more slices or tiles. A single picture can consist of one or more tile groups. A tile group can contain one or more tiles. A brick may represent a rectangular region of CTU rows within a tile in a picture. A tile may be partitioned into multiple bricks, each of which consists of one or more CTU rows within the tile. A tile that is not partitioned into multiple bricks may also be referred to as a brick.A brick scan may represent a specific sequential ordering of CTUs partitioning a picture, where the CTUs are ordered consecutively in a CTU raster scan within a brick, bricks within a tile are ordered consecutively in a raster scan of the bricks of the tile, and tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A tile is a rectangular region of CTUs within a particular tile column and a particular tile row in a picture. The tile column is a rectangular region of CTUs having a height equal to the height of the picture and a width specified by syntax elements in the picture parameter set.The tile row is a rectangular region of CTUs having a height specified by syntax elements in the picture parameter set and a width equal to the width of the picture. A tile scan may indicate a specific sequential ordering of CTUs partitioning a picture in which the CTUs are ordered consecutively in CTU raster scan in a tile, whereas tiles in a picture are ordered consecutively in a raster scan of the tiles of the picture. A slice may include an integer number of bricks of a picture that may be exclusively contained in a single NAL unit. A slice may consist of either a number of complete tiles or only a consecutive sequence of complete bricks of one tile.In this document, tile groups and slices may be used interchangeably. For example, in this document, a tile group / tile group header may be referred to as a slice / slice header.

[0054] A pixel or pel can refer to the smallest unit that makes up a picture (or image). Alternatively, the term "sample" may be used as a counterpart to pixel. A sample generally refers to a pixel or a pixel value, sometimes only the luma component pixel / pixel value, or sometimes only the chroma component pixel / pixel value. Alternatively, a sample may refer to a pixel value in the spatial domain, and when such a pixel value is converted to the frequency domain, it may refer to the conversion coefficient in the frequency domain.

[0055] A unit can represent a basic unit of image processing. A unit can include a specific region of a picture and at least one piece of information relating to that region. A unit can include one luma block and two chroma (e.g., cb, cr) blocks. The term unit may be used interchangeably with terms such as block or area, as appropriate. In general, an M×N block can include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0056] The preferred embodiments of this document will be described in more detail below with reference to the attached drawings. Hereafter, the same reference numerals will be used for the same components in the drawings, and redundant descriptions of the same components may be omitted.

[0057] Figure 1 is a schematic diagram illustrating the configuration of a video / image encoding device that can be applied to the embodiments described herein. Hereinafter, the term "video encoding device" may include an image encoding device.

[0058] Referring to Figure 1, the encoding device 100 may be configured to include an image partitioner 110, a predictor 120, a residual processor 130, an entropy encoder 140, an adder 150, a filter 160, and a memory 170. The predictor 120 may include an inter-predictor 121 and an intra-predictor 122. The residual processor 130 may include a transformer 132, a quantizer 133, a dequantizer 134, and an inverse transformer 135. The residual processor 130 may further include a subtractor 131. The adder 150 may be called a reconstructor or a reconstructed block generator. The aforementioned video splitting unit 110, prediction unit 120, residual processing unit 130, entropy encoding unit 140, addition unit 150, and filtering unit 160 can be configured by one or more hardware components (e.g., an encoder chipset or processor) depending on the embodiment. Furthermore, the memory 170 may include a DPB (decoded picture buffer) and may be configured by a digital storage medium. The hardware components may further include the memory 170 as an internal / external component.

[0059] The video splitting unit 110 can split the input video (or picture, frame) input to the encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). In this case, the coding units can be recursively split from a coding tree unit (CTU) or the largest coding unit (LCU) using a QTBTTT (Quad-tree binary-tree ternary-tree) structure. For example, one coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quad-tree structure may be applied first, followed by the binary tree structure and / or the ternary structure. Alternatively, the binary tree structure may be applied first. Based on the final coding unit that is not further split, the coding procedure described in this document may be performed. In this case, based on coding efficiency due to video characteristics, the largest coding unit can be immediately used as the final coding unit, or, if necessary, the coding unit can be recursively divided into lower-depth coding units so that the optimally sized coding unit is used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and restoration, which will be described later. As another example, the processing unit may further include a prediction unit (PU) or a transformation unit (TU). In this case, the prediction unit and the transformation unit can each be separated or partitioned from the final coding unit described above.The prediction unit may be a unit of sample prediction, and the conversion unit may be a unit for deriving a conversion coefficient and / or a unit for deriving a residual signal from the conversion coefficient.

[0060] The term "unit" can be used interchangeably with terms such as "block" or "area," depending on the context. Generally, an M×N block can represent a set of samples or transform coefficients consisting of M columns and N rows. A sample can generally represent a pixel or a pixel value, and may represent only the luminance (luma) component pixel / pixel value, or only the chroma component pixel / pixel value. A sample can be used as the term corresponding to a single picture (or image) pixel or pel.

[0061] The encoding device 100 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal (predicted block, predicted sample array) output from the inter-prediction unit 121 or intra-prediction unit 122 from the input video signal (original block, original sample array), and the generated residual signal is transmitted to the conversion unit 132. In this case, as shown, the unit that subtracts the predicted signal (predicted block, predicted sample array) from the input video signal (original block, original sample array) within the encoding device 100 may be called the subtraction unit 131. The prediction unit can make predictions for the block to be processed (hereinafter referred to as the current block) and generate a predicted block that includes predicted samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied on a current block or CU basis. The prediction unit can generate various information related to prediction, such as prediction mode information, and transmit it to the entropy encoding unit 140, as will be described later in the explanation of each prediction mode. The prediction information can be encoded by the entropy encoding unit 140 and output in the form of a bitstream.

[0062] The intra-prediction unit 122 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance, depending on the prediction mode. The prediction mode in intra-prediction can include multiple non-directional modes and multiple directional modes. Non-directional modes can include, for example, DC mode and planar mode. Directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of fineness of the prediction direction. However, this is illustrative, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 122 can also determine the prediction mode to apply to the current block using the prediction modes applied to the surrounding blocks.

[0063] The interprediction unit 121 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding block and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding block may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, col CU, etc., and the reference picture containing the temporal neighboring block may also be called a collocated picture (colPic). For example, the interpretation unit 121 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 121 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of the surrounding blocks is used as a motion vector predictor, and the motion vector difference is signaled to indicate the motion vector of the current block.

[0064] The prediction unit 120 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for predictions on a single block, and can also apply intra-prediction and inter-prediction simultaneously. This may be called combined inter and intra prediction (CIIP). The prediction unit may also be based on intra-block copy (IBC) prediction mode or palette mode for predictions on blocks. The IBC prediction mode or palette mode can be used for coding content images / videos such as games, for example, in SCC (screen content coding). IBC basically performs predictions within the current picture, but can be performed similarly to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can use at least one of the inter-prediction techniques described in this document. Palette mode can be seen as an example of intra-coding or intra-prediction. When palette mode is applied, sample values ​​within the picture can be signaled based on information about the palette table and palette index.

[0065] The prediction signal generated via the prediction unit (including the inter-prediction unit 121 and / or the intra-prediction unit 122) can be used to generate a reconstructed signal or a residual signal. The transformation unit 132 can apply a transformation technique to the residual signal to generate transformation coefficients. For example, the transformation technique may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph when relational information between pixels is represented by this graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels and based on that. The transformation process may also be applied to pixel blocks of the same size that are square, or to non-square blocks of variable size.

[0066] The quantization unit 133 quantizes the conversion coefficients and transmits them to the entropy encoding unit 140, which can encode the quantized signal (information about the quantized conversion coefficients) and output it as a bitstream. The information about the quantized conversion coefficients may be called residual information. The quantization unit 133 can rearrange the block-shaped quantized conversion coefficients into a one-dimensional vector based on the scan order of the coefficients, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector of quantized conversion coefficients. The entropy encoding unit 140 can perform various encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). The entropy encoding unit 140 can encode, either together with or separately, information necessary for video / image reconstruction (e.g., values ​​of syntax elements) in addition to the quantized conversion coefficients. The encoded information (e.g., encoded video / image information) can be transmitted or stored in the form of a bitstream in units of NAL (network abstraction layer) units. The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. In this document, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded via the encoding procedure described above and included in the bitstream.The bitstream may be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The signal output from the entropy encoding unit 140 may be transmitted by a transmitting unit (not shown) and / or stored by a storage unit (not shown) which are configured as internal / external elements of the encoding device 100, or the transmitting unit may be included in the entropy encoding unit 140.

[0067] The quantized conversion coefficients output from the quantization unit 133 can be used to generate a prediction signal. For example, a residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 134 and the inverse transformation unit 135. The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 121 or the intra-prediction unit 122. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 250 may be called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, can be filtered and used for inter-prediction of the next picture.

[0068] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture encoding and / or restoration process.

[0069] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering-related information and transmit it to the entropy encoding unit 140, as will be described later in the explanation of each filtering method. The filtering-related information can be encoded by the entropy encoding unit 140 and output in the form of a bitstream.

[0070] The corrected restored picture sent to memory 170 can be used as a reference picture in the interpretation unit 121. When interpretation is applied via this, the encoding device can avoid prediction mismatches between the encoding device 100 and the decoding device, and can also improve encoding efficiency.

[0071] The DPB in memory 170 can store the corrected restored picture for use as a reference picture in the inter-prediction unit 121. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 121 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 122.

[0072] Figure 2 is a schematic diagram illustrating the configuration of a video / image decoding device that can be applied to the embodiments described herein.

[0073] Referring to Figure 2, the decoding device 200 can be configured to include an entropy decoder 210, a residual processor 220, a predictor 230, an adder 240, a filter 250, and a memory 260. The predictor 230 may include an inter-predictor 231 and an intra-predictor 232. The residual processor 220 may include a dequantizer 221 and an inverse transformer 221. The aforementioned entropy decoder 210, residual processor 220, predictor 230, adder 240, and filtering device 250 can be configured by a single hardware component (e.g., a decoder chipset or processor) depending on the embodiment. The memory 260 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The aforementioned hardware component may further include memory 260 as an internal / external component.

[0074] When a bitstream containing video / image information is input, the decoding device 200 can reconstruct the image in a manner corresponding to the process by which the video / image information was processed in the encoding device shown in Figure 1. For example, the decoding device 200 can derive units / blocks based on block division information obtained from the bitstream. The decoding device 200 can perform decoding using the processing units applied in the encoding device. Therefore, the decoding processing unit may be, for example, a coding unit, which can be divided from a coding tree unit or a maximum coding unit into a quad-tree structure, a binary tree structure, and / or a terminally tree structure. One or more conversion units can be derived from the coding unit. The reconstructed video signal decoded and output via the decoding device 200 can then be reproduced via a playback device.

[0075] The decoding device 200 can receive the signal output from the encoding device shown in Figure 1 in the form of a bitstream, and the received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as the adaptation parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The decoding device can further decode the picture based on the parameter set information and / or the general constraint information. The signaling / received information and / or syntax elements described later in this document can be decoded via the decoding procedure and obtained from the bitstream. For example, the entropy decoding unit 210 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and quantized values ​​of conversion coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element in the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding and decoded blocks, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, performs arithmetic decoding of the bins, and generates symbols corresponding to the values ​​of each syntax element.At this time, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the next symbol / bin's context model after determining the context model. Of the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction unit (inter-prediction unit 232 and intra-prediction unit 231), and the residual values ​​for which entropy decoding has been performed in the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the residual processing unit 220. The residual processing unit 220 can derive residual signals (residual blocks, residual samples, residual sample arrays). In addition, of the information decoded by the entropy decoding unit 210, information related to filtering can be provided to the filtering unit 250. On the other hand, a receiving unit (not shown) that receives signals output from the encoding device may be further configured as an internal / external element of the decoding device 200, or the receiving unit may be a component of the entropy decoding unit 210. On the other hand, the decoding device relating to this document may be called a video / image / picture decoding device, and the decoding device may also be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include the entropy decoding unit 210, and the sample decoder may include at least one of the inverse quantization unit 221, inverse transformation unit 222, addition unit 240, filtering unit 250, memory 260, inter-prediction unit 232, and intra-prediction unit 231.

[0076] The inverse quantization unit 221 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 221 can rearrange the quantized transformation coefficients in a two-dimensional block form. In this case, the rearrangement can be performed based on the scan order of the coefficients performed by the encoding device. The inverse quantization unit 221 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0077] In the inverse conversion unit 222, the conversion coefficients are inversely converted to obtain a residual signal (residual block, residual sample array).

[0078] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block, and can determine a specific intra / inter-prediction mode.

[0079] The prediction unit 220 can generate prediction signals based on various prediction methods described later. For example, the prediction unit can apply intra-prediction or inter-prediction for prediction of a single block, and can also apply intra-prediction and inter-prediction simultaneously. This may be called combined inter and intra prediction (CIIP). The prediction unit may also be based on intra-block copy (IBC) prediction mode or palette mode for prediction of a block. The IBC prediction mode or palette mode can be used for content video / movie coding such as games, for example, as in SCC (screen content coding). IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives reference blocks within the current picture. That is, IBC can use at least one of the inter-prediction techniques described in this document. Palette mode can be seen as an example of intra-coding or intra-prediction. When palette mode is applied, information about the palette table and palette index can be included in the video / movie information and signaled.

[0080] The intra-prediction unit 231 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) of the current block or at a distance from it, depending on the prediction mode. The prediction mode in intra-prediction can include a plurality of non-directional modes and a plurality of directional modes. The intra-prediction unit 231 can also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0081] The interprediction unit 232 can derive a predicted block for the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in blocks, subblocks, or samples based on the correlation of motion information between neighboring blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 232 can construct a motion information candidate list based on neighboring blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes, and the prediction information may include information indicating the mode of interprediction for the current block.

[0082] The summing unit 240 can generate a restored signal (restored picture, restored block, restored sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 232 and / or intra-prediction unit 231). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the restored block.

[0083] The summing unit 240 may be called the restoration unit or restoration block generation unit. The generated restoration signal can be used for intra-prediction of the next block to be processed in the current picture, and may be output after filtering, as described later, or may be used for intra-prediction of the next picture.

[0084] On the other hand, LMCS (luma mapping with chroma scaling) can also be applied during the picture decoding process.

[0085] The filtering unit 250 can apply filtering to the restored signal to improve subjective / objective image quality. For example, the filtering unit 250 can apply various filtering methods to the restored picture to generate a modified restored picture, and can transmit the modified restored picture to the memory 260, specifically to the DPB of the memory 260. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0086] The restored picture stored (modified) in the DPB of memory 260 can be used as a reference picture by the inter-prediction unit 232. Memory 260 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 232 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 260 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 231.

[0087] In this document, the embodiments described for the filtering unit 160, inter-prediction unit 121, and intra-prediction unit 122 of the encoding device 100 can also be applied identically or in a corresponding manner to the filtering unit 250, inter-prediction unit 232, and intra-prediction unit 231 of the decoding device 200, respectively.

[0088] As mentioned above, predictions are made to improve compression efficiency when performing video coding. Through this, a predicted block containing predicted samples for the current block, which is the block to be coded, can be generated. Here, the predicted block contains predicted samples in the spatial domain (or pixel domain). The predicted block is derived similarly by the encoding and decoding devices, and the encoding device can improve the efficiency of video coding by signaling the decoding device not the original sample values ​​of the original block themselves, but information about the residuals between the original block and the predicted block (residual information). Based on the residual information, the decoding device can derive a residual block containing residual samples, and by combining the residual block and the predicted block, it can generate a restored block containing restored samples, and thus a restored picture containing the restored block can be generated.

[0089] The residual information can be generated through transformation and quantization procedures. For example, an encoding device can derive a residual block between the original block and the predicted block, perform a transformation procedure on the residual samples (residual sample array) contained in the residual block to derive transformation coefficients, perform a quantization procedure on the transformation coefficients to derive quantized transformation coefficients, and signal the associated residual information (via a bitstream) to the decoding device. Here, the residual information may include information such as the value information of the quantized transformation coefficients, position information, transformation technique, transformation kernel, and quantization parameters. The decoding device can perform an inverse quantization / inverse transformation procedure based on the residual information to derive a residual sample (or residual block). The decoding device can generate a reconstructed picture based on the predicted block and the residual block. The encoding device can also derive a residual block by inverse quantization / inverse transformation of the quantized transformation coefficients for reference for subsequent picture interpretations, and generate a reconstructed picture based on this.

[0090] On the other hand, as mentioned above, encoding devices can perform a variety of encoding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). Decoding devices can decode information within a bitstream based on coding methods such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction, as well as quantized values ​​of conversion coefficients related to resistivity.

[0091] For example, the coding method described above can be implemented as described below.

[0092] Figure 3 illustrates CABAC (context-adaptive binary arithmetic coding) for encoding syntax elements. For example, in the CABAC encoding process, if the input signal is a syntax element rather than a binary value, the encoding device can convert the input signal to a binary value by binaryizing the value of the input signal. If the input signal is already a binary value (i.e., the value of the input signal is a binary value), binaryization can be bypassed. Here, each binary digit 0 or 1 that makes up the binary value can be called a bin. For example, if the binary string after binaryization is 110, then 1, 1, and 0 are each called one bin. The bins for a syntax element can represent the value of the syntax element. Such binaryization can be based on various binaryization methods such as Truncated Rice binarization process and Fixed-length binarization process, and the binaryization method for the target syntax element can be predefined. The aforementioned binary evolution procedure can be performed by the binary evolution unit within the entropy encoding unit.

[0093] Subsequently, the binary-evolved bins of the syntax elements can be input to a regular encoding engine or a bypass encoding engine. The encoding device's regular encoding engine can assign a context model that reflects the probability value to the bin and encode the bin based on the assigned context model. After performing encoding for each bin, the encoding device's regular encoding engine can update the context model for that bin. As described above, the bins to be encoded can be referred to as context-coded bins.

[0094] On the other hand, when the binary-evolved bins of the syntax elements are input to the bypass coding engine, they can be coded as follows. For example, the bypass coding engine of the encoding device omits the steps of estimating probabilities for the input bins and updating the probabilistic model applied to the bins after coding. When bypass encoding is applied, the encoding device can encode the input bins by applying a uniform probability distribution instead of assigning a context model, thereby improving the encoding speed. As described above, the bins to be encoded can be referred to as bypass bins.

[0095] Entropy decoding can be described as a process that performs the same steps as entropy encoding, but in reverse order.

[0096] The decoding device (entropy decoding unit) can decode encoded video information. The video information may include partitioning information, prediction information (e.g., inter / intra prediction classification information, intra prediction mode information, inter prediction mode information, etc.), residual information, in-loop filtering information, etc., or may include various syntax elements related thereto. The entropy coding can be performed on a unit of syntax elements.

[0097] The decoding device can perform binary evolution on a target syntax element. Here, the binary evolution can be based on various binary methods such as a Truncated Rice binarization process or a Fixed-length binarization process, and the binary evolution method for the target syntax element can be predefined. Through the binary evolution procedure, the decoding device can derive available binstrings (candidate binstrings) for the available values ​​of the target syntax element. The binary evolution procedure can be performed by a binary evolution unit within the entropy decoding unit.

[0098] The decoding device sequentially decodes and parses each bin for the target syntax element from the input bits in the bitstream, and compares the derived binstring with the available binstrings for the syntax element. If the derived binstring is the same as one of the available binstrings, the value corresponding to that binstring is derived as the value of the syntax element. Otherwise, the next bit in the bitstream is further parsed, and the above procedure is repeated. Through this process, the relevant information can be signaled using a variable length of bits without using start bits or end bits for specific information (specific syntax elements) in the bitstream. This allows for the allocation of relatively fewer bits to lower values, thereby improving overall coding efficiency.

[0099] The decoding device can decode each bin in the bin string from the bitstream based on a context model or by bypass, using an entropy coding technique such as CABAC or CAVLC.

[0100] When a syntax element is decoded based on a context model, the decoding device can receive the bin corresponding to the syntax element via a bitstream, determine the context model using the syntax element and the decoding information of the block or surrounding blocks to be decoded, or the symbol / bin information decoded in a previous step, predict the probability of the received bin occurring based on the determined context model, perform arithmetic decoding of the bin, and derive the value of the syntax element. Subsequently, the context model of the next bin to be decoded can be updated using the determined context model.

[0101] Context models can be assigned and updated for each bin that is context-coded (canonically coded), and context models can be indicated based on ctxIdx or ctxInc. ctxIdx can be derived based on ctxInc. Specifically, for example, a context index (ctxIdx) that points to the context model for each of the canonically coded bins can be derived as the sum of context index increment (ctxInc) and context index offset (ctxIdxOffset), where ctxInc can be derived differently for each bin. ctxIdxOffset can be represented by the lowest value of ctxIdx. ctxIdxOffset is a value generally used to distinguish context models from those for other syntax elements, and a context model for a single syntax element can be distinguished / derived based on ctxInc.

[0102] The entropy encoding procedure allows you to decide whether to perform encoding via the regular coding engine or the bypass coding engine, thereby switching the coding path. Entropy decoding performs the same process as entropy encoding, but in reverse order.

[0103] On the other hand, for example, if a syntax element is bypass-decoded, the decoding device can receive the bins corresponding to the syntax element via a bitstream and decode the input bins by applying a uniform probability distribution. In this case, the decoding device may omit the steps of deriving a context model of the syntax element and updating the context model applied to the bins after decoding.

[0104] As mentioned above, residual samples can be derived as quantized transformation coefficients through a transformation and quantization process. These quantized transformation coefficients may also be called transformation coefficients. In this case, the transformation coefficients within a block can be signaled in the form of residual information. This residual information may include residual coding syntax. That is, an encoding device can construct residual coding syntax with residual information, encode it, and output it in the form of a bitstream, and a decoding device can decode the residual coding syntax from the bitstream to derive residual (quantized) transformation coefficients. As will be described later, this residual coding syntax may include syntax elements that indicate whether a transformation has been applied to the block in question, where the last effective transformation coefficient in the block is located, whether effective transformation coefficients exist in subblocks, and what the magnitude / sign of the effective transformation coefficients are.

[0105] On the other hand, when intraprediction is performed, the correlation between samples can be used to obtain the difference between the original block and the predicted block, i.e., the residual. The transformation and quantization described above can be applied to the residual, thereby eliminating spatial redundancy. The encoding and decoding methods used in intraprediction will be described in detail below.

[0106] Intra prediction refers to a prediction that generates prediction samples for the current block based on reference samples outside the current block within the picture containing the current block (hereinafter referred to as the current picture). Here, reference samples outside the current block refer to samples located around the current block. When intra prediction is applied to the current block, the neighboring reference samples used for intra prediction of the current block can be derived.

[0107] For example, when the current block size (width × height) is nW × nH, the surrounding reference samples of the current block can include 2 × nH samples adjacent to the left boundary and the bottom-left boundary of the current block, 2 × nW samples adjacent to the top boundary and the top-right boundary of the current block, and 1 sample adjacent to the top-left boundary of the current block. Alternatively, the surrounding reference samples of the current block can include upper surrounding samples in multiple columns and left surrounding samples in multiple rows. Furthermore, the surrounding reference samples of the current block can also include nH samples adjacent to the right boundary of the nW × nH current block, nW samples adjacent to the bottom boundary of the current block, and 1 sample adjacent to the bottom-right boundary of the current block.

[0108] However, some of the surrounding reference samples in a block may not yet be decoded or available. In this case, the decoding device can substitute the unavailable samples with available samples to construct the surrounding reference samples used for prediction. Alternatively, it can construct the surrounding reference samples used for prediction through interpolation of available samples.

[0109] If peripheral reference samples are derived, (i) predicted samples can be derived based on the average or interpolation of the peripheral reference samples of the current block, or (ii) predicted samples can be derived based on reference samples from the peripheral reference samples of the current block that are located in a specific (predicted) direction relative to the predicted samples. Case (i) can be applied when the intra-prediction mode is non-directional or non-angular mode, and case (ii) can be applied when the intra-prediction mode is directional or angular mode.

[0110] Furthermore, among the surrounding reference samples, prediction samples can also be generated through interpolation between a first surrounding sample located in the prediction direction of the current block's intra-prediction mode and a second surrounding sample located in the opposite direction to the prediction direction, based on the prediction sample of the current block. In the case described above, this may be called linear interpolation intra-prediction (LIP). Alternatively, chroma prediction samples can be generated based on chroma samples using a linear model. In this case, this may be called LM mode.

[0111] Alternatively, a provisional predicted sample for the current block can be derived based on filtered peripheral reference samples, and the provisional predicted sample can be weighted summed with at least one reference sample derived by the intra-prediction mode from the existing peripheral reference samples, i.e., the unfiltered peripheral reference samples, to derive the predicted sample for the current block. In the case described above, it may be called PDPC (Position dependent intra-prediction).

[0112] Furthermore, intra-prediction encoding can be performed by selecting the reference sample line with the highest prediction accuracy from among the multiple reference sample lines surrounding the current block, deriving a predicted sample using the reference sample located in the prediction direction on that line, and then instructing (signaling) the decoding device to use the reference sample line used. In the above case, it may be called multi-reference line (MRL) intra prediction or MRL-based intra prediction.

[0113] Furthermore, the current block can be divided into vertical or horizontal subpartitions, intra-prediction can be performed based on the same intra-prediction mode, and peripheral reference samples can be derived and used on a subpartition-by-subpartition basis. In other words, in this case, the intra-prediction mode for the current block is applied similarly to the subpartitions, and by deriving and using peripheral reference samples on a subpartition-by-subpartition basis, the performance of intra-prediction can be improved as needed. Such a prediction method may be called intra-subpartitions (ISP) or intra-prediction based on ISP.

[0114] The intra-prediction method described above can be distinguished from the intra-prediction mode and may be called an intra-prediction type. An intra-prediction type may be referred to by various terms such as an intra-prediction technique or an additional intra-prediction mode. For example, an intra-prediction type (or additional intra-prediction mode, etc.) may include at least one of the aforementioned LIP, PDPC, MRL, or ISP. A general intra-prediction method that excludes the specific intra-prediction types such as LIP, PDPC, MRL, and ISP may be called a normal intra-prediction type. A normal intra-prediction type can be generally applied when the aforementioned specific intra-prediction types are not applicable, and predictions may be made based on the aforementioned intra-prediction modes. On the other hand, post-processing filtering may be performed on the derived prediction samples as needed.

[0115] On the other hand, in addition to the intra-prediction types mentioned above, matrix-based intra-prediction (MIP) can be used as another method for intra-prediction. MIP may also be referred to as affine linear weighted intra-prediction (ALWIP) or matrix weighted intra-prediction (MWIP).

[0116] When MIP is applied to a current block, prediction samples for the current block can be derived by i) using the surrounding reference samples that have undergone an averaging procedure, ii) performing a matrix-vector-multiplication procedure, and iii) further performing horizontal / vertical interpolation procedures as needed. The intra-prediction mode used for the MIP can be configured differently from the intra-prediction modes used in the aforementioned LIP, PDPC, MRL, ISP intra-prediction and normal intra-prediction.

[0117] The intra-prediction mode for MIP may be called an "affine linear weighted intra prediction mode" or a matrix-based intra-prediction mode. For example, the intra-prediction mode for MIP may allow different matrices and offsets to be set for matrix-vector multiplication. Here, the matrix may be called an (affine) weighted matrix, and the offset may be called an (affine) offset vector or an (affine) bias vector. In this document, the intra-prediction mode for MIP may be called an MIP intra-prediction mode, a linear weighted intra prediction mode, a matrix weighted intra prediction mode, or a matrix-based intra prediction mode. Specific MIP methods will be described later.

[0118] The following drawings were created to illustrate a specific example of this document. The names of specific devices, terms, and names (e.g., syntax names) shown in the drawings are presented illustratively, and the technical features of this document are not limited to the specific names used in the following drawings.

[0119] Figure 4 shows an example of a schematic intra-prediction-based video encoding method to which the embodiments described in this document can be applied, and Figure 5 schematically shows an intra-prediction unit in an encoding device. The intra-prediction unit in the encoding device in Figure 5 can be applied to the intra-prediction unit 122 of the encoding device 100 in Figure 1 described above, either identically or in correspondence to it.

[0120] Referring to Figures 4 and 5, S400 can be performed by the intra-prediction unit 122 of the encoding device, and S410 can be performed by the residual processing unit 130 of the encoding device. Specifically, S410 can be performed by the subtraction unit 131 of the encoding device. In S420, the prediction information is derived by the intra-prediction unit 122 and can be encoded by the entropy encoding unit 140. In S420, the residual information is derived by the residual processing unit 130 and can be encoded by the entropy encoding unit 140. The residual information is information about the residual sample. The residual information may include information about the quantized conversion coefficients for the residual sample. As mentioned above, the residual sample is derived as a conversion coefficient via the conversion unit 132 of the encoding device, and the conversion coefficient can be derived as a quantized conversion coefficient via the quantization unit 133. Information about the quantized conversion coefficients can be encoded by the entropy encoding unit 140 via the residual coding procedure.

[0121] The encoding device performs intraprediction for the current block (S400). The encoding device can derive the intraprediction mode / type for the current block and the surrounding reference samples for the current block, and generate predicted samples within the current block based on the intraprediction mode / type and the surrounding reference samples. Here, the steps of determining the intraprediction mode / type, deriving the surrounding reference samples, and generating predicted samples may be performed simultaneously, or one of the steps may be performed before the others.

[0122] For example, the intra-prediction unit 122 of the encoding device may include an intra-prediction mode / type determination unit 122-1, a reference sample derivation unit 122-2, and a prediction sample derivation unit 122-3. The intra-prediction mode / type determination unit 122-1 determines the intra-prediction mode / type for the current block, the reference sample derivation unit 122-2 derives the surrounding reference samples for the current block, and the prediction sample derivation unit 122-3 derives the prediction samples for the current block. On the other hand, although not shown, if a prediction sample filtering procedure is performed, the intra-prediction unit 122 may further include a prediction sample filter unit (not shown). The encoding device can determine which mode / type to apply to the current block from among a plurality of intra-prediction modes / types. The encoding device can compare the RD costs (RD costs) for the intra-prediction modes / types and determine the optimal intra-prediction mode / type for the current block.

[0123] As mentioned above, the encoding device can also perform a filtering procedure for the predicted samples. This filtering of predicted samples may be called post-filtering. The filtering procedure for predicted samples can filter out some or all of the predicted samples. Depending on the circumstances, the filtering procedure for predicted samples may be omitted.

[0124] The encoding device generates a residual sample for the current block based on the (filtered) predicted sample (S410). The encoding device can derive the residual sample by comparing the predicted sample with the original sample of the current block on a phase basis.

[0125] The encoding device can encode video information including intra-prediction information (prediction information) and residual information regarding residual samples (S420). The prediction information may include intra-prediction mode information and intra-prediction type information. The residual information may include the syntax of residual coding. The encoding device can convert / quantize residual samples and derive quantized conversion coefficients. The residual information may include information regarding the quantized conversion coefficients.

[0126] The encoding device can output the encoded video information in the form of a bitstream. The output bitstream can be transmitted to the decoding device via a storage medium or a network.

[0127] As mentioned above, the encoding device can generate a restored picture (including restored samples and restored blocks). Therefore, the encoding device can decrypt the quantized conversion coefficients again by inverse quantization / inverse transformation to derive (corrected) residual samples. The reason for decrypting / inversely transforming the residual samples after transformation / quantization is, as mentioned above, to derive residual samples identical to those derived from the decoding device. Based on the predicted samples and the (corrected) residual samples, the encoding device can generate a restored block containing restored samples for the current block. Based on the restored block, a restored picture for the current picture can be generated. As mentioned above, in-loop filtering procedures and the like can be further applied to the restored picture.

[0128] Figure 6 shows a schematic example of an intra-prediction-based video decoding method to which the embodiments described in this document can be applied, and Figure 7 schematically shows an intra-prediction unit in a decoding device. The intra-prediction unit in the decoding device in Figure 7 can be applied to the intra-prediction unit 231 of the decoding device 200 in Figure 2 described above, either identically or in correspondence to it.

[0129] Referring to Figures 6 and 7, the decoding device can perform operations corresponding to those performed by the encoding device described above. S600 to S620 can be performed by the intra-prediction unit 231 of the decoding device, and the prediction information in S600 and the residual information in S630 can be obtained from the bitstream by the entropy decoding unit 210 of the decoding device. The residual processing unit 220 of the decoding device can derive a residual sample for the current block based on the residual information. Specifically, the inverse quantization unit 221 of the residual processing unit 220 derives conversion coefficients by performing inverse quantization based on the quantized conversion coefficients derived from the residual information, and the inverse transformation unit 222 of the residual processing unit can perform an inverse transformation on the conversion coefficients to derive a residual sample for the current block. S640 can be performed by the addition unit 240 or the restoration unit of the decoding device.

[0130] The decoding device can derive the intra-prediction mode / type for the current block based on the received prediction information (intra-prediction mode / type information) (S600). The decoding device can derive the surrounding reference samples for the current block (S610). The decoding device generates prediction samples within the current block based on the intra-prediction mode / type and the surrounding reference samples (S620). In this case, the decoding device can perform a prediction sample filtering procedure. This filtering of prediction samples may be called post-filtering. The prediction sample filtering procedure can filter out some or all of the prediction samples. Depending on the circumstances, the prediction sample filtering procedure may be omitted.

[0131] The decoding device generates a residual sample for the current block based on the received residual information (S630). Based on the predicted sample and the residual sample, the decoding device can generate a restored sample for the current block and derive a restored block containing the restored sample (S640). Based on the restored block, a restored picture for the current picture can be generated. As previously mentioned, in-loop filtering procedures and the like can be further applied to the restored picture.

[0132] Here, the intra-prediction unit 231 of the decoding device may include an intra-prediction mode / type determination unit 231-1, a reference sample derivation unit 231-2, and a prediction sample derivation unit 231-3. The intra-prediction mode / type determination unit 231-1 determines the intra-prediction mode / type for the current block based on the intra-prediction mode / type information obtained from the entropy decoding unit 210, the reference sample derivation unit 231-2 derives the surrounding reference samples of the current block, and the prediction sample derivation unit 231-3 derives the prediction samples of the current block. On the other hand, although not shown, if the aforementioned prediction sample filtering procedure is performed, the intra-prediction unit 231 may further include a prediction sample filter unit (not shown).

[0133] The intra-prediction mode information may include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether MPM (most probable mode) is applied to the current block or whether remaining mode is applied. In this case, if MPM is applied to the current block, the prediction mode information may further include index information (e.g., intra_luma_mpm_idx) pointing to one of the intra-prediction mode candidates (MPM candidates). The intra-prediction mode candidates (MPM candidates) may consist of an MPM candidate list or an MPM list. If MPM is not applied to the current block, the intra-prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) pointing to one of the remaining intra-prediction modes excluding the intra-prediction mode candidates (MPM candidates). The decoding device can determine the intra-prediction mode of the current block based on the intra-prediction mode information.

[0134] Furthermore, intra-prediction type information can be embodied in various forms. For example, intra-prediction type information may include intra-prediction type index information indicating one of the intra-prediction types. As another example, intra-prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) indicating whether MRL is applied to the current block and, if so, which reference sample line is used; ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether ISP is applied to the current block; ISP type information (e.g., intra_subpartitions_split_flag) indicating the subpartition splitting type if ISP is applied; flag information indicating whether PDCP is applicable; or flag information indicating whether LIP is applicable. In addition, intra-prediction type information may include an MIP flag indicating whether MIP is applied to the current block.

[0135] The intra-prediction mode information and / or intra-prediction type information described above can be encoded / decoded through the coding methods described in this document. For example, the intra-prediction mode information and / or intra-prediction type information described above can be encoded / decoded via entropy coding (e.g., CABAC, CAVLC) based on truncated (rice) binary code.

[0136] On the other hand, when intra-prediction is applied, the intra-prediction mode applied to the current block can be determined using the intra-prediction modes of the surrounding blocks. For example, the decoding device can select one of the mpm candidates in the mpm (most probable mode) list derived based on the intra-prediction modes of the surrounding blocks of the current block (e.g., the surrounding blocks to the left and / or above) and additional candidate modes, based on the received mpm index, or it can select one of the remaining intra-prediction modes not included in the mpm candidates (and planar modes) based on the remaining intra-prediction mode information. The mpm list can be configured to include or exclude planar modes as candidates. For example, if the mpm list includes planar modes as candidates, the mpm list may have 6 candidates, and if the mpm list does not include planar modes as candidates, the mpm list may have 5 candidates. If the mpm list does not include planar mode as a candidate, a not-planar flag (e.g., intra_luma_not_planar_flag) can be signaled to indicate whether the current intra-prediction mode of the block is not planar mode. For example, the mpm flag is signaled first, and the mpm index and not-planar flag can be signaled if the value of the mpm flag is 1. Also, the mpm index can be signaled if the value of the not-planar flag is 1. Here, the reason why the mpm list is configured not to include planar mode as a candidate is not because planar mode is not an mpm, but because planar mode is always considered as an mpm, so the flag (not-planar flag) is signaled first to check whether or not it is planar mode.

[0137] For example, whether the intra-prediction mode currently applied to a block is among the MPM candidates (and planar modes) or in the remaining mode can be indicated by the mpm flag (ex.intra_luma_mpm_flag). A value of 1 for the mpm flag indicates that the intra-prediction mode for the current block is among the MPM candidates (and planar modes), and a value of 0 for the mpm flag indicates that the intra-prediction mode for the current block is not among the MPM candidates (and planar modes). A value of 0 for the not planar flag (ex.intra_luma_not_planar_flag) indicates that the intra-prediction mode for the current block is planar mode, and a value of 1 for the not planar flag indicates that the intra-prediction mode for the current block is not planar mode. The mpm index can be signaled in the form of the syntax element mpm_idx or intra_luma_mpm_idx, and the remaining intra-prediction mode information can be signaled in the form of the syntax element rem_intra_luma_pred_mode or intra_luma_mpm_remainder. For example, the remaining intra-prediction mode information can be an index of the remaining intra-prediction modes from the overall intra-prediction modes that are not included in the mpm candidates (and planar modes), in order of prediction mode number, and point to one of them. The intra-prediction mode may be an intra-prediction mode for a luma component (sample). The intra-prediction mode information below may include at least one of the following: mpm flag (e.g., intra_luma_mpm_flag), not planar flag (e.g., intra_luma_not_planar_flag), mpm index (e.g., mpm_idx or intra_luma_mpm_idx), or remaining intra-prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder).In this document, the MPM list may be referred to by various terms such as the MPM candidate list, the candidate mode list (candModeList), and the candidate intra-predictive mode list.

[0138] Generally, when video is divided into blocks, the current block and surrounding blocks to be coded will have similar video characteristics. Therefore, there is a high probability that the current block and surrounding blocks are identical or have similar intra-prediction modes. Thus, the encoder can use the intra-prediction mode of the surrounding blocks to encode the intra-prediction mode of the current block. For example, an encoder / decoder can construct an MPM (most probable modes) list for the current block. The MPM list can also be referred to as an MPM candidate list. Here, MPM can mean a mode used to improve coding efficiency by considering the similarity between the current block and surrounding blocks when coding the intra-prediction mode.

[0139] Figure 8 shows an example of an intra-prediction mode to which the embodiments described in this document can be applied.

[0140] Referring to Figure 8, we can distinguish between intra-prediction modes with horizontal directionality and intra-prediction modes with vertical directionality, centered around intra-prediction mode 34, which has a prediction direction in the upper left diagonal. In Figure 8, H and V represent horizontal and vertical directionality, respectively, and the numbers -32 to 32 indicate displacement in units of 1 / 32 on the sample grid position. Intra-prediction modes 2 through 33 have horizontal directionality, while intra-prediction modes 34 through 66 have vertical directionality. Intra-prediction modes 18 and 50 represent horizontal intra-prediction modes and vertical intra-prediction modes, respectively. Intra-prediction mode 2 can be called the intra-prediction mode in the lower left diagonal, intra-prediction mode 34 the intra-prediction mode in the upper left diagonal, and intra-prediction mode 66 the intra-prediction mode in the upper right diagonal.

[0141] On the other hand, the intra-prediction mode used for the aforementioned MIP can specify the matrix and offset used for intra-prediction, rather than the existing directional mode. That is, the matrix and offset for intra-prediction can be derived via the intra-mode for MIP. In this case, when deriving the intra-mode for generating the conventional intra-prediction or MPM list described above, the intra-prediction mode for the MIP and the predicted block can be set to a previously configured mode, such as planar mode or DC mode. Alternatively, by another example, the intra-mode for MIP can be mapped to planar mode, DC mode, or directional intra-mode based on the block size.

[0142] Below, we will look at MIP (Matrix-based intra-prediction), one method of intra-prediction.

[0143] As mentioned above, matrix-based intraprediction (MIP) may also be referred to as affine linear weighted intraprediction (ALWIP) or matrix weighted intraprediction (MWIP). To predict samples of a rectangular block having width (W) and height (H), MIP uses one H line from the reconstructed left boundary samples around the block and one W line from the reconstructed upper boundary samples around the block as input values. If the reconstructed samples are not available, reference samples can be generated using the interpolation method applied in normal intraprediction.

[0144] Figure 9 illustrates the MIP-based prediction sample generation procedure for one example. Referring to Figure 9, the MIP procedure is as follows:

[0145] 1. Averaging process

[0146] Of the boundary samples, if W=H=4, 4 samples are extracted by the averaging procedure; in all other cases, 8 samples are extracted.

[0147] 2. Matrix-vector multiplication procedure

[0148] The averaged samples are used as input for matrix-vector multiplication, followed by the addition of an offset. Through this operation, reduced predicted samples can be derived for the subsampled sample set within the original block.

[0149] 3. Interpolation procedure (linear interpolation process)

[0150] Predicted samples for the remaining positions are generated from predicted samples of a sample set subsampled by linear interpolation, which is a single-step linear interpolation in each direction.

[0151] The matrices and offset vectors required to generate prediction blocks or prediction samples can be selected from three sets of matrices: S0, S1, and S2.

[0152] Set S0 consists of 16 matrices A0 i The matrix consists of i∈{0,…,15}, and each matrix has 16 rows, 4 columns, and 16 offset vectors b0 i It can be composed of i ∈ {0, ..., 15}. The matrices and offset vectors of set S0 can be used for a 4x4 block. Another example shows that set S0 can also contain 18 matrices.

[0153] Set S1 consists of 8 Matrix A1 i The matrix consists of i∈{0,…,7}, with each matrix having 16 rows, 8 columns, and 8 offset vectors b1 i It can be composed of i∈{0,…,7}. Another example shows that set S1 can also contain 6 matrices. The matrices and offset vectors of set S1 can be used for blocks of size 4×8, 8×4, and 8×8. Alternatively, the matrices and offset vectors of set S1 can be used for blocks of size 4×H or W×4.

[0154] Finally, set S2 consists of six matrix A2 i The matrix consists of i∈{0,…,5}, with each matrix having 64 rows, 8 columns, and 6 offset vectors b2 iIt can be composed of i∈{0,…,5}. The matrix and offset vector of set S2, or a part thereof, can be used for block forms of all other sizes to which sets S0 and S1 do not apply. For example, the matrix and offset vector of set S2 can be used for operations on blocks with height and width of 8 or more.

[0155] The total number of multiplications required for matrix-vector multiplication is always less than or equal to 4 × W × H. That is, the MIP mode requires a maximum of 4 multiplications per sample.

[0156] The following provides a general overview of the MIP procedure. Any remaining blocks not described below can be handled in one of the four cases described.

[0157] Figures 10 to 13 show MIP procedures based on block size. Figure 10 shows an MIP procedure for a 4x4 block, Figure 11 shows an MIP procedure for an 8x8 block, Figure 12 shows an MIP procedure for an 8x4 block, and Figure 13 shows an MIP procedure for a 16x16 block.

[0158] As shown in Figure 10, given a 4x4 block, MIP takes the average of two samples along each boundary axis. Consequently, the four input samples become the input values ​​for the matrix vector multiplication, and the matrix is ​​taken from set S0. With the offset added, 16 final predicted samples are generated. In the case of a 4x4 block, linear interpolation is not required to generate the predicted samples. Therefore, each sample can be multiplied by (4x16) / (4x4)=4.

[0159] As shown in Figure 11, given an 8x8 block, MIP takes the average of 4 samples along each boundary axis. Consequently, 8 input samples become the input values ​​for matrix-vector multiplication, and the matrix is ​​taken from set S1. Matrix-vector multiplication generates 16 samples at odd positions.

[0160] In the case of an 8x8 block, each sample is multiplied by (8x16) / (8x8)=2 to generate a predictive sample. After adding an offset, the sample is interpolated vertically using the reduced upper boundary sample and horizontally using the original left boundary sample. In this case, since no multiplication operation is required in the interpolation procedure, a total of two multiplication operations are required per sample for MIP.

[0161] As shown in Figure 12, given an 8x4 block, MIP takes the average of 4 samples along the horizontal axis of the boundary, and uses the 4 sample values ​​from the left boundary for the vertical axis. As a result, 8 input samples become the input values ​​for matrix vector multiplication, and the matrix is ​​taken from set S1. Matrix vector multiplication generates 16 samples at odd horizontal positions and corresponding vertical positions.

[0162] In the case of an 8x4 block, each sample is multiplied by (8x16) / (8x4)=4 to generate a prediction sample. After adding an offset, the original left boundary sample is used for horizontal interpolation. In this case, since the interpolation procedure does not require multiplication, a total of 4 multiplication operations are required per sample for MIP.

[0163] As shown in Figure 13, given a 16x16 block, MIP takes the average of 4 samples along each axis, resulting in 8 input samples as input values ​​for matrix vector multiplication, with the matrix taken from set S2. Matrix vector multiplication generates 64 samples at odd positions. For a 16x16 block, each sample is multiplied by (8x64) / (16x16)=2 to generate the predicted samples. After adding the offset, the samples are interpolated vertically using the 8 reduced upper boundary samples and horizontally using the original left boundary samples. In this case, since no multiplication operation is required in the interpolation procedure, a total of 2 multiplication operations are required per sample for MIP.

[0164] For larger blocks, the MIP procedure is essentially identical to the procedure described in detail, and it can be easily confirmed that the number of multiplications per sample is less than 4.

[0165] For W×8 blocks with a width greater than 8 (W>8), only horizontal interpolation is needed because samples are generated at odd horizontal and vertical positions. In this case, a multiplication of (8×64) / (W×8)=64 / W is performed per sample for predicting the reduced samples. When W=16, no additional multiplication is needed for linear interpolation, and when W>16, the number of additional multiplications per sample required for linear interpolation is less than 2. That is, the total number of multiplications per sample is less than or equal to 4.

[0166] Furthermore, in the case of a W×4 block with a width greater than 4 (W>4), the resulting matrix is ​​A, which is obtained by omitting all rows corresponding to odd entries along the horizontal axis of the downsampled block. kLet it be so. Therefore, the size of the output is 32, and only horizontal interpolation is performed. For the prediction operation of the downsampled samples, a multiplication of (8×32) / (W×4)=64 / W per sample is performed. When W = 16, no additional multiplication is required, and when W > 16, the number of additional multiplications per sample required for linear interpolation is less than 2. That is, the total number of multiplications per sample is less than or equal to 4.

[0167] When a matrix is prefixed, it can thereby be processed.

[0168] FIG. 14 is a diagram for explaining the boundary averaging procedure among the MIP procedures. Referring to FIG. 14, the averaging procedure will be specifically described.

[0169] According to the averaging procedure, averaging is applied to each boundary, that is, the left boundary or the upper boundary. Here, the boundary indicates the peripheral reference samples adjacent to the boundary of the current block as shown in FIG. 16. For example, the left boundary (bdry left ) indicates the left peripheral reference samples adjacent to the left boundary of the current block, and the upper boundary (bdry top ) indicates the upper peripheral reference samples adjacent to the upper side.

[0170] When the current block is a 4×4 block, the size of each boundary can be reduced to 2 samples through the averaging procedure. If the current block is not a 4×4 block, the size of each boundary can be reduced to 4 samples through the averaging procedure.

[0171] The first step of the averaging procedure is to reduce the input boundaries (bdry left and bdry top ) to smaller boundaries. JPEG2026050419000002.jpg10127. JPEG2026050419000003.jpg11125 is composed of 2 samples in the case of a 4×4 block and is composed of 4 samples in all other different cases.

[0172] For a 4x4 block, for 0 ≤ i < 2 The mathematical formula for JPEG2026050419000004.jpg13150 is as follows: JPEG2026050419000005.jpg13143 can also be defined in a similar manner.

[0173]

number

[0174] On the other hand, for 0 ≤ i < 4, the width of the block is W = 4 × 2 k Given this, JPEG2026050419000007.jpg14156 can be expressed by the following formula: JPEG2026050419000008.jpg17151 can also be defined in a similar manner.

[0175]

number

[0176] Two reduced boundaries JPEG2026050419000010.jpg12121 is a reduced boundary vector bdry red Since it is linked to bdry red For a 4x4 block, the size is 4, and for all other block types, the size is 8.

[0177] When "mode" refers to the MIP mode, the reduced boundary vector bdry red The range of the MIP mode value can be defined based on the block size and the intra_mip_transposed_flag value, as shown in the formula below.

[0178]

number

[0179] In the above formula, intra_mip_transposed_flag can be referred to as MIP transpose, and such flag information can indicate whether the reduced prediction samples are transposed or not. The semantics for such a syntactic element can be expressed as "intra_mip_transposed_flag[x0][y0] specifies whether the input vector for matrix-based intra prediction mode for luma samples is transposed or not."

[0180] Ultimately, for interpolation of the subsampled predicted samples, a second version of the averaged boundary is needed for larger blocks. That is, if the smaller of the width and height is greater than 8 JPEG2026050419000012.jpg14148 If the width is equal to or greater than the height (W≧H), then W=8*2 l Let 0≦i<8, JPEG2026050419000013.jpg14154 can be defined as shown in the following formula. Also, the smaller of the width and height is greater than 8. JPEG2026050419000014.jpg12142, when height is greater than width (H>W) JPEG2026050419000015.jpg14145 can also be defined in a similar manner.

[0181]

number

[0182] Next, let's look at the procedure for generating reduced prediction samples using matrix-vector multiplication.

[0183] Reduced input vector bdry red One of them is a reduced prediction sample Generate JPEG2026050419000017.jpg12156. The predicted sample is width W. red and height H red This is the signal for the downsampled block. Here, W red and H red It is defined as follows:

[0184]

number

[0185] Reduced prediction sample JPEG2026050419000019.jpg15163 can be calculated by adding an offset after performing matrix-vector multiplication, and can be derived through the following formula.

[0186]

number

[0187] Here, A is W red ×h red A matrix having rows and 4 columns when W and H are 4 (W=H=4), and 8 columns in all other cases, where b is W red ×h red This is a vector of magnitude.

[0188] Matrix A and vector b are selected from sets S0, S1, and S2 as shown below, and the index idx = idx(W,H) can be defined as shown in equation 7 or equation 8.

[0189]

number

[0190]

number

[0191] If idx is less than or equal to 1 (idx ≤ 1), or if idx is 2 and the smaller of W and H is greater than 4, JPEG2026050419000023.jpg14144, A is In JPEG2026050419000024.jpg13137, b is It is set to JPEG2026050419000025.jpg12132. The idx is 2, and the smaller of W and H is 4. In JPEG2026050419000026.jpg11140, if W is 4, then A corresponds to the odd x-coordinates in the downsampled block. This is the matrix obtained by removing each row from JPEG2026050419000027.jpg12139. Alternatively, if H is 4, A corresponds to the odd y coordinates within the downsampled block. This is the matrix obtained by removing each column from JPEG2026050419000028.jpg12141.

[0192] Finally, the reduced prediction sample can be replaced by its own transpose in Equation 9.

[0193]

number

[0194] The number of multiplications needed to calculate JPEG2026050419000030.jpg11137 is 4 when W=H=4, because A consists of 4 columns and 16 rows. In all other cases, A consists of 8 columns and W red ×h red Since it consists of individual rows, To calculate JPEG2026050419000031.jpg10127, we can see that a maximum of 4 multiplications per sample are required.

[0195] FIG. 15 is a diagram for explaining linear interpolation in the MIP procedure. Referring to FIG. 15, the linear interpolation procedure is specifically described as follows.

[0196] The interpolation procedure may be referred to as a linear interpolation or a bilinear interpolation procedure. As shown, the interpolation procedure can include two steps: 1) vertical interpolation and 2) horizontal interpolation.

[0197] If W >= H, vertical linear interpolation can be applied first, followed by horizontal linear interpolation. If W < H, horizontal linear interpolation can be applied first, followed by vertical linear interpolation. In the case of a 4×4 block, the interpolation procedure may be omitted.

[0198] For a W×H block where JPEG2026050419000032.jpg11133, the predicted samples are W red ×H red downsampled predicted samples above derived from JPEG2026050419000033.jpg13149. Depending on the block form, linear interpolation is performed vertically, horizontally, or in both directions. When linear interpolation is applied in both directions, if W < H, it is applied first in the horizontal direction; otherwise, it is applied first in the vertical direction.

[0199] For a W×H block where JPEG2026050419000034.jpg13143 and W >= H, it can be considered that there is no loss of generality. Then, one-dimensional linear interpolation is performed as follows. If there is no loss of generality, the linear interpolation for the vertical direction is fully explained.

[0200] First, the downsampled predicted samples are extended upward by the boundary signal. The coefficient of vertical upsampling is defined as JPEG2026050419000035.jpg13134, By setting JPEG2026050419000036.jpg14130, you can set the extended reduction prediction sample as shown in the following formula.

[0201]

number

[0202] Subsequently, prediction samples for vertical linear interpolation can be generated from these expanded and reduced prediction samples using the following formula.

[0203]

number

[0204] Here, x is 0 ≤ x <W red y is 0 ≤ y <H red , k is 0 ≤ k ver It is possible.

[0205] Below, we will examine methods for reducing the complexity of MIP techniques while maximizing performance. The embodiments described later may be implemented independently or in combination.

[0206] On the other hand, if MIP is applied to the current block, a separate MPM list can be configured for the current block to which the MIP is applied. This MPM list may be called by various names such as MIP MPM list (or LWIP MPM list, candLwipModeList) to distinguish it from the MPM list when ALWIP is not applied to the current block. Hereinafter, for the sake of distinction, it will be referred to as the MIP MPM list, but it may also be called the MPM list.

[0207] ​The MIP MPM list may contain n candidates, for example, n may be 3. The MIP MPM list may be constructed based on the left-side and upper-side surrounding blocks of the current block. Here, the left-side surrounding block may represent the uppermost block among the surrounding blocks adjacent to the left-side boundary of the current block. The upper-side surrounding block may represent the leftmost block among the surrounding blocks adjacent to the upper boundary of the current block.

[0208] For example, if MIP is applied to the left peripheral block, the first candidate intra prediction mode (or candLwipModeA) can be set in the same way as the MIP mode of the left peripheral block. Also, for example, if MIP is applied to the upper peripheral block, the second candidate intra prediction mode (or candLwipModeB) can be set in the same way as the prediction mode of the MIP mode of the upper peripheral block.

[0209] On the other hand, the left peripheral block and the upper peripheral block can be coded based on an intra-prediction other than MIP. That is, when coding the left peripheral block or the upper peripheral block, other intra-prediction types other than MIP can be applied. In this case, it is not appropriate to use the general intra-prediction mode number of the peripheral block to which MIP is not applied (left peripheral block / upper peripheral block) as a candidate intra-mode for the current block to which MIP is applied. Therefore, in this case, as an example, the MIP mode of the peripheral block to which MIP is not applied (left peripheral block / upper peripheral block) can be considered as a prediction mode for a specific value (e.g., 0, 1, or 2). Alternatively, as another example, the general intra-prediction mode of the peripheral block to which MIP is not applied (left peripheral block / upper peripheral block) can be mapped to an MIP mode based on a predetermined mapping table and used to construct the MIP MPM list. In this case, the mapping can be performed based on the block size type of the current block.

[0210] Furthermore, if the surrounding blocks (e.g., left surrounding block / upper surrounding block) are unavailable (e.g., located outside the current picture, located outside the current tile / tile group, etc.), even if MIP is applied, an MIP mode that is not currently available for the block may be used depending on the block size type. In this case, a predefined specific MIP mode can be used as the first candidate intra-prediction mode or the second candidate intra-prediction mode for the first candidate and / or second candidate. Also, a predefined specific MIP prediction mode can be used as the third candidate intra-prediction mode for the third candidate.

[0211] On the other hand, existing MIP modes were divided into non-MPM modes and MPM modes, similar to the derivation method for existing intra-prediction modes, and an MPM flag was sent. Based on whether it was an MPM mode or a non-MPM mode, the MIP mode of the current block was coded.

[0212] For example, a structure can be proposed that directly codes the MIP mode for blocks to which the MIP technique is applied, without distinguishing between MPM mode and non-MPM mode. Such a video coding structure can simplify complex syntax. Furthermore, since the actual occurrence frequency of MIP modes is distributed relatively uniformly for each mode and is clearly different from the occurrence frequency shown for existing intra-modes, the efficiency of encoding and decoding MIP mode information can be maximized through the proposed coding structure.

[0213] The video information transmitted and received for the MIP according to this embodiment is as follows. The syntax described below may be included in the video information transmitted from the encoding device to the decoding device, which can be composed / encoded by the encoding device and signaled to the decoding device in the form of a bitstream, and the decoding device can parse / decode the information (syntax elements) included in the syntax according to the conditions / order disclosed therein.

[0214] [Table 1]

[0215] As shown in Table 1, the syntax for MIP mode, intra_mip_flag and intra_mip_mode_idx, can now be included in the syntax information for the coding unit and signaled within the block.

[0216] A value of 1 for `intra_mip_flag` indicates that the intra-prediction type for LumaSample is matrix-based intra-prediction, while a value of 0 indicates that the intra-prediction type for LumaSample is not matrix-based intra-prediction.

[0217] When signaled with intra_mip_flag = 1, intra_mip_mode_idx indicates a matrix-based intra-prediction mode for the luma sample. Such a matrix-based intra-prediction mode can indicate the matrix and offset or matrix for MIP, as described above.

[0218] Furthermore, for example, a flag information, such as intra_mip_transposed_flag, can be signaled through the coding unit's syntax to indicate whether or not the input vector for matrix-based intra prediction is transposed. If intra_mip_transposed_flag is 1, the input vector for matrix-based intra prediction is transposed, and such flag information can reduce the number of matrices for matrix-based intra prediction.

[0219] On the other hand, intra_mip_mode_idx can be encoded and decoded using the truncated binarization method, as shown in the table below.

[0220] [Table 2]

[0221] As shown in Table 2, intra_mip_flag is binary-coded into a Fixed Length Code, while intra_mip_mode_idx is binary-coded using a truncated binary method, and the maximum binary length (cMax) can be set by the size of the coding block. The maximum binary length is set to 34 if the width and height of the coding block are 4 (cbWidth==4 && cbHeight==4), and otherwise it can be set to 18 or 10 depending on whether the width and height of the coding block are 8 or less ((cbWidth<=8 && cbHeight<=8)?).

[0222] On the other hand, intra_mip_mode_idx can be coded using a bypass method rather than being based on the context model. By coding using a bypass method, coding speed and efficiency can be increased.

[0223] In another example, when intra_mip_mode_idx is converted to binary using truncated binary evolution, the maximum length of the binary is as shown in the following table.

[0224] [Table 3]

[0225] As shown in Table 3, the maximum binary length for intra_mip_mode_idx is set to 15 if the width and height of the coding block are 4 ((cbWidth==4 && cbHeight==4)), otherwise it is set to 7 if the width or height of the coding block is 4 ((cbWith==4||cbHeight==4)) or if the width and height of the coding block are 8 (cbWith==8 && cbHeight==8), and it can be set to 5 if the width or height of the coding block is not 4 ((cbWith==4||cbHeight==4)) or if the width and height of the coding block are not 8 (cbWith==8 && cbHeight==8).

[0226] In another example, intra_mip_mode_idx can be encoded with a fixed-length code. In this case, to improve encoding efficiency, the number of available MIP modes is limited to an exponential of 2 for each block size (for example, A=2). K1 -1, B=2 K2 -1, 2 K3 -1 (where K1, K2, and K3 are positive constants) is also possible.

[0227] This can be shown in the following table.

[0228] [Table 4]

[0229] In Table 4, if K1=5, K2=4, and K3=3, intra_mip_mode can be binary-encoded as follows:

[0230] [Table 5]

[0231] Alternatively, by way of example, if K1 is set to 4, the maximum length of the binary evolution for the intra_mip_mode of a block with a coding block width and height of 4 can be set to 15. Also, if K2 is set to 3, when the width or height of the coding block is 4 or the width and height of the coding block are 8, the maximum length of the binary evolution can be set to 7.

[0232] On the other hand, by way of example, a method of using MIP only for specific blocks where the MIP technique can be efficiently applied can be proposed. When the method according to this embodiment is applied, the number of matrix vectors required for MIP decreases, and the memory required to store the matrix vectors can be significantly reduced (by 50%). Despite such an effect, the coding efficiency is maintained almost constant (less than 0.1%).

[0233] The syntax including the specific conditions under which the MIP technique is applied according to this embodiment is as follows in the following table.

[0234]

Table 6

[0235] As shown in Table 6, a condition (cbWidth>K1||cbHeight>K2) is added so that MIP is only applied to large blocks, and the size of the block can be determined by preset values (K1 and K2). The reason for applying MIP only to large blocks is that the coding efficiency of MIP is shown in relatively large blocks.

[0236] The following table shows an example where K1 and K2 are predefined as 8 in Table 6.

[0237]

Table 7

[0238] The semantics for intra_mip_flag and intra_mip_mode_idx in Tables 6 and 7 are the same as in Table 1.

[0239] On the other hand, when intra_mip_mode_idx has 11 possible modes, it can be encoded in a truncated binary evolution (cMax = 10) manner as follows.

[0240] [Table 8]

[0241] Alternatively, when the available MIP modes are limited to 8, intra_mip_mode_idx[x0][y0] can be encoded with a fixed-length code as follows.

[0242] [Table 9]

[0243] Both Tables 8 and 9 show that intra_mip_mode_idx can be coded in a bypass manner.

[0244] On the other hand, by way of an example, a weighted value matrix (A k ) and an offset vector (b k ) used for a large block so that the MIP technique can be efficiently applied from the perspective of memory saving can be proposed for a small block. When the method according to this embodiment is applied, the number of matrix vectors required for MIP can be reduced, and the memory required to store the matrix vectors can be significantly reduced (by 50%). While such an effect is accompanied, the coding efficiency is maintained almost constant (less than 0.05%).

[0245] FIG. 16 is a diagram for explaining the MIP technique according to an example of this document.

[0246] As shown, Figure 16(a) shows the matrix and offset vector operations for index i of the larger block, and Figure 16(b) shows the sampled matrix and offset vector operations applied to the smaller block.

[0247] As shown in Figure 16, the weighted matrix used for the larger block was subsampled to obtain the subsampled weighted matrix (Sub(A k )) and the offset vector obtained by subsampling the offset vector used for the large block (Sub(b k By considering )) as a weighted matrix and offset vector for smaller blocks, respectively, existing MIP procedures can be applied.

[0248] Here, subsampling may be applied to only one of the horizontal or vertical directions, or to both directions. In particular, the subsampling factor (e.g., 1 out of 2, or 1 out of 4) and the sampling direction (vertical or horizontal) can be set based on the width and height of the block in question.

[0249] Furthermore, the number of intra-prediction modes for the MIP to which this embodiment applies can be set differently based on the size of the current block. For example, i) if the height and width of the current block (coding block or transformation block) are both 4, then 35 intra-prediction modes (i.e., intra-prediction modes 0 to 34) may be available; ii) if both the height and width of the current block are 8 or less, then 19 intra-prediction modes (i.e., intra-prediction modes 0 to 18) may be available; and iii) in other cases, then 11 intra-prediction modes (i.e., intra-prediction modes 0 to 10) may be available.

[0250] For example, when the height and width of the current block are both 4, it is called block size type 0; when the height and width of the current block are both 8 or less, it is called block size type 1; and when it is any other case, it is called block size type 2. In this case, the number of intra prediction modes for MIP can be organized as shown in the following table.

[0251]

Table 10

[0252] To apply the weighted value matrix and offset vector used for a large block (e.g., block size type = 2) to a small block (e.g., block size = 0 or block size = 1), the number of available intra prediction modes for each block size can be applied in the same way as shown in the following table.

[0253]

Table 11

[0254] Alternatively, as shown in Table 12 below, MIP is applied only to block size types 1 and 2. For block size type 1, the weighted value matrix and offset vector defined for block size type 2 can be subsampled and used. Through this, memory can be efficiently saved (50%).

[0255]

Table 12

[0256] The following drawings are created to illustrate a specific example of this specification. Since the names of specific devices and the names of specific signals / messages / fields described in the drawings are presented exemplarily, the technical features of this specification are not limited to the specific names used in the following drawings.

[0257] The table below shows the experimental results when MIP is applied to the current block as in the embodiment described above, omitting the derivation of the MPM and also not signaling the signals associated with the MPM.

[0258] The experiment was conducted based on the VTM 5.0 reference software under the standard test conditions defined in JVET-N1010.

[0259] [Table 13]

[0260] [Table 14]

[0261] [Table 15]

[0262] Figure 17 is a flowchart illustrating a decoding method that can be performed by a decoding device according to one embodiment of this document.

[0263] The method disclosed in Figure 17 can be performed by the decoding device 200 disclosed in Figure 2. Specifically, steps S1700 to S1750 in Figure 17 can be performed by the entropy decoding unit 210 and / or prediction unit 230 (specifically, the intra-prediction unit 231) disclosed in Figure 2, and step S1760 in Figure 17 can be performed by the addition unit 240 disclosed in Figure 2. Furthermore, the method disclosed in Figure 17 may include embodiments described above in this document. Therefore, in Figure 17, specific explanations of content that overlaps with the embodiments described above will be omitted or simplified.

[0264] Referring to Figure 17, the decoding device can receive, i.e., obtain from the bitstream, flag information indicating whether or not matrix-based intra prediction (MIP) is used for the current block (S1700).

[0265] Such flag information can be signaled by including it in the syntax information of the coding unit using a syntax like intra_mip_flag.

[0266] The decoding device can receive matrix-based intra-predictive (MIP) mode information based on the received flag information (S1710).

[0267] MIP mode information can be represented by intra_mip_mode_idx, which can be signaled if intra_mip_flag is 1. intra_mip_mode_idx can be index information indicating the MIP mode currently applied to a block, and such index information can be used to derive the matrix when generating prediction samples.

[0268] Additionally, for example, the coding unit syntax may be further signaled with flag information, such as intra_mip_transposed_flag, indicating whether or not the input vector for matrix-based intra prediction is transposed.

[0269] The decoding device can generate intra-predictive samples for the current block based on MIP information. To generate intra-predictive samples, the decoding device can derive at least one peripheral reference sample of the current block's peripheral reference samples, and can generate predictive samples based on the peripheral reference samples.

[0270] The decoding device can decode the binstring of the syntax element for the MIP mode information using a truncated binary evolution method and a bypass method (S1720).

[0271] As mentioned above, the maximum length of the binstring of the syntax element for MIP mode information can be set to different values ​​depending on the current block size. Such a maximum length can be set to three values ​​depending on the current block size, as shown in Table 2 or Table 3, and the maximum length may be largest when the current block width and height are 4.

[0272] For example, in Table 2, if the width and height of the coding block are 4, the maximum length of the binary code is set to 34, and in Table 3, it can be set to 15.

[0273] Furthermore, this MIP mode information can be decoded using a bypass method rather than a context modeling method.

[0274] On the other hand, intra_mip_flag can be binary-coded using a fixed-length code.

[0275] When MIP is applied, the decoding device can downsample the reference sample adjacent to the current block to derive a reduced boundary sample (S1730).

[0276] The reduced boundary samples can be derived by downsampling through averaging of the reference samples.

[0277] When the boundary samples are reduced, if the current block width and height are 4, 4 samples can be derived, and in the remaining other case, 8 samples can be derived.

[0278] The averaging procedure for downsampling can be applied to each boundary, left boundary, or upper boundary of the current block, and this can be applied to the surrounding reference samples adjacent to the boundaries of the current block.

[0279] For example, if the current block is a 4x4 block, the size of each boundary can be reduced to 2 samples through the averaging procedure, and if the current block is not a 4x4 block, the size of each boundary can be reduced to 4 samples through the averaging procedure.

[0280] Subsequently, the decoding device can derive reduced predicted samples based on a multiplication operation between the MIP matrix derived based on the current block size and index information and the reduced boundary samples (S1740).

[0281] The MIP matrix can be derived based on the current block size and the received index information.

[0282] The MIP matrix can be selected from one of three matrix sets, which are currently classified according to the size of the blocks, and each of the three matrix sets can contain multiple MIP matrices.

[0283] In other words, three matrix sets can be set up for MIP, and each matrix set can consist of multiple matrices and offset vectors. Such matrix sets can now be divided and applied according to the size of the block.

[0284] For example, a 4x4 block can be fitted with a set of matrices containing 18 or 16 matrices, each consisting of 16 rows and 4 columns, and 18 or 16 offset vectors. Index information can indicate any of the multiple matrices contained within a single set of matrices.

[0285] For 4×8, 8×4, and 8×8 blocks, or 4×H or W×4 blocks, a matrix set including 10 or 8 matrices composed of 16 rows and 8 columns, and 10 or 8 offset vectors can be applied.

[0286] Alternatively, for blocks other than the aforementioned blocks or blocks with a height and width of 8 or more, a matrix set including 6 matrices composed of 64 rows and 8 columns, and 6 offset vectors can be applied.

[0287] After the operation of multiplying the MIP matrix by the reduced boundary samples, a reduced predicted sample, that is, a predicted sample to which the MIP matrix is applied, is derived based on the operation of adding an offset.

[0288] The decoding device can upsample the reduced predicted sample to generate an intra prediction sample for the current block (S1750).

[0289] The intra prediction sample can be upsampled by linear interpolation of the reduced predicted sample.

[0290] The interpolation procedure can be referred to as a linear interpolation or bilinear interpolation procedure and can include two steps of 1) vertical interpolation and 2) horizontal interpolation.

[0291] If W>=H, vertical linear interpolation can be applied first, followed by horizontal linear interpolation. If W<H, horizontal linear interpolation can be applied first, followed by vertical linear interpolation. In the case of a 4×4 block, the interpolation procedure can be omitted.

[0292] The decoding device can generate a restored sample for the current block based on the predicted sample (S1760).

[0293] In one embodiment, the decoding device may immediately use the predicted sample as the reconstructed sample depending on the prediction mode, or it may generate a reconstructed sample by adding a residual sample to the predicted sample.

[0294] The decoding device can receive information about the residual for the current block if a residual sample exists for the current block. The information about the residual may include conversion factors for the residual sample. Based on the residual information, the decoding device can derive a residual sample (or residual sample array) for the current block. Based on the predicted sample and the residual sample, the decoding device can generate a reconstructed sample, and based on the reconstructed sample, can derive a reconstructed block or reconstructed picture. Subsequently, as described above, the decoding device may apply deblocking filtering and / or in-loop filtering procedures such as the SAO procedure to the reconstructed picture to improve subjective / objective image quality as needed.

[0295] Figure 18 is a flowchart illustrating a schematic encoding method that can be performed by an encoding device according to one embodiment of this document.

[0296] The method disclosed in Figure 18 can be performed by the encoding device 100 disclosed in Figure 1. Specifically, steps S1800 to S1830 in Figure 18 can be performed by the prediction unit 120 (specifically the intra-prediction unit 122) disclosed in Figure 1, step S1840 in Figure 18 can be performed by the subtraction unit 131 disclosed in Figure 1, and steps S1850 and S1860 in Figure 18 can be performed by the entropy encoding unit 140 disclosed in Figure 1. Furthermore, the method disclosed in Figure 18 may include embodiments described above in this document. Therefore, in Figure 18, specific explanations of content that overlaps with the embodiments described above will be omitted or simplified.

[0297] Referring to Figure 18, the encoding device can derive whether or not matrix-based intra prediction (MIP) is applied to the current block (S1800).

[0298] The encoding device can apply a variety of prediction techniques to find the optimal prediction mode for the current block, and can determine the optimal intra-prediction mode based on RDO (rate-distortion optimization).

[0299] If it is determined that MIP is to be applied to the current block, the encoding device can downsample the reference samples adjacent to the current block to derive reduced boundary samples (S1810).

[0300] The reduced boundary samples can be derived by downsampling through averaging of the reference samples.

[0301] When the boundary samples are reduced, if the current block width and height are 4, 4 samples can be derived, and in the remaining other case, 8 samples can be derived.

[0302] The averaging procedure for downsampling can be applied to each boundary, left boundary, or upper boundary of the current block, and this can be applied to the surrounding reference samples adjacent to the boundaries of the current block.

[0303] For example, if the current block is a 4x4 block, the size of each boundary can be reduced to 2 samples through the averaging procedure, and if the current block is not a 4x4 block, the size of each boundary can be reduced to 4 samples through the averaging procedure.

[0304] Once the reduced boundary samples are derived, the encoding device can derive reduced predicted samples based on the multiplication of the MIP matrix, which is selected based on the current block size, with the reduced boundary samples (S1820).

[0305] The MIP matrix can be selected from one of three matrix sets, which are currently classified according to the size of the blocks, and each of the three matrix sets can contain multiple MIP matrices.

[0306] In other words, three matrix sets can be set up for MIP, and each matrix set can consist of multiple matrices and offset vectors. Such matrix sets can now be divided and applied according to the size of the block.

[0307] For example, a 4x4 block can be fitted with a set of matrices containing 18 or 16 matrices, each consisting of 16 rows and 4 columns, and 18 or 16 offset vectors. Index information can indicate any of the multiple matrices contained within a single set of matrices.

[0308] A set of matrices can be applied to 4x8, 8x4, and 8x8 blocks, or 4xH or Wx4 blocks, which can consist of 10 or 8 matrices with 16 rows and 8 columns, and 10 or 8 offset vectors.

[0309] Alternatively, for blocks other than those mentioned above, or for blocks with a height and width of 8 or more, a matrix set containing six matrices consisting of 64 rows and 8 columns, and six offset vectors can be applied.

[0310] After performing a multiplication operation between the MIP matrix and the downsampled boundary samples, a downsampled predicted sample, i.e., a predicted sample to which the MIP matrix has been applied, can be derived based on an operation of adding an offset.

[0311] Thereafter, the encoding device can upsample the downsampled predicted sample to generate an intra-predicted sample for the current block (S1830).

[0312] The intra-predicted sample can be upsampled by linear interpolation of the downsampled predicted sample.

[0313] The interpolation procedure can be referred to as a linear interpolation or a bilinear interpolation procedure and can include two steps: 1) vertical interpolation, and 2) horizontal interpolation.

[0314] If W >= H, vertical linear interpolation can be applied first, followed by horizontal linear interpolation. If W < H, horizontal linear interpolation can be applied first, followed by vertical linear interpolation. In the case of a 4×4 block, the interpolation procedure can be omitted.

[0315] Also, the encoding device can derive a residual sample for the current block based on the predicted sample of the current block and the original sample of the current block (S1840).

[0316] Then, the encoding device can evolve the bit string of the syntax element for the MIP mode information in a truncated binary evolution method and encode it in a bypass mode (S1850).

[0317] As described above, the maximum length of the bit string of the syntax element for the MIP mode information can be set to different values according to the size of the current block. Such a maximum length can be set to three values according to the size of the current block as in Table 2 or Table 3, and may be the largest when the width and height of the current block are 4.

[0318] For example, in Table 2, if the width and height of the coding block are 4, the maximum length of the binary code can be set to 34, and in Table 3, it can be set to 15.

[0319] Furthermore, this type of MIP mode information can be encoded using a bypass method rather than a context modeling method.

[0320] On the other hand, intra_mip_flag can be binary-coded using a fixed-length code.

[0321] The encoding device generates residual information for the current block based on the residual sample and can output video information in bitstream form, including the generated residual information, flag information indicating whether MIP is applied, and MIP mode information (S1860).

[0322] Here, the residual information may include information such as the values ​​of the quantized conversion coefficients derived from the conversion and quantization of the residual sample, position information, conversion technique, conversion kernel, and quantization parameters.

[0323] Flag information indicating whether or not MIP is applied can be encoded in the syntax information of the coding unit using a syntax such as intra_mip_flag.

[0324] Furthermore, MIP mode information can be represented by intra_mip_mode_idx and can be encoded when intra_mip_flag is 1. intra_mip_mode_idx can be index information indicating the MIP mode currently applied to a block, and such index information can be used to derive a matrix when generating prediction samples. The index information can indicate any of the multiple MIP matrices contained in a single matrix set.

[0325] Additionally, for example, a flag information, such as intra_mip_transposed_flag, can be signaled via the coding unit's syntax to indicate whether or not the input vector for matrix-based intra prediction is transposed.

[0326] In other words, the encoding device can encode video information including the MIP mode information and / or residual information of the current block as described above, and output it to a bitstream.

[0327] The bitstream can be transmitted to a decoding device via a network or (digital) storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include a variety of storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc.

[0328] The process of generating predicted samples for the current block described above can be performed by the intra-prediction unit 122 of the encoding device 100 disclosed in Figure 1, the process of deriving residual samples can be performed by the subtraction unit 131 of the encoding device 100 disclosed in Figure 1, and the process of generating and encoding residual information can be performed by the residual processing unit 130 and the entropy encoding unit 140 of the encoding device 100 disclosed in Figure 1.

[0329] In the embodiments described above, the method is explained based on a flowchart as a series of steps or blocks; however, the embodiments in this document are not limited to the order of the steps, and some steps may occur in a different order or simultaneously with other steps than those described above. Furthermore, those skilled in the art will understand that the steps shown in the flowchart are not exclusive, and that different steps may be included, or one or more steps in the flowchart may be deleted without affecting the scope of this document.

[0330] The methods described in this document above can be implemented in software form, and the encoding and / or decoding devices described in this document may be included in devices that perform video processing, such as TVs, computers, smartphones, set-top boxes, and display devices.

[0331] In this document, when embodiments are implemented in software, the methods described above can be implemented by modules (processes, functions, etc.) that perform the functions described above. These modules are stored in memory and can be executed by a processor. The memory may be internal or external to the processor and may be connected to the processor by various well-known means. The processor may include an ASIC (application-specific integrated circuit), other chipsets, logic circuits, and / or data processing devices. The memory may include ROM (read-only memory), RAM (random access memory), flash memory, memory cards, storage media, and / or other storage devices. That is, the embodiments described in this document can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each drawing can be implemented and executed on a computer, processor, microprocessor, controller, or chip. In this case, information on instructions or algorithms for implementation may be stored on a digital storage medium.

[0332] Furthermore, decoding and encoding devices to which this document applies may include multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video interaction devices, real-time communication devices such as video communications, mobile streaming devices, storage media, camcorders, customized video (VoD) service providers, OTT video (Over the Top Video) devices, internet streaming service providers, 3D video devices, VR (virtual reality) devices, AR (argumente reality) devices, image phone video devices, transportation terminals (e.g., vehicle terminals (including autonomous vehicles), airplane terminals, ship terminals, etc.), and medical video equipment, and can be used to process video signals or data signals. For example, OTT video (Over the Top Video) devices may include game consoles, Blu-ray players, internet access TVs, home theater systems, smartphones, tablet PCs, DVRs (Digital Video Recorders), etc.

[0333] Furthermore, the processing methods to which this document applies can be produced in the form of programs executed on a computer and stored on a computer-readable recording medium. Multimedia data having the data structure described herein can also be stored on a computer-readable recording medium. The computer-readable recording medium includes all types of storage devices and distributed storage devices on which computer-readable data is stored. The computer-readable recording medium may include, for example, Blu-ray discs (BDs), general-purpose serial buses (USBs), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, and optical data storage devices. The computer-readable recording medium may also include media embodied in the form of carrier waves (e.g., transmission over the Internet). Furthermore, a bitstream generated by an encoding method may be stored on a computer-readable recording medium or transmitted over a wireless network.

[0334] Furthermore, embodiments of this document can be embodied in computer program products comprising program code, and said program code can be executed on a computer according to the embodiments of this document. The said program code can be stored on a computer-readable carrier.

[0335] Figure 19 schematically shows an example of a video / image coding system that can be applied to the embodiments described herein.

[0336] Referring to Figure 19, a video / image coding system may include a first device (source device) and a second device (receiving device). The source device can transmit encoded video / image information or data to the receiving device in the form of a file or streaming via a digital storage medium or network.

[0337] The source device may include a video source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be called a video / image encoding device, and the decoding device may be called a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, which may consist of a separate device or external component.

[0338] A video source can acquire video / images through processes such as video / image capture, synthesis, or generation. A video source may include video / image capture devices and / or video / image generation devices. Video / image capture devices may include, for example, one or more cameras, or video / image archives containing previously captured video / images. Video / image generation devices may include, for example, computers, tablets, and smartphones, which can generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by the process of generating the associated data.

[0339] An encoding device can encode input video / image data. For compression and coding efficiency, the encoding device can perform a series of steps, including prediction, transformation, and quantization. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0340] The transmitting unit can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit of a receiving device via a digital storage medium or network in the form of a file or streaming. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitting unit may include elements for generating media files via a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiving unit can receive / extract the bitstream and transmit it to a decoding device.

[0341] A decoding device can decode video / images by performing a series of steps, such as inverse quantization, inverse transformation, and prediction, corresponding to the operation of an encoding device.

[0342] The renderer can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0343] Figure 20 shows an example of a content streaming system to which the embodiments disclosed in this document can be applied.

[0344] Referring to Figure 20, the content streaming system applicable to the embodiments described in this document may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0345] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and then transmitting this bitstream to the streaming server. As an alternative, if a multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server may be omitted.

[0346] The bitstream can be generated by an encoding method or a bitstream generation method applicable to the embodiments of this document, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0347] The streaming server transmits multimedia data to user devices based on user requests via a web server, and the web server acts as an intermediary to inform users about available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server transmits multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server controls the commands and responses between the devices within the content streaming system.

[0348] The streaming server can receive content from media storage and / or encoding servers. For example, if it starts receiving content from the encoding server, it can receive the content in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0349] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (such as smartwatches, smart glasses, and HMDs), digital TVs, desktop computers, and digital signage.

[0350] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received by each server can be processed in a distributed manner.

[0351] The claims described in this document can be combined in various ways. For example, the technical features of the method claims in this document can be combined to be implemented or performed in an apparatus, and the technical features of the apparatus claims in this document can be combined to be implemented or performed in a method. Furthermore, the technical features of the method claims and the technical features of the apparatus claims in this document can be combined to be implemented or performed in an apparatus, and the technical features of the method claims and the technical features of the apparatus claims in this specification can be combined to be implemented or performed in a method.

Claims

1. In a video decoding method performed by a decoding device, Steps to obtain residual information from a bitstream, The steps include obtaining flag information from the bitstream related to whether MIP (matrix-based intraprediction) is currently used for the block, A step of obtaining MIP mode information based on the value of the flag information being equal to 1, wherein the value of the flag information being equal to 1 is related to the fact that the MIP is used for the current block. The steps include: deriving the MIP matrix based on the MIP mode information and the current block size; The steps include generating intra-prediction samples for the current block based on the MIP matrix, The steps include: deriving a conversion coefficient based on the residual information; A step of generating a residual sample based on the conversion coefficient, The step of generating a reconstructed sample for the current block based on the intra-predicted sample and the residual sample, The MIP mode information is index information related to the MIP matrix applied to the current block, The binstring of the syntax element for the aforementioned MIP mode information is binary-evolved using the TB (truncated binary) binary evolution method. The maximum length of the binstring of the syntax element is: i) Whether the width and height of the current block are 4, ii) Whether the width and height of the current block are 8 or less, iii) Based on whether the width and height of the current block are greater than 8, they are set to three different values. A method in which, based on the fact that the width and height of the current block are 4, the maximum length has the largest value.

2. In a video encoding method performed by an encoding device, The steps include: deriving whether MIP (matrix-based intra prediction) is applicable to the current block, The steps include: deriving an MIP matrix based on the application of the MIP to the current block; The steps include generating intra-prediction samples for the current block based on the MIP matrix, The steps include generating flag information related to whether the MIP is applied to the current block, The steps include: deriving a residual sample for the current block based on the intra-predicted sample; The steps include: deriving conversion coefficients based on the aforementioned residual sample; A step of generating residual information based on the aforementioned conversion coefficient, The step of encoding video information including the flag information and the residual information, Based on the fact that the value of the flag information is equal to 1, the video information includes MIP mode information, The value of the flag information equal to 1 is related to the fact that the MIP is applied to the current block, The MIP mode information is index information related to the MIP matrix applied to the current block, The binstring of the syntax element for the aforementioned MIP mode information is binary-evolved using the TB (truncated binary) binary evolution method. The maximum length of the binstring of the syntax element is: i) Whether the width and height of the current block are 4, ii) Whether the width and height of the current block are 8 or less, iii) Based on whether the width and height of the current block are greater than 8, they are set to three different values. Based on the fact that the width and height of the current block are 4, the maximum length has the largest value, A method for deriving an MIP matrix based on the MIP mode information and the current block size.

3. Regarding methods for transmitting video data, A step of obtaining a bitstream relating to the video, wherein the bitstream is The steps include: deriving whether MIP (matrix-based intra prediction) is applicable to the current block, The steps include: deriving an MIP matrix based on the application of the MIP to the current block; The steps include generating intra-prediction samples for the current block based on the MIP matrix, The steps include generating flag information related to whether the MIP is applied to the current block, The steps include: deriving a residual sample for the current block based on the intra-predicted sample; The steps include: deriving conversion coefficients based on the aforementioned residual sample; A step of generating residual information based on the aforementioned conversion coefficient, A step of encoding video information including the flag information and the residual information, and a step of generating based on, The step of transmitting the data, which includes the bitstream, Based on the fact that the value of the flag information is equal to 1, the video information includes MIP mode information, The value of the flag information equal to 1 is related to the fact that the MIP is applied to the current block, The MIP mode information is index information related to the MIP matrix applied to the current block, The binstring of the syntax element for the aforementioned MIP mode information is binary-evolved using the TB (truncated binary) binary evolution method. The maximum length of the binstring of the syntax element is: i) Whether the width and height of the current block are 4, ii) Whether the width and height of the current block are 8 or less, iii) Based on whether the width and height of the current block are greater than 8, they are set to three different values. Based on the fact that the width and height of the current block are 4, the maximum length has the largest value, A method for deriving an MIP matrix based on the MIP mode information and the current block size.

Images