Matrix-based intra-prediction method, device, and recording medium

The matrix-based intra-frame prediction method improves video encoding and decoding accuracy by using matrix multiplication of restored samples around the current block, addressing the limitations of traditional spatial prediction methods.

WO2026235384A1PCT designated stage Publication Date: 2026-11-12KWANGWOON UNIVERSITY INDUSTRY ACADEMIC COLLABORATION FOUNDATION
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/KR2026/005947
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-09
Filing Date
2026-05-08
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

Existing video encoding and decoding technologies face challenges in improving the accuracy of intra-frame prediction, particularly in determining the predictive pixel values within a single frame using spatial prediction methods.

Method used

The method employs matrix-based intra-frame prediction by performing matrix multiplication using restored samples around the current block, utilizing reference values from adjacent or non-adjacent regions within the frame, and generating a prediction signal through matrix operations.

Benefits of technology

This approach enhances the accuracy of intra-frame prediction by leveraging matrix-based methods, improving the efficiency and precision of video encoding and decoding processes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure KR2026005947_12112026_PF_FP_ABST
    Figure KR2026005947_12112026_PF_FP_ABST
Patent Text Reader

Abstract

An image encoding / decoding method, device, and recording medium of the present disclosure may comprise the steps of: acquiring, from a bitstream, intra-prediction information for a current block; and generating a prediction signal for the current block on the basis of the intra-prediction information, wherein the intra-prediction information comprises information indicating whether a matrix-based intra-prediction mode is applied to the current block, and when the matrix-based intra-prediction mode is applied, the prediction signal may be generated on the basis of a multiplication operation between an input vector including a plurality of reference values obtained from reconstructed samples around the current block and a matrix used for prediction of the current block.
Need to check novelty before this filing date? Find Prior Art

Description

Matrix-based intra-frame prediction method, device, and recording medium

[0001] The present invention relates to video encoding and decoding technology, and more specifically, to a method and apparatus for performing intra-frame prediction based on matrix multiplication of restored samples around the current block.

[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art. In order to efficiently record or transmit still images or video to a storage medium, it is necessary to encode the image signal. Various methods have been proposed to improve encoding efficiency, the most representative of which are methods using temporal prediction and methods using spatial prediction.

[0003] Temporal prediction involves detecting the predictive block with the smallest residual coefficients for the current frame's object block from other frames temporally adjacent to the current frame (picture), and is also known as inter-frame prediction. Spatial prediction involves calculating the predicted pixel value of the object block using the reconstructed pixel values ​​of reference blocks that are adjacent or non-adjacent to the object block within a single frame, and is also known as intra-frame prediction.

[0004] The present invention may aim to perform intra-frame prediction for inter-frame prediction blocks and intra-frame prediction blocks in a video sub / decoder using information from previously restored regions adjacent to or non-adjacent to various current blocks.

[0005] The image encoding / decoding method, apparatus, and recording medium of the present disclosure include: a step of obtaining intra-frame prediction information for a current block from a bitstream; and a step of generating a prediction signal for the current block based on the intra-frame prediction information, wherein the intra-frame prediction information includes information indicating whether a matrix-based intra-frame prediction mode is applied to the current block, and if the matrix-based intra-frame prediction mode is applied, the prediction signal may be generated based on a multiplication operation between an input vector including a plurality of reference values ​​obtained from reconstructed samples around the current block and a matrix used for prediction of the current block.

[0006] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the plurality of reference values ​​may be obtained based on at least one of the top reference sample, left reference sample, top-left reference sample, top-right reference sample, or bottom-left reference sample of the current block.

[0007] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, at least one of the plurality of reference values ​​may be obtained based on the average value of a plurality of previously restored samples around the current block.

[0008] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the matrix may be determined based on matrix information that specifies the matrix to be used for predicting the current block.

[0009] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the matrix information may include index information obtained from the bitstream.

[0010] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the matrix may be determined based on at least one of the size of the current block or the aspect ratio of the current block.

[0011] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, a plurality of prediction samples are generated based on the multiplication operation, and a prediction signal for the current block can be generated based on the plurality of prediction samples.

[0012] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the prediction signal for the current block may be generated based on interpolation or resampling of the plurality of prediction samples.

[0013] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, whether the matrix-based intra-frame prediction mode is allowed may be determined based on at least one of whether the intra-frame prediction mode of the current block or the palette mode is applied to the current block.

[0014] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the matrix-based intra-frame prediction mode may be allowed when the intra-frame prediction mode of the current block is a DC mode and a palette mode is not applied to the current block.

[0015] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the intra-frame prediction information includes transposition information for determining the arrangement order of the plurality of reference values ​​included in the input vector when the matrix-based intra-frame prediction mode is applied, and the arrangement order of the plurality of reference values ​​included in the input vector can be determined based on the transposition information.

[0016] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the matrix information may include at least one of matrix set information in which a plurality of matrices are defined or index information indicating a matrix within the matrix set.

[0017] According to one embodiment of the present invention, the accuracy of the prediction signal can be improved by performing matrix-based intra-frame prediction using adjacent or non-adjacent reconstructed regions around the current block in a video sub / decoder.

[0018] FIG. 1 is an exemplary block diagram of a video encoding device capable of implementing the technologies of the present disclosure.

[0019] FIG. 2 is a block diagram showing in detail a part of a video decoding device according to one embodiment of the present disclosure.

[0020] Figure 3 is a diagram illustrating a method for dividing blocks.

[0021] Figure 4 is a diagram showing multiple directional in-frame prediction modes.

[0022] Figure 5 is a diagram of the entire or partial process of parsing in-screen prediction mode information.

[0023] Figure 6 is a diagram of information parsing and prediction execution for performing prediction in the DIP mode proposed in the present invention.

[0024] FIG. 7 is a diagram showing an example for explaining the reference sample acquisition step of FIG. 6 proposed in the present invention.

[0025] FIG. 8 is a diagram showing an example for performing matrix multiplication operations proposed in the present invention.

[0026] FIG. 9 is a diagram showing an example of a process for generating a prediction signal from a value obtained by performing matrix multiplication as proposed in the present invention.

[0027] FIG. 10 is a diagram showing an example of a process for generating a prediction signal from a value obtained by performing a matrix multiplication operation proposed in the present invention.

[0028] FIG. 11 is an example showing a block diagram of the process in which the inverse transformation of the current block is performed in the decoder.

[0029] FIG. 12 is a diagram showing an example of a process for determining a kernel based on directionality derived during the kernel candidate list construction process when the current block proposed in the present invention is predicted in DIP mode.

[0030] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In describing the embodiments of this specification, detailed descriptions of related known configurations or functions are omitted if it is determined that such detailed descriptions may obscure the gist of this specification.

[0031] In the present invention, the description of a specific configuration as "including" does not exclude configurations other than said configuration, and means that additional configurations may be included within the scope of the practice of the present invention or the technical concept of the present invention.

[0032] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component.

[0033] Furthermore, the components shown in the embodiments of the present invention are depicted independently to represent different characteristic functions and do not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for convenience of explanation; however, at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform a function, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not deviate from the essence of the invention.

[0034] Furthermore, some components may not be essential components performing an essential function in the present invention, but merely optional components for enhancing performance. The present invention may be implemented by including only the components essential for realizing the essence of the present invention, excluding components used solely for performance enhancement, and a structure including only the essential components, excluding optional components used solely for performance enhancement, is also included within the scope of the rights of the present invention.

[0035] The blocks used in the invention can be basic block units of coding, prediction block units, and transformation block units. Additionally, block boundaries can be boundaries of coding blocks, prediction block boundaries, and transformation block boundaries.

[0036]

[0037] FIG. 1 is an exemplary block diagram of a video encoding device capable of implementing the technologies of the present disclosure. It compresses an input image composed of multiple pictures to generate a bitstream which is the result of the compression, and may be composed of a picture splitting unit, a prediction performing unit (inter-frame prediction unit, intra-frame prediction unit), a conversion unit, a quantization unit, a reordering unit, an entropy encoding unit, an inverse quantization unit, an inverse conversion unit, a deblocking filter unit, a sample-adaptive offset unit, an adaptive loop filter unit, a restored picture buffer, etc.

[0038]

[0039] FIG. 2 is a block diagram showing in detail a part of a video decoding device according to one embodiment of the present disclosure. The video encoding device receives a bitstream, which is the result of compressing an input image composed of multiple pictures, and performs decoding. It may be composed of an entropy decoding unit, a reordering unit, an inverse quantization unit, an inverse transform unit, a prediction performing unit (inter-frame prediction unit, intra-frame prediction unit), a deblocking filter unit, a sample-adaptive offset unit, an adaptive loop filter unit, a restored picture buffer, etc.

[0040]

[0041] FIG. 3 is a diagram illustrating a method for dividing blocks. The current picture can be divided into multiple super blocks or coding tree units (CTUs), and each super block can be divided into various forms as shown in FIG. 3. Depending on the embodiment, it may include a 1:2:1 or 1:2:4:1 division form, and can be recursively divided up to a block shape where the quad-tree shape is constrained. Additionally, the minimum unit pixel number of each divided block may differ between the luminance block and the chrominance block, for example, 4 pixels and 2 pixels, respectively. Furthermore, the size of the super block and each block may be 256x256 or larger, and the aspect ratio of the available blocks may be 8:1 or 1:8 or larger.

[0042]

[0043] The in-frame prediction unit performs prediction of pixels within the current block using adjacent or non-adjacent pre-recovered reference samples of the current block within the current picture in which the current block exists. At this time, there are multiple directional in-frame prediction modes depending on the prediction direction, and a non-directional prediction mode may also be included. The multiple in-frame prediction modes according to the prediction direction may be as shown in the example of FIG. 4. According to the embodiment, the directional in-frame prediction modes may use a directional prediction mode in the opposite direction (e.g., rotated 180 degrees) that is implicitly rotated by a specific angle depending on the aspect ratio of the current block.

[0044] According to an embodiment, non-directional intra-frame prediction may include at least one of DC, Planar, Smooth, Filtered-based prediction, Intra-block copy (IntraBC), palette mode, matrix-based prediction mode (DIP; Data-driven intra prediction), etc., and information of the prediction mode may be determined through parsed indicators.

[0045]

[0046] FIG. 5 is a diagram illustrating the entire or partial process of parsing intra-frame prediction mode information. According to an embodiment, the example of FIG. 5 may be applied to luminance components. In the intra-frame prediction mode parsing step, if the intra-frame prediction mode parsed is not a DC mode, the mode may be determined as one of the intra-frame prediction modes excluding the DC mode, and intra-frame prediction may be performed using the information parsed through intra-frame prediction information parsing. At this time, the information to be used for prediction may refer to at least one of reference line information (MRL information to be described later), reference sample filtering information, etc.

[0047] According to an embodiment, if the parsed intra-frame prediction mode is DC mode, it can be checked whether palette prediction is allowed for the current block. The palette mode can be determined by signaling / parsing permission information at a higher level, such as SPS (Sequence parameter set) or PPS (Picture parameter set), or / and can be determined as allowed if the block area is greater than or equal to M or less than or equal to N. The values ​​of M and N can be determined without transmission from the encoder to the decoder through an agreement between the encoder and the decoder. In this case, M and N can be fixed values ​​or values ​​that are adaptively determined according to the resolution of the image. Alternatively, they can be values ​​transmitted through a higher-level syntax. In this case, for transmission efficiency, the transmitted values ​​can be in the form of an exponent of N and M or a value obtained by subtracting a specific value from the exponent. In one embodiment, M may be 64 and N may be 4096, and M may be 64 and the width and height of the block may both be greater than or equal to 8, or N may be 4096 and the width and height of the block may both be greater than or equal to 64. If a palette mode is allowed, a palette flag may be parsed, and if the flag is true, information to be used for palette prediction may be parsed, and then palette prediction may be performed.

[0048] According to an embodiment, if the palette mode is not allowed or the parsed palette flag is false, it can be determined whether filter-based intra-frame prediction is allowed. For example, this can be determined by signaling / parsing permission information at a higher level, such as SPS (Sequence parameter set) or PPS (Picture parameter set), or / and it may be allowed if the MRL (Multiple reference line) index is 0, or / and the width of the current block is greater than or equal to P and the height is greater than or equal to Q. For example, P and Q may be 32 or the exponent when expressed in the form of an exponent of 2, or a value obtained by subtracting a specific value from the exponent. Here, the case where the MRL index is 0 may mean that the reference sample to be used for intra-frame prediction is a reference sample located at a position adjacent to the current block. According to an embodiment, if the filter-based intra-frame prediction mode flag is true, filter-based intra-frame prediction mode information may be parsed. For example, this may be index information regarding which filter to use. Subsequently, filter-based intra-frame prediction may be performed.

[0049] According to an embodiment, if the filter-based intra-frame prediction flag is false, it can be checked whether DIP mode is allowed. In another embodiment, whether DIP mode is allowed may be checked independently of the value of the filter-based intra-frame prediction mode flag.

[0050] For example, the allowance of DIP mode can be determined by signaling / parsing permission information at a higher level, such as SPS (Sequence parameter set) or PPS (Picture parameter set). Alternatively, the allowance of DIP mode can be determined by considering the block size and / or MRL index. For example, DIP mode may be allowed when the block size is greater than W and the MRL index is 0, or when the block size is greater than R. In this case, W and R may be fixed values ​​or values ​​that are adaptively determined based on the image resolution. Alternatively, they may be values ​​transmitted through higher-level syntax. For transmission efficiency, the transmitted values ​​may take the form of an exponent of R and W or a value obtained by subtracting a specific value from the exponent. According to an embodiment, W may be 128 and the width and height of the block may be equal to or greater than 8, and R may be 64 and the width and height may be equal to or greater than 8, wherein the transmitted W may be 128 or 7 or a value less than 7 minus some value, and R may be 64 or 6 or a value less than 6 minus some value. According to an embodiment, DIP mode may be allowed even when the MRL index is not 0.

[0051] According to an embodiment, when a DIP mode is allowed, a DIP flag can be parsed, and if the flag is true, mode information to be used for DIP prediction can be parsed. The mode information to be used for DIP prediction may include matrix information to be used for prediction (e.g., a range index of 0 to 5, or / and a combination of matrix set information and index information within the set) and transpose information (transpose flag) regarding whether to perform a transpose process during the prediction process. According to an embodiment, the transpose information may be implicitly derived in the decoder. Subsequently, prediction can be performed in DIP mode.

[0052] According to the embodiment, if DIP mode is not allowed or if the DIP flag is false, prediction can be performed in DC prediction mode.

[0053]

[0054] FIG. 6 is a diagram illustrating information parsing and prediction execution for performing prediction in DIP mode. Depending on the embodiment, the order of each process in FIG. 6 may be changed, or / and some processes may be omitted.

[0055] According to the embodiment, the transpose information may be signaled / parsed in the form of a flag, and accordingly, the process of obtaining and placing reference samples for performing matrix multiplication may differ.

[0056] According to an embodiment, matrix information may be signaled / parsed in the form of an index, and each index may be information indicating a matrix to be used for prediction of the current block through a table determined by an agreement between the decoding and decoding units. In this case, each matrix may be defined by an agreement between the decoding and decoding units, and different matrices may be used based on the size of the current block or / and aspect ratio, etc. Here, the matrix may be a weight matrix.

[0057] According to an embodiment, matrix information may be signaled / parsed in the form of a combination of set information in which multiple matrices are defined and index information indicating matrices within the set, wherein there may be information that is implicitly derived among the set information or the index information.

[0058] For example, for blocks within a restored area surrounding the current block, set information or / and index information may be implicitly determined based on the number of blocks predicted by DIP or the matrix information used for DIP prediction.

[0059] For example, set information or / and index information may be implicitly determined based on the directionality of the restored samples within the restored area surrounding the current block. According to an embodiment, the directionality may be directionality information used for the prediction of the surrounding block, or / and derived directionality information. For example, directionality derivation may be achieved by constructing a histogram of region-specific directionality and amplitude information obtained by performing region-specific filtering on the restored area surrounding the current block using a 3x3 Sobel filter, and using the directionality with the highest amplitude within the histogram as the derived directionality. Subsequently, a directionality within-frame prediction mode corresponding to the derived directionality may be used in the matrix information derivation process.

[0060]

[0061] Examples of the reference sample acquisition step, matrix multiplication operation execution step, and vertical / horizontal interpolation execution step of Fig. 6 are described in detail below.

[0062]

[0063] Below, we describe an example of using samples within a reference line adjacent to the current block for the prediction of the current block. This may also apply when using samples within a non-adjacent reference line of the current block. For example, when the MRL index is 0, samples within a reference line adjacent to the current block may be used, and when the MRL index is not 0, samples within a reference line non-adjacent to the current block may be used.

[0064]

[0065] FIG. 7 is a diagram showing an example for explaining the reference sample acquisition step of FIG. 6.

[0066] For a current block of size WxH, DIP prediction can be performed using reference samples from the top reference sample group (A), top-left reference sample group (AL), top-right reference sample group (AR), left reference sample group (L), and bottom-left reference sample group (BL). In this case, a, b, c, and d may represent integers greater than or equal to 1, and according to an embodiment, b and c may be 1, and a and d may be values ​​determined by the width and height of the current block, respectively. For example, a may have a value of W / 4 and d may have a value of H / 4. According to an embodiment, if there are no available reference samples in the AR area or / and the BL area, adjacent reconstructed samples may be padded and used.

[0067] According to an embodiment, a number of reference samples determined by an agreement between the encoder and decoder in each of the reference sample groups described above can be used to perform subsequent matrix multiplication operations. For example, M reference samples can be obtained for the top and left reference sample groups, and N reference samples can be obtained for the top-left, top-right, and bottom-left reference sample groups. According to an embodiment, M can be 4 and N can be 1. In this case, the total number of reference samples used for matrix multiplication operations may be 11.

[0068] According to an embodiment, the process of obtaining four reference samples from the upper reference sample group can be done by using the average value of each (W / 4) xb reference sample as shown in FIG. 7(b). Similarly, the process of obtaining four reference samples from the left reference sample group can be done by using the average value of each (H / 4) xc reference sample as shown in FIG. 7(c). For the upper-left, upper-right, and lower-left groups, one reference sample can be obtained each by using the average value of the reference samples in the axb, axb, and cxd regions, respectively.

[0069] FIG. 8 is a diagram illustrating an embodiment for performing matrix multiplication. According to the embodiment, to perform matrix multiplication, multiple reference samples obtained in the above process can be arranged in one dimension. The one-dimensional array can be used as an input vector for matrix multiplication. At this time, the arrangement order may be changed according to transposition information. For example, if the transposition information is true, they may be arranged in the order AL -> L -> A -> BL -> AR as in the example of FIG. 8(b), and when multiple reference samples are arranged within each group (AL, L, A, BL, AR), they may be arranged sequentially starting from the left or top position.

[0070] For example, if the preposition information is false, it can be arranged in the order AL -> A -> L -> AR -> BL, as in the example of Fig. 8(a).

[0071] According to an embodiment, the transposition information can be obtained by parsing from a bitstream and can be implicitly derived in a decoder. For example, the transposition information can be implicitly determined based on the directionality of the recovered samples within a recovered region surrounding the current block. For example, the directionality of the recovered samples can be determined by constructing a histogram of region-specific directionality and amplitude information obtained by performing region-specific filtering on the recovered region surrounding the current block using a 3x3 Sobel filter, and using the directionality with the highest amplitude within the histogram as the derived directionality. Subsequently, the transposition information can be determined based on the derived directionality. For example, if there is a vertical directionality, the transposition information can be derived as true, and if there is a horizontal directionality, the transposition information can be derived as false. Alternatively, the derivation of the transposition information can be performed in the opposite embodiment.

[0072]

[0073] According to an embodiment, prediction samples can be generated by performing a multiplication operation between reference samples arranged as in the example of FIG. 8 and a matrix obtained through matrix information parsed from a bitstream. In this case, according to an embodiment, when 11 reference samples are used as in the example of FIG. 8, the matrix may have dimensions of Kx11. According to an embodiment, the K prediction samples generated by performing the matrix multiplication operation may be prediction samples of all or part of the current block. For example, K may have a value of 64. In this case, the K prediction samples may correspond to intermediate prediction samples in the form of an 8x8.

[0074]

[0075] FIG. 9 is a diagram showing an example of a process for generating a prediction signal from a value obtained by performing matrix multiplication.

[0076] According to an embodiment, when P samples are generated from a Px1 vector obtained by performing matrix multiplication, a prediction signal can be generated by filling the P samples into each position of the current block or the sample position corresponding to the current block, as shown in the example of FIG. 9. According to an embodiment, P may be 64. In this case, the P samples may be prediction samples arranged in an 8×8 shape.

[0077] Below, an example is described for the case where 64 samples are generated by performing matrix multiplication. This is just one example, and it implies that the same mechanism can be applied even when a number of samples other than 64 are generated by performing matrix multiplication.

[0078] According to an embodiment, 64 samples can be filled into the current block or sample locations corresponding to the current block in an order determined by an agreement between the decoding and encoding units. For example, they can be filled in raster scan order or Z-scan order, and the filled locations can be sampled locations based on the height and width of the current block. For example, in the row direction, samples can be filled in every W / 8 location. That is, samples can be filled in for locations within the current block where the x-coordinate is W / 8, W / 8x2, W / 8x3, W / 8x4, …, W-1. Also, in the vertical direction, samples can be filled in every H / 8 location. That is, samples can be filled in for locations within the current block where the y-coordinate is H / 8, H / 8x2, H / 8x3, H / 8x4, …, H-1. Based on the samples filled in this way, a prediction signal of the current block can be generated.

[0079] According to another embodiment, 64 samples may be filled into the current block or sample locations corresponding to the current block in an order determined by an agreement between the decoding and encoding units. A scanning method may be implicitly determined based on the orientation of the reconstructed blocks surrounding the current block, or / and the orientation of samples within the reconstructed area surrounding the current block, or / and transposition information. According to an embodiment, it may be determined as at least one of a raster scan, a Z-scan, an inverted Z-scan, a diagonal scan, etc. According to an embodiment, the orientation of samples within the reconstructed area surrounding the current block may be obtained through derivation, and the region-specific orientation and amplitude information regarding the direction obtained by performing region-specific filtering using a 3x3 Sobel filter, etc., on the reconstructed area surrounding the current block may be constructed into a histogram, and the orientation with the greatest amplitude within the histogram may be used as the derived orientation.

[0080]

[0081] FIG. 10 is a diagram showing an example of a process for generating a prediction signal from a value obtained by performing matrix multiplication.

[0082] According to an embodiment, after filling the current block with a matrix multiplication-based prediction signal as in the above embodiment, a prediction signal can be generated using an interpolation technique for areas where the prediction signal has not yet been filled. According to an embodiment, at least one of bilinear interpolation, bilateral interpolation, etc. may be used for interpolation. According to an embodiment, the number of samples used in the interpolation process to generate one sample may be two or more, and interpolation may be performed using surrounding prediction signals or / and restored samples in various directions, such as vertical, horizontal, and diagonal interpolation. In one embodiment, interpolation may be performed to extend the matrix multiplication-based prediction signal into a prediction signal corresponding to the width and height of the current block.

[0083] For example, a prediction signal can be generated by sequentially performing vertical and horizontal interpolation within the current block. In this case, the result of the interpolation performed first can be used as the input for the subsequent interpolation process. In this case, there may be instances where interpolation is not performed because the area filled with the matrix multiplication-based prediction signal is already filled with the prediction signal. According to an embodiment, for the interpolation target location sample as shown in the example of FIG. 10, interpolation can be performed using left / right restored samples or / and matrix multiplication-based prediction signals for horizontal interpolation, and for vertical interpolation, interpolation can be performed using up / down restored samples or / and matrix multiplication-based prediction signals.

[0084] According to an embodiment, if both vertical and horizontal interpolation are required, horizontal interpolation may be performed first, followed by vertical interpolation, by agreement between the encoder and decoder. In another embodiment, the interpolation order may be adaptively determined based on matrix information used for the prediction of the current block or the size of the current block. For example, if a specific matrix is ​​used, interpolation may be performed in the order of vertical to horizontal, and for other matrices, interpolation may be performed in the order of horizontal to vertical.

[0085]

[0086] According to an embodiment, for restoration after the prediction is performed, an inverse transformation process may be performed to obtain a residual signal from the transformation coefficients.

[0087]

[0088] FIG. 11 is an example showing a block diagram of the process in which the inverse transformation of the current block is performed in the decoder.

[0089] According to an embodiment, as in (a), using information on whether to perform a second inverse transform and information on whether to perform a first inverse transform, 1) a second inverse transform -> a first inverse transform, 2) a first inverse transform, and 3) a transform skip process may be performed. In another embodiment, as in (b), using information on whether to perform a transform skip and information on whether to perform a second inverse transform, 1) a second inverse transform -> a first inverse transform and 2) a first inverse transform and 3) a transform skip process may be performed.

[0090] According to an embodiment, when a second inverse transformation is performed, or when a first inverse transformation is performed without a second inverse transformation, a second inverse transformation kernel candidate list configuration process or a first inverse transformation kernel candidate list configuration process may be performed in response to the inverse transformation process being performed. For example, if the prediction of the current block is performed in a directional in-frame prediction mode, the candidate list may be configured as an adaptive kernel candidate list based on directionality. According to an embodiment, when a transformation skip process is performed, the inverse transformation kernel candidate list configuration process may be omitted.

[0091] According to an embodiment, when DIP prediction is performed, different kernel lists may be configured depending on the matrix used for prediction. For example, depending on the matrix information used for DIP prediction, at least one of the first-order inverse transformation kernel candidate list or the second-order inverse transformation kernel candidate list may be configured differently.

[0092] According to an embodiment, when DIP prediction is performed, a dominant directionality is induced in the prediction signal of the current block after the prediction is completed, and a kernel candidate list corresponding to the directional in-frame prediction mode corresponding to the directionality can be constructed.

[0093] According to an embodiment, when DIP prediction is performed, samples of vectors generated by the matrix multiplication operation of FIG. 8 are configured into a two-dimensional form in Z-scan order as in FIG. 12, and then the dominant directionality of the corresponding region is derived, and a kernel candidate list corresponding to the directionality in-frame prediction mode corresponding to the directionality can be configured.

[0094] According to an embodiment, for a region to be directionally guided, the region-specific directionality and amplitude information obtained by performing region-specific filtering using a 3x3 Sobel filter, etc., are constructed into a histogram, and the directionality with the highest amplitude in the histogram can be used as the directionality. Subsequently, the directionality-in-frame prediction mode corresponding to the directionality can be used in the process of constructing a kernel candidate list.

[0095]

[0096] The exemplary methods of the present disclosure are described as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. To implement the method according to the present disclosure, additional steps may be included in addition to the steps exemplified, steps excluding some steps and including the remaining steps, or steps excluding some steps and including additional steps.

[0097] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0098] In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0099] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0100] The present disclosure relates to video encoding and decoding technology and can be used in a video encoding device, a video decoding device, a video transmitting and receiving device, a video storage device, a video playback device, etc., which performs intra-frame prediction based on the multiplication of a matrix and a restored sample around the current block.

[0101] In addition, the present invention can be applied to various industrial fields that compress, transmit, store, or restore video data, such as broadcasting systems, streaming systems, video call systems, surveillance camera systems, mobile terminals, servers, and cloud-based video processing systems.

Claims

1. In an image decoding method performed by a decoding device, A step of obtaining in-frame prediction information for the current block from the bitstream; and It includes the step of generating a prediction signal for the current block based on the prediction information within the screen above, and The above-mentioned in-screen prediction information includes information indicating whether a matrix-based in-screen prediction mode is applied to the above-mentioned current block, and A video decoding method in which, when the above matrix-based intra-frame prediction mode is applied, the prediction signal is generated based on a multiplication operation between an input vector containing multiple reference values ​​obtained from restored samples around the current block and a matrix used for prediction of the current block.

2. In Paragraph 1, An image decoding method in which the plurality of reference values ​​are obtained based on at least one of the top reference sample, left reference sample, top-left reference sample, top-right reference sample, or bottom-left reference sample of the current block.

3. In Paragraph 2, An image decoding method in which at least one of the plurality of reference values ​​is obtained based on the average value of a plurality of restored samples around the current block.

4. In Paragraph 1, An image decoding method in which the above matrix is ​​determined based on matrix information that specifies the matrix to be used for the prediction of the current block.

5. In Paragraph 4, An image decoding method in which the matrix information includes index information obtained from the bitstream.

6. In Paragraph 1, An image decoding method in which the above matrix is ​​determined based on at least one of the size of the current block or the aspect ratio of the current block.

7. In Paragraph 1, Multiple prediction samples are generated based on the above multiplication operation, and An image decoding method in which a prediction signal for the current block is generated based on the plurality of prediction samples.

8. In Paragraph 7, An image decoding method in which a prediction signal for the current block is generated based on interpolation or resampling of the plurality of prediction samples.

9. In Paragraph 1, A video decoding method in which the permission of the matrix-based intra-frame prediction mode is determined based on at least one of the intra-frame prediction mode of the current block or the application of a palette mode to the current block.

10. In Paragraph 9, The above matrix-based intra-frame prediction mode is an image decoding method that is allowed when the intra-frame prediction mode of the current block is DC mode and a palette mode is not applied to the current block.

11. In Paragraph 1, The above-mentioned in-screen prediction information includes transposition information for determining the array order of the plurality of reference values ​​included in the input vector when the above-mentioned matrix-based in-screen prediction mode is applied, and An image decoding method in which the array order of the plurality of reference values ​​included in the input vector is determined based on the transposition information.

12. In Paragraph 4, An image decoding method wherein the matrix information comprises at least one of matrix set information in which a plurality of matrices are defined or index information indicating a matrix within the matrix set.

13. In an image encoding method performed by an encoding device, A step of generating in-screen prediction information for the current block; and It includes the step of generating a prediction signal for the current block based on the prediction information within the screen above, and The above-mentioned in-screen prediction information includes information indicating whether a matrix-based in-screen prediction mode is applied to the above-mentioned current block, and When the above matrix-based in-frame prediction mode is applied, the prediction signal is generated based on a multiplication operation between an input vector containing multiple reference values ​​obtained from restored samples around the current block and a matrix used for prediction of the current block, and A video encoding method in which the above-mentioned in-screen prediction information is encoded into a bitstream.

14. A bitstream transmission method for transmitting a bitstream generated by an image encoding method performed by an encoding device, wherein Step of generating in-screen prediction information for the current block; A step of generating a prediction signal for the current block based on the prediction information within the screen above; and The method includes the step of transmitting the bitstream generated based on the above prediction signal, The above-mentioned in-screen prediction information includes information indicating whether a matrix-based in-screen prediction mode is applied to the above-mentioned current block, and When the above matrix-based in-frame prediction mode is applied, the prediction signal is generated based on a multiplication operation between an input vector containing multiple reference values ​​obtained from restored samples around the current block and a matrix used for prediction of the current block, and A bitstream transmission method in which the above-mentioned in-screen prediction information is included in the above-mentioned bitstream.