Decoding methods, encoding methods, readable media, and methods for transmitting image data.

Abstract

Decoding method, encoding method, readable medium, and method for transmitting image data. A method for decoding an image signal according to an embodiment of the present invention includes the following steps: for a current block whose width and height are different from each other, determining a modified intra-prediction mode based on the ratio between the width and height of the current block and an intra-prediction mode; determining an inverse inseparable transform set based on the modified intra-prediction mode; and applying an inverse inseparable transform matrix selected from the inverse inseparable transform set relative to the upper left region determined according to the width and height of the current block.

Description

[0001] This application is a divisional application of the original invention patent application No. 201980066907.9 (International Application No.: PCT / KR2019 / 010231, Application Date: August 12, 2019, Invention Title: Method and Apparatus for Processing Image Signals). Technical Field

[0002] This disclosure relates to a method and apparatus for processing video signals, and more specifically, to a method and apparatus for encoding or decoding video signals, wherein the video signals are applied with consideration of a wide-angle intra-frame prediction mode transformation. Background Technology

[0003] Compression coding refers to a series of signal processing techniques used to transmit digital information over communication lines or to store information in a form suitable for storage media. Media including video, images, and audio can be targets of compression coding. In particular, techniques used to perform compression coding on images are called image compression.

[0004] Next-generation image content will feature high spatial resolution, high frame rates, and high-dimensional scene representation. Processing such content will lead to a dramatic increase in memory storage, memory access speeds, and processing power.

[0005] Therefore, there is a need to design an encoding tool for efficiently processing next-generation image content. In particular, along with high-precision prediction techniques, image codec standards following the High Efficiency Image Coding (HEVC) standard require an efficient transformation technique for converting image signals in the spatial domain into signals in the frequency domain. Summary of the Invention

[0006] Technical issues

[0007] There is a need for a method and apparatus for providing effective transformation techniques based on prediction patterns and prediction schemes with higher accuracy.

[0008] Therefore, embodiments of this disclosure are intended to provide an image signal processing method and apparatus for providing transformations capable of supporting high-precision prediction patterns.

[0009] Furthermore, embodiments of this disclosure provide an image signal processing method and apparatus for providing transformations that can support effective prediction modes by incorporating the derivation of low-frequency inseparable transformations of wide-angle intra-frame prediction modes.

[0010] The technical objectives to be achieved by this disclosure are not limited to those described above, and other technical objectives not described above can be clearly understood by those skilled in the art from the following description.

[0011] Technical solution

[0012] A method for decoding an image signal according to an embodiment of the present disclosure includes: determining a modified intra-prediction mode with different widths and heights for the current block based on the ratio between the width and height of the current block and an intra-prediction mode; determining an inverse inseparable transform set based on the modified intra-prediction mode; and applying an inverse inseparable transform matrix selected from the inverse inseparable transform set to the upper left region of the current block determined based on the width and height of the current block.

[0013] In addition, determining the modified intra-prediction mode may include: when the width is greater than the height and the intra-prediction mode is greater than or equal to 2 and less than the first reference value, the value obtained by adding 65 to the intra-prediction mode shall be determined as the modified intra-prediction mode.

[0014] Furthermore, when the ratio of the width to the height of the current block is less than or equal to 2, the first reference value can be set to 8, and when the ratio of the width to the height of the current block is greater than 2, the first reference value can be set to 12.

[0015] In addition, determining the modified intra-prediction mode may include: when the width is less than the height and the intra-prediction mode is greater than or equal to the second reference value and less than or equal to 66, the value obtained by subtracting 67 from the intra-prediction mode shall be determined as the modified intra-prediction mode.

[0016] Furthermore, when the ratio of the current block's height to its width is less than or equal to 2, the second reference value can be set to 61, and when the ratio of the current block's height to its width is greater than 2, the second reference value can be set to 57.

[0017] In addition, determining the inverse inseparable transform set may include: determining the index of the inverse inseparable transform set corresponding to the modified intra-prediction mode.

[0018] Furthermore, when the modified intra-prediction mode is greater than or equal to 0 and less than or equal to 1, the index can be determined as the first index value; when the modified intra-prediction mode is greater than or equal to 2 and less than or equal to 12, the index can be determined as the second index value; when the modified intra-prediction mode is greater than or equal to 13 and less than or equal to 23, the index can be determined as the third index value; when the modified intra-prediction mode is greater than or equal to 24 and less than or equal to 44, the index can be determined as the fourth index value; when the modified intra-prediction mode is greater than or equal to 45 and less than or equal to 55, the index can be determined as the third index value; when the modified intra-prediction mode is greater than or equal to 56, the index can be determined as the second index value; and when the modified intra-prediction mode is less than 0, the index can be determined as the second index value.

[0019] An apparatus for decoding an image signal according to another embodiment of the present disclosure includes: a memory storing the image signal; and a processor coupled to the memory and configured to process the image signal, wherein the processor is configured to determine a modified intra-prediction mode with a different width and height for the current block based on the ratio between the width and height of the current block and the intra-prediction mode, determine an inverse inseparable transform set based on the modified intra-prediction mode, and apply an inverse inseparable transform matrix selected from the inverse inseparable transform set to the upper left region of the current block determined based on the width and height of the current block.

[0020] Furthermore, the processor can be configured to determine the modified intra-prediction mode by adding 65 to the intra-prediction mode when the width is greater than the height and the intra-prediction mode is greater than or equal to 2 and less than the first reference value.

[0021] Furthermore, the processor can be configured to determine the modified intra-prediction mode as the value obtained by subtracting 67 from the intra-prediction mode when the width is less than the height and the intra-prediction mode is greater than or equal to the second reference value and less than or equal to 66.

[0022] In addition, the processor can be configured to determine the index of the inverse inseparable transform set corresponding to the modified intra-prediction mode.

[0023] Furthermore, when the modified intra-prediction mode is greater than or equal to 0 and less than or equal to 1, the index can be determined as the first index value; when the modified intra-prediction mode is greater than or equal to 2 and less than or equal to 12, the index can be determined as the second index value; when the modified intra-prediction mode is greater than or equal to 13 and less than or equal to 23, the index can be determined as the third index value; when the modified intra-prediction mode is greater than or equal to 24 and less than or equal to 44, the index can be determined as the fourth index value; when the modified intra-prediction mode is greater than or equal to 45 and less than or equal to 55, the index can be determined as the third index value; when the modified intra-prediction mode is greater than or equal to 56, the index can be determined as the second index value; and when the modified intra-prediction mode is less than 0, the index can be determined as the second index value.

[0024] Beneficial effects

[0025] According to embodiments of this disclosure, a transformation matrix capable of supporting prediction patterns with high accuracy can be designed.

[0026] Furthermore, according to embodiments of this disclosure, by deriving a low-frequency inseparable transform that incorporates a wide-angle intra-frame prediction mode, prediction accuracy can be improved and the design complexity of the transform matrix can be reduced.

[0027] The effects that can be achieved by this disclosure are not limited to those described above, and those skilled in the art to which this disclosure pertains will more clearly understand other effects and advantages of this disclosure from the following description. Attached Figure Description

[0028] The accompanying drawings are included to provide a further understanding of the present disclosure and form part of the detailed description. The drawings illustrate embodiments of the present disclosure and, together with the description, serve to explain the technical features of the present disclosure.

[0029] Figure 1 This is an embodiment of the present disclosure, and a schematic block diagram of an encoder that performs the encoding of image signals is shown.

[0030] Figure 2 This is an embodiment of the present disclosure, and a schematic block diagram of a decoder that performs decoding of image signals is shown.

[0031] Figures 3a-3d Implementation methods to which this disclosure can be applied are shown. Figure 3a , Figure 3b , Figure 3c and Figure 3d It is a graph used to describe block partitioning structures based on quadtrees, binary trees, ternary trees, and asymmetric trees, respectively.

[0032] Figure 4 and Figure 5 This is an implementation method of the present disclosure, wherein, Figure 4 A schematic block diagram of the transform and quantization units, as well as the inverse quantization and inverse transform units within the encoder, is shown. Figure 5 A schematic block diagram of the inverse quantization unit and inverse transform unit within the decoder is shown.

[0033] Figure 6a and Figure 6b An example table is shown for determining the transformation type in the horizontal and vertical directions for each prediction pattern.

[0034] Figure 7 This is an embodiment of the present disclosure, and is a flowchart illustrating the encoding process performed by Multiple Transform Selection (MTS).

[0035] Figure 8 This is an embodiment of the present disclosure, and a flowchart illustrating the decoding process of performing MTS.

[0036] Figure 9 This is an implementation of the present disclosure and a flowchart describing the process of encoding MTS tags and MTS indexes.

[0037] Figure 10This is an implementation of the present disclosure, and is a flowchart illustrating the decoding process of applying horizontal or vertical transformations to rows or columns based on MTS tags and MTS indexes.

[0038] Figure 11 This describes an implementation of the present disclosure, and illustrates a flowchart of performing an inverse transformation based on transformation-related parameters.

[0039] Figure 12 This is an implementation of the present disclosure, and is a table showing the assignment of transform sets to each intra-frame prediction mode in the Non-Separable Auxiliary Transform (NSST).

[0040] Figure 13 This describes an implementation of the present disclosure, and a flowchart of the calculation of Givens rotation is shown.

[0041] Figure 14 This is an embodiment of the present disclosure, and a round configuration in a 4×4 NSST consisting of Givens rotating layers and displacement is shown.

[0042] Figure 15 This is an embodiment of the present disclosure and a block diagram for describing the operations of the forward reduction transform and the inverse reduction transform.

[0043] Figure 16 This is an embodiment of the present disclosure, and is a diagram illustrating the process of performing a reverse scan from the 64th to the 17th according to the reverse scan sequence.

[0044] Figure 17 This describes an implementation of the present disclosure and illustrates a flowchart of encoding an image signal through a primary transform and an auxiliary transform.

[0045] Figure 18 This describes an implementation of the present disclosure, and illustrates a flowchart of decoding an image signal through inverse auxiliary transformation and inverse main transformation.

[0046] Figure 19 This is an implementation of the present disclosure, and an example of an intra-prediction mode configuration consisting of 67 modes is shown.

[0047] Figure 20 This is an implementation of the present disclosure, and an example of a mapping table for 67 intra-prediction modes for 35 intra-prediction modes is shown.

[0048] Figure 21 This describes an implementation of the present disclosure and illustrates an example of configuring two additional wide-angle modes in 35 intra-frame prediction modes.

[0049] Figure 22This describes an implementation of the present disclosure, and illustrates an example of configuring an additional 10 wide-angle modes among 35 intra-frame prediction modes.

[0050] Figure 23 This is an implementation of the present disclosure, and an example of an index mapping table between wide-angle modes added in 35 intra-frame prediction modes and 67 intra-frame prediction modes is shown.

[0051] Figure 24 and Figure 25 This describes an implementation of the present disclosure, and an example of intra-frame prediction for non-square blocks is shown.

[0052] Figures 26a-26b This describes an implementation of the present disclosure and illustrates an example of the order in which pixels are arranged when a two-dimensional block is transformed into a one-dimensional vector, wherein... Figure 26a An example of row precedence is shown, and Figure 26b An example of column priority order is shown.

[0053] Figures 27a-27b This describes an implementation of the present disclosure, and illustrates an example of a mapping table between 35 transform sets and intra-frame prediction modes, wherein... Figure 27a An example is shown where the transform set is symmetrically assigned to each wide-angle mode. Figure 27b An example is shown where transform set 2 is assigned to all wide-angle modes, and Figure 27c An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0054] Figures 28a-28b This describes an implementation of the present disclosure, and an example of a mapping table between 10 transform sets and intra-frame prediction modes is shown, wherein... Figure 28a An example is shown where transform set 2 is assigned to each wide-angle mode, and Figure 28b An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0055] Figures 29a-29b This describes an implementation of the present disclosure, and illustrates an example of a mapping table between six transform sets and intra-frame prediction modes, wherein... Figure 29a An example is shown where transform set 2 is assigned to each wide-angle mode, and Figure 29b An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0056] Figures 30a-30b This describes an implementation of the present disclosure, and illustrates an example of a mapping table between four transform sets and intra-frame prediction modes, wherein... Figure 30aAn example is shown where transform set 2 is assigned to each wide-angle mode, and Figure 30b An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0057] Figure 31 This is an example of an implementation of the present disclosure, illustrating a flowchart of how the encoder performs a transformation by taking into account wide-angle intra-frame prediction (WAIP).

[0058] Figure 32 This is an example of an implementation of the present disclosure, and illustrates a flowchart of the decoder performing a transformation by taking WAIP into account.

[0059] Figure 33 This is an implementation of the present disclosure, and another example of a flowchart showing the decoder performing a transformation by taking WAIP into account.

[0060] Figure 34 This is an example of an embodiment of the present disclosure, and shows a block diagram of an image processing apparatus.

[0061] Figure 35 An example of an image encoding system in which the present disclosure is applied is shown.

[0062] Figure 36 An example of an image streaming system in which embodiments of the present disclosure are applied is shown. Detailed Implementation

[0063] Embodiments of this disclosure will now be described in detail, examples of which are illustrated in the accompanying drawings. The following is related to the accompanying drawings. Figure 1 The detailed description provided herein is intended to describe exemplary embodiments of this disclosure and not to describe the only embodiments for carrying out this disclosure. The following detailed description includes details that provide a complete understanding of this disclosure. However, those skilled in the art will recognize that this disclosure can be performed without these details.

[0064] In some cases, to prevent ambiguity in the concepts of this disclosure, known structures and devices may be omitted or shown in block diagram form based on the core functions of each structure and device.

[0065] Furthermore, while this disclosure uses widely used and common terms as much as possible, terms arbitrarily chosen by the applicant are used in specific cases. Since the meaning of the terms will be clearly described in the relevant sections of the specification in such cases, it should be understood that this disclosure is not to be simply interpreted by the terms used only in the description of this disclosure, but rather the meaning of the terms should be indicated.

[0066] Specific terminology used in the following description may be provided to aid in understanding this disclosure. Furthermore, within the scope of the technical concepts of this disclosure, specific terminology may be modified to other forms. For example, signals, data, samples, images, frames, blocks, etc., may be appropriately replaced and interpreted in each encoding process.

[0067] In this disclosure, "processing unit" refers to a unit that performs encoding / decoding processes such as prediction, transformation, and / or quantization. A processing unit can also be interpreted to include units for the luma component and units for the chroma component. For example, a processing unit may correspond to a block, coding unit (CU), prediction unit (PU), or transform unit (TU).

[0068] The processing unit can also be interpreted as a unit for the luma component or a unit for the chroma component. For example, a processing unit may correspond to a coding tree block (CTB), coding block (CB), prediction unit (PU), or transform block (TB) for the luma component. Alternatively, a processing unit may correspond to a CTB, CB, PU, ​​or TB for the chroma component. The processing unit is not limited thereto and can be interpreted to include the meaning of units for both the luma and chroma components.

[0069] Furthermore, the processing unit is not limited to a square block and can be configured as a polygon shape with three or more vertices.

[0070] In this disclosure, pixels are generally referred to as samples. Additionally, using samples can mean using pixel values, etc.

[0071] Figure 1 This is an embodiment of the present disclosure, and a schematic block diagram of an encoder that performs encoding of image signals is shown.

[0072] Reference Figure 1 The encoder 100 can be configured to include an image segmentation unit 110, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, a filtering unit 160, a decoded image buffer (DPB) 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy coding unit 190.

[0073] The image partitioning unit 110 can divide the input image (or picture or frame) input to the encoder 100 into one or more processing units. For example, the processing unit may be a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), or a transform unit (TU). In the following description, the transform unit (TU) (i.e., the unit that performs the transform) is collectively referred to as a transform block.

[0074] However, these terms are used only for the convenience of describing this disclosure, and this disclosure is not limited to the definitions of these terms. Furthermore, in this disclosure, for ease of description, the term "encoding unit" is used as a unit used when encoding or decoding image signals, but this disclosure is not limited thereto and can be appropriately interpreted according to this disclosure.

[0075] The encoder 100 subtracts the prediction signal output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 from the input image signal to generate a residual signal, and transmits the generated residual signal to the transformation unit 120.

[0076] Transform unit 120 can generate transform coefficients by applying transform techniques to the residual signal. The transform process can be applied to square blocks in a quadtree structure and blocks (squares or rectangles) divided by a binary tree structure, ternary tree structure, or asymmetric tree structure.

[0077] Transformer 120 can perform transformations based on multiple transformations (or combinations of transformations), and this transformation scheme can be referred to as Multiple Transform Selection (MTS). MTS can also be referred to as Adaptive Multiple Transform (AMT) or Enhanced Multiple Transform (EMT).

[0078] MTS (or AMT or EMT) can refer to a transformation scheme performed based on an adaptively selected transformation (or combination of transformations) from multiple transformations (or combinations of transformations).

[0079] Multiple transforms (or combinations of transforms) can be determined based on kernels of the Discrete Cosine Transform (DCT) or Discrete Sine Transform (DST) types, as disclosed herein. Figure 6a and Figure 6b As shown. In this disclosure, the transformation type can be expressed as, for example, DCT type 2, DCT-II, DCT-2, DCT2, and is generally indicated as DCT-2 in the following description.

[0080] The transformation unit 120 according to the embodiments of this disclosure can perform the following embodiments.

[0081] Transform unit 120 can apply a forward primary transform to each of the horizontal and vertical directions of the transform block, which includes residual samples, and can apply secondary transforms based on the size of the transform block (or the current block). For example, when the size of the current block is greater than or equal to 4×4, the Inseparable Secondary Transform (NSST) and / or Reduced Secondary Transform (RST), which will be described later, can be applied. In one implementation, NSST and / or RST can be applied to the top-left 4×4 or 8×8 region based on the width and height of the current block. In this document, “M×N” indicates the size of the block (matrix), referring to the width and height of the block. For example, if the size of the block is greater than or equal to 4×4, this means that both the width and height of the block are greater than or equal to 4.

[0082] The quantization unit 130 can quantize the transform coefficients and send the quantized transform coefficients to the entropy coding unit 190. The entropy coding unit 190 can entropy code the quantized signal and output the entropy-coded quantized signal as a bit stream.

[0083] Although the transformation unit 120 and the quantization unit 130 are described as separate functional units, this disclosure is not limited thereto, and they can be combined into a single functional unit. Similarly, the inverse quantization unit 140 and the inverse transformation unit 150 can also be combined into a single functional unit.

[0084] The quantized signal output from quantization unit 130 can be used to generate a prediction signal. For example, inverse quantization and inverse transform are applied to the quantized signal in a cyclic manner by inverse quantization unit 140 and inverse transform unit 1850 to reconstruct the residual signal. The reconstructed residual signal is added to the prediction signal output from inter-frame prediction unit 180 or intra-frame prediction unit 185 to generate the reconstructed signal.

[0085] Furthermore, due to quantization errors that occur during such compression, degradation that reveals block boundaries may occur. This phenomenon is known as block artifacts and is one of the key factors in evaluating image quality. Filtering can be performed to reduce this degradation. By eliminating block degradation and reducing errors in the current image through the filtering process, image quality is improved.

[0086] The filtering unit 160 applies filtering to the reconstructed signal and outputs the applied reconstructed signal to the playback device, or transmits the output reconstructed signal to the decoding image buffer 170. The inter-frame prediction unit 170 can use the filtered signal transmitted to the decoding image buffer 180 as a reference image. In this way, the filtered image is used as a reference image in inter-frame prediction mode to improve image quality and coding efficiency.

[0087] The decoded image buffer 170 can store the filtered image so that it can be used as a reference image in the inter-frame prediction unit 180.

[0088] Inter-frame prediction unit 180 performs temporal and / or spatial prediction to remove temporal and / or spatial redundancy by reconstructing a reference image. In this case, block artifacts or ringing artifacts may exist because the reference image used for prediction is a transformed signal that has been quantized and dequantized in blocks during encoding / decoding at a previous time.

[0089] Therefore, the inter-frame prediction unit 180 can interpolate the signal between pixels on a sub-pixel basis by applying a low-pass filter to address the performance degradation caused by signal discontinuities or quantization. In this case, a sub-pixel refers to a virtual pixel generated by applying an interpolation filter, while an integer pixel refers to an actual pixel present in the reconstructed image. Interpolation methods such as linear interpolation, bilinear interpolation, and Wiener filters can be used.

[0090] Interpolation filters are applied to reconstruct the image to improve prediction accuracy. For example, the inter-frame prediction unit 180 applies interpolation filters to integer pixels to generate interpolated pixels, and prediction can be performed by using interpolated blocks composed of interpolated pixels as prediction blocks.

[0091] Furthermore, the intra-prediction unit 185 can predict the current block by referencing samples near the block to be encoded. The intra-prediction unit 185 can perform the following process to perform intra-prediction: First, reference samples can be prepared, which is necessary for generating the prediction signal. Alternatively, the prediction signal can be generated using the prepared reference samples. Afterward, the prediction mode is encoded. In this case, reference samples can be prepared by reference sample padding and / or reference sample filtering. Since the reference samples have undergone the prediction and reconstruction process, quantization errors may exist. Therefore, a reference sample filtering process can be performed for each prediction mode used for intra-prediction to reduce such errors.

[0092] The prediction signal generated by the inter-frame prediction unit 180 or the intra-frame prediction unit 185 can be used to generate the reconstructed signal or the residual signal.

[0093] Figure 2 This is an embodiment of the present disclosure, and a schematic block diagram of a decoder that performs decoding of image signals is shown.

[0094] Reference Figure 2 The decoder 200 can be configured to include a parsing unit (not shown), an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, a filtering unit 240, a decoded picture buffer (DPB) unit 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265.

[0095] In addition, the reconstructed image signal output by the decoder 200 can be reproduced using a reproduction device.

[0096] Decoder 200 can receive from Figure 1 The signal output by the encoder 100, and the received signal can be entropy decoded by the entropy decoding unit 210.

[0097] The inverse quantization unit 220 obtains the transform coefficients from the entropy-decoded signal by using quantization step size information.

[0098] The inverse transform unit 230 performs an inverse transform on the transform coefficients to obtain the residual signal.

[0099] When the size of the transform block is greater than or equal to 4×4, the inverse transform unit 230 according to an embodiment of the present disclosure can be configured to apply the inverse inseparable auxiliary transform to the transform block, separate the transform block in the horizontal and vertical directions, and apply the inverse main transform. For example, the inverse NSST and / or inverse RST can be applied to the upper left 4×4 or 8×8 region based on the width and height of the current block.

[0100] Although the dequantization unit 220 and the inverse transform unit 230 are described as separate functional units, this disclosure is not limited thereto and they can be combined into a single functional unit.

[0101] The obtained residual signal is added to the prediction signal output from the inter-frame prediction unit 260 or the intra-frame prediction unit 265 to generate the reconstructed signal.

[0102] The filtering unit 240 filters the reconstructed signal and outputs the applied reconstructed signal to the generation device, or transmits the output reconstructed signal to the decoding image buffer unit 250. The inter-frame prediction unit 250 can use the filtered signal sent to the decoding image buffer unit 260 as a reference image.

[0103] In this disclosure, the embodiments described in the various functional units of the transform unit 120 and the encoder 100 can be equivalently applied to the corresponding functional units of the inverse transform unit 230 and the decoder, respectively.

[0104] Figure 3 illustrates an implementation method to which this disclosure can be applied. Figure 3a This is a diagram used to illustrate a block partitioning structure based on a quadtree (hereinafter referred to as "QT"). Figure 3b This is a diagram used to illustrate a block partitioning structure based on a binary tree (hereinafter referred to as "BT"). Figure 3c It is a graph used to describe a block partitioning structure based on a ternary tree (hereinafter referred to as "TT"), and Figure 3d It is a graph used to describe the block partitioning structure based on an asymmetric tree (hereinafter referred to as "AT").

[0105] In image encoding, a block can be segmented based on a quadtree (QT). Furthermore, a sub-block segmented by a QT can be recursively segmented. Leaf blocks that are no longer segmented by a QT can be segmented using at least one of binary trees (BT), ternary trees (TT), or asymmetric trees (AT). BT can have two types of segmentation: horizontal BT (2N×N, 2N×N) and vertical BT (N×2N, N×2N). TT can have two types of segmentation: horizontal TT (2N×1 / 2N, 2N×N, 2N×1 / 2N) and vertical TT (1 / 2N×2N, N×2N, 1 / 2N×2N). AT can be divided into four types: horizontal upward AT (2N×1 / 2N, 2N×3 / 2N), horizontal downward AT (2N×3 / 2N, 2N×1 / 2N), vertical leftward AT (1 / 2N×2N, 3 / 2N×2N), and vertical rightward AT (3 / 2N×2N, 1 / 2N×2N). Each BT, TT, or AT can be further recursively divided using BT, TT, or AT.

[0106] Figure 3a An example of QT partitioning is shown. Block A can be partitioned into four sub-blocks A0, A1, A2, and A3 using QT. Sub-block A1 can be partitioned into four sub-blocks B0, B1, B2, and B3 using QT.

[0107] Figure 3b An example of BT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical BT C0 and C1 or horizontal BT D0 and D1. As in block C0, each sub-block can be recursively partitioned in the form of horizontal BT E0 and E1 or vertical BT F0 and F1.

[0108] Figure 3c An example of TT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical TT C0, C1, and C2 or horizontal TT D0, D1, and D2. As in block C1, each sub-block can be recursively partitioned in the form of horizontal TT E0, E1, and E2 or vertical TT F0, F1, and F2.

[0109] Figure 3d An example of AT partitioning is shown. Block B3, which is no longer partitioned by QT, can be partitioned into vertical AT C0 and C1 or horizontal AT D0 and D1. As in block C1, each sub-block can be recursively partitioned in the form of horizontal AT E0 and E1 or vertical TT F0 and F1.

[0110] Furthermore, BT, TT, and AT partitions can be performed together. For example, a sub-block partitioned by BT can be partitioned by TT or AT. Similarly, a sub-block partitioned by TT can be partitioned by BT or AT. Sub-blocks partitioned by AT can be partitioned by BT or TT. For example, after a horizontal BT partition, each sub-block can be partitioned into vertical BT partitions, or after a vertical BT partition, each sub-block can be partitioned into horizontal BT partitions. The partitioning order differs between the two types of partitioning methods, but the final partitioned shape is the same.

[0111] Furthermore, if a block is segmented, the order in which the blocks are searched can be defined in various ways. Typically, the search is performed from left to right or from top to bottom. The search block can refer to a sequence of additional blocks used to determine whether to split each segmented sub-block, or it can refer to the encoded sequence of each sub-block when the block is no longer segmented, or it can refer to the search sequence when information from another neighboring block is referenced in a sub-block.

[0112] Transformations can be performed on each processing unit (or transformation unit) partitioned by the partitioning structure, such as... Figures 3a to 3d Specifically, partitioning can be performed for each row direction and each column direction, and a transformation matrix can be applied. According to embodiments of this disclosure, different transformation types can be used based on the length of the processing unit (or transformation unit) in the row or column direction.

[0113] Figure 4 and Figure 5 This is an implementation method that applies the present disclosure. Figure 4 A schematic block diagram of the transform and quantization unit 120 / 130 and the dequantization and transform unit 140 / 150 within the encoder is shown, and Figure 5 A schematic block diagram of the dequantization and transformation units 220 / 230 within the decoder is shown.

[0114] Reference Figure 4 The transformation and quantization units 120 / 130 may include a main transformation unit 121, an auxiliary transformation unit 122, and a quantization unit 130. The dequantization and transformation units 140 / 150 may include a dequantization unit 140, an inverse auxiliary transformation unit 151, and an inverse main transformation unit 152.

[0115] Reference Figure 5 The dequantization and transformation unit 220 / 230 may include a dequantization unit 220, an inverse auxiliary transformation unit 231, and an inverse main transformation unit 232.

[0116] In this disclosure, when performing a transformation, it can be done through multiple steps. For example, as... Figure 4 As shown, according to the algorithm, two steps—the main transform and the auxiliary transform—can be applied, or more transform steps can be used. In this case, the main transform can be referred to as the core transform.

[0117] The main transform unit 121 can apply the main transform to the residual signal. In this case, the main transform can be predefined in tabular form in the encoder and / or decoder.

[0118] Furthermore, combinations of several transform types of MTS (DCT-2, DST-7, DCT-8) can be used for the main transform. For example, it can be used as follows: Figure 6a and Figure 6b The transformation type is determined as shown in the table.

[0119] The auxiliary transform unit 122 can apply the auxiliary transform to the main transform signal. In this case, the auxiliary transform can be predefined as a table in the encoder and / or decoder.

[0120] In one implementation, the Inseparable Secondary Transform (NSST) can be conditionally applied to the secondary transform. For example, the NSST can be applied only to intra-prediction blocks and can have a transform set that can be applied to each prediction mode group.

[0121] In this case, prediction mode groups can be configured based on symmetry relative to the prediction direction. For example, since prediction mode 52 and prediction mode 16 are symmetric based on prediction mode 34 (diagonal direction), they can be formed as a group to apply the same set of transformations. In this case, when the transformation for prediction mode 52 is applied, the input data is transposed and then applied because prediction mode 52 has the same set of transformations as prediction mode 16.

[0122] Furthermore, since there is no directional symmetry in the planar and DC modes, each mode has a different transformation set, and the corresponding transformation set can include two transformations. For the remaining directional modes, each transformation set can include three transformations.

[0123] The quantization unit 130 can perform quantization on the auxiliary transform signal.

[0124] The inverse quantization unit and the inverse transformation unit 140 / 150 perform the above process in reverse, so their redundant description is omitted.

[0125] Figure 5 This is a schematic block diagram of the dequantization unit 220 and the inverse transform unit 230 in the decoder.

[0126] Referring to the above Figure 5 The dequantization and inverse transformation units 220 and 230 may include a dequantization unit 220, an inverse auxiliary transformation unit 231, and an inverse main transformation unit 232.

[0127] The dequantization unit 220 obtains the transform coefficients from the entropy-decoded signal by using quantization step size information.

[0128] The inverse auxiliary transformation unit 231 performs the inverse auxiliary transformation of the transformation coefficients. In this case, the inverse auxiliary transformation represents... Figure 4 The inverse transform of the auxiliary transform described in [the text].

[0129] The inverse master transform unit 232 performs the inverse master transform of the inverse auxiliary transform signal (or block) and obtains the residual signal. In this case, the inverse master transform represents... Figure 4 The inverse transform of the principal transform described in [the text].

[0130] In one implementation, a combination of multiple transforms of the MTS (DCT-2, DST-7, DCT-8) can be applied to the main transform. For example, it can be done as follows: Figure 6a and Figure 6b The transformation type is determined as shown in the table.

[0131] Figure 6a and Figure 6b An example table is shown for determining the transformation type in the horizontal and vertical directions for each prediction pattern. Figure 6a An example of a table is shown for determining the transformation type in the horizontal / vertical direction in intra-frame prediction mode. Figure 6b An example of a table is shown for determining the transformation type in the horizontal / vertical direction in inter-frame prediction mode. Figure 6a and Figure 6b This is an example of a combination table used to determine the transformation type, showing an MTS combination applied to the Joint Exploration Model (JEM). Another combination can also be used. For example, Figure 6b The table can be used for both intra-frame prediction and inter-frame prediction. Basically, refer to... Figure 6a and Figure 6b Describe an example applied to JEM.

[0132] In JEM, the application of MTS can be switched on / off at the block level (CU level in the case of HEVC) due to the introduction of a syntax element called EMT_CU_flag (or MTS_CU_flag). That is, in intra-prediction mode, when MTS_CU_flag is 0, DCT-2 or DST-7 from existing High Efficiency Image Coding (HEVC) is used (for 4×4 blocks). When MTS_CU_flag is 1, it uses... Figure 6a The MTS combination proposed in [the text]. For example... Figure 6aAs shown, the possible MTS combinations can vary depending on the intra-frame prediction mode. For example, compared to modes 14, 15, 16, 17, 18, 19, 20, 21, and 22, a total of four combinations are allowed because DST-7 and DCT-5 are used in the horizontal direction and DST-7 and DCT-8 are used in the vertical direction. Therefore, it is necessary to signal the use of one of the four combinations. One of the four combinations is selected using a two-bit MTS_TU_index. Figure 6b This illustrates an MTS combination that can be applied in inter-frame prediction mode. Figure 6a The difference lies in the fact that the determination may be based solely on a combination of DST-7 and DCT-8. According to embodiments of this disclosure, EMT_CU_flag can be used instead of MTS_CU_flag. For example, if intra-frame prediction mode is applied, then... Figure 6a Set 2, and if inter-frame prediction mode is applied, then it can be used Figure 6b .

[0133] Figure 7 This is an embodiment of the present disclosure, and a flowchart illustrating the encoding process of performing MTS.

[0134] In this disclosure, implementations of applying transformations in the horizontal and vertical directions are described substantially. However, the transformation combination may be configured with inseparable transformations.

[0135] Alternatively, a mixture of separable and non-separable transformations can be configured. In this case, if a non-separable transformation is used, the selection of transformations for each row / column or for each horizontal / vertical direction becomes unnecessary. The transformation can only be used when a separable transformation is selected. Figure 6a or Figure 6b Transformation and combination.

[0136] Furthermore, the methods proposed in this disclosure can be applied regardless of whether it is a primary or secondary transformation. That is, there is no limitation on applying the method to either the primary or secondary transformation alone, and the method can be applied to both the primary and secondary transformations. In this case, the primary transformation can refer to the transformation used to first transform the residual block. The secondary transformation can refer to the transformation used to apply the transformation to the block generated as a result of the primary transformation. According to embodiments of this disclosure, when the size of the transformation block is greater than or equal to 4×4, the secondary transformation can be applied to the upper left 4×4 or 8×8 region of the transformation block based on the width and height of the transformation block.

[0137] First, encoder 100 can determine the transformation configuration group corresponding to the current block (S710). In this case, the transformation configuration group can be composed of, for example, Figure 6a and Figure 6b Composed of combinations.

[0138] The encoder can perform transformations on the available candidate transformation combinations within the transformation configuration group (S720).

[0139] As a result of performing the transformation, the encoder can determine or select the transformation combination with the lowest rate distortion (RD) cost (S730).

[0140] The encoder can encode the transform combination index corresponding to the selected transform combination (S740).

[0141] Figure 8 This is an embodiment of the present disclosure, and a flowchart illustrating the decoding process of performing MTS.

[0142] First, decoder 200 can determine the transform configuration group of the current block (S810). Decoder 200 can parse (or obtain) the transform combination index based on the image signal. In this case, the transform combination index can correspond to one of multiple transform combinations within the transform configuration group (S820). For example, the transform configuration group can include DST-4, DCT-4, DST-7, and DCT-8. The transform combination index can be referred to as the MTS index. In one implementation, the transform configuration group can be configured based on at least one of the prediction mode, block size, or block shape of the current block.

[0143] Decoder 100 can derive a transform combination corresponding to the transform combination index (S830). In this case, the transform combination can consist of horizontal and vertical transforms, and can include at least one of DCT-2, DST-7, or DCT-8. Furthermore, refer to... Figure 6a or Figure 6b The described transformation combinations can be used for transformation combinations. That is, in this disclosure, a configuration based on another transformation combination according to another embodiment is possible.

[0144] Decoder 100 can perform an inverse transform on the current block based on the derived transform combination (S840). If the transform combination is configured using row (horizontal) transform and column (vertical) transform, the column (vertical) transform can be applied after the row (horizontal) transform is applied first, but this disclosure is not limited thereto. If the transform combination is configured in the opposite manner or using an inseparable transform configuration, an inseparable transform can be applied.

[0145] In one implementation, if the vertical or horizontal transform is DST-7 or DCT-8, the inverse transform of DST-7 or DST-8 can be applied to each column, and then to each row. Furthermore, in the vertical or horizontal transform, different transforms can be applied to each row and / or each column.

[0146] In one implementation, the transform combination index can be obtained based on an MTS flag indicating whether an MTS is performed. That is, the transform combination index can be obtained only when an MTS is performed based on the MTS flag. Furthermore, the decoder 100 can check whether the number of non-zero coefficients is greater than a threshold. In this case, the transform combination index can be obtained only when the number of non-zero coefficients is greater than the threshold.

[0147] In one implementation, an MTS tag or MTS index can be defined at at least one level of sequence, image, slice, block, coding unit, transform unit, or prediction unit.

[0148] In one implementation, the inverse transformation can only be applied when both the width and height of the transformation unit are 32 or less.

[0149] In another embodiment, the process of determining the transform configuration group and the process of parsing the transform combination index can be performed simultaneously. Alternatively, step S810 can be pre-configured and omitted in the encoder 100 and / or decoder 200.

[0150] Figure 9 This is an implementation of the present disclosure and a flowchart describing the process of encoding MTS tags and MTS indexes.

[0151] Encoder 100 can determine whether to apply MTS to the current block (S910).

[0152] If MTS is applied, encoder 100 can encode the MTS tag as 1 (S920).

[0153] Furthermore, encoder 100 can determine the MTS index based on at least one of the prediction mode, horizontal transform, or vertical transform of the current block (S930). In this case, the MTS index refers to an index that indicates any one of the multiple transform combinations used for each intra-frame prediction mode, and the MTS index can be sent for each transform unit.

[0154] Once the MTS index is determined, the encoder 100 can encode the MTS index determined in step S930 (S940).

[0155] Furthermore, if MTS is not applied, encoder 100 can encode the MTS mark as 0 (S950).

[0156] Figure 10 This is an implementation of the present disclosure, and is a flowchart illustrating the decoding process of applying horizontal or vertical transformations to rows or columns based on MTS tags and MTS indexes.

[0157] Decoder 200 can parse the MTS flag (S1010) based on the bitstream. In this case, the MTS flag can indicate whether to apply the MTS to the current block.

[0158] Decoder 200 can check whether to apply MTS to the current block based on the MTS flag (S1020). For example, decoder 200 can check whether the MTS flag is 1.

[0159] When the MTS flag is 1, the decoder 200 can check whether the number of non-zero coefficients is greater than (or equal to or greater than) a threshold (S1030). For example, the threshold for the number of transform coefficients can be set to 2. The threshold can be set based on the block size or the size of the transform unit.

[0160] When the number of non-zero coefficients exceeds a threshold, decoder 200 can resolve the MTS index (S1040). In this case, the MTS index refers to an index indicating any one of multiple transform combinations for each intra-frame prediction mode or each inter-frame prediction mode. The MTS index can be sent for each transform unit. Furthermore, the MTS index can represent an index indicating any transform combination defined in a pre-configured transform combination table. In this case, the pre-configured transform combination table can refer to... Figure 6a or Figure 6b The table is provided, but this disclosure is not limited thereto.

[0161] Decoder 100 can derive or determine the horizontal and vertical transforms based on at least one of the MTS index or the prediction mode (S1050). Furthermore, decoder 100 can derive transform combinations corresponding to the MTS index. For example, decoder 100 can derive or determine the horizontal and vertical transforms corresponding to the MTS index.

[0162] Furthermore, when the number of non-zero coefficients is no greater than a threshold, the decoder 200 can apply a pre-configured vertical inverse transform (S1060) to each column. For example, the vertical inverse transform can be the inverse transform of DST-7. Alternatively, the vertical inverse transform can be the inverse transform of DST-8.

[0163] Furthermore, the decoder can apply a pre-configured horizontal inverse transform (S1070) to each line. For example, the horizontal inverse transform can be the inverse transform of DST-7. Alternatively, the horizontal inverse transform can be the inverse transform of DST-8.

[0164] That is, when the number of non-zero coefficients is no greater than a threshold, the transform type pre-configured in encoder 100 or decoder 200 can be used. For example, it is not necessary to use the transform combination table (such as...). Figure 6a or Figure 6b The transformation type defined in the document is used instead of a general transformation type (e.g., DCT-2).

[0165] Furthermore, when the MTS is marked as 0, the decoder 200 can apply a pre-configured vertical inverse transform (S1080) to each column. For example, the vertical inverse transform can be the inverse transform of DCT-2.

[0166] Furthermore, the decoder 200 can apply a pre-configured horizontal inverse transform for each line (S1090). For example, the horizontal inverse transform can be the inverse transform of DCT-2. That is, when the MTS flag is 0, a transform type pre-configured in the encoder or decoder can be used. For example, it is not necessary to use a transform combination table (such as...) Figure 6a or Figure 6b The transformation type is defined in the original text, but a generic transformation type is used instead.

[0167] Figure 11 This describes an implementation of the present disclosure, and illustrates a flowchart of performing an inverse transformation based on transformation-related parameters.

[0168] The decoder 200 of this disclosure can be used to obtain either `sps_mts_intra_enabled_flag` or `sps_mts_inter_enabled_flag` (S1110). In this case, `sps_mts_intra_enabled_flag` indicates whether `tu_mts_flag` exists in the residual coding syntax of the coding unit (intra-coding unit) where intra-frame prediction is applied. For example, when `sps_mts_intra_enabled_flag` = 0, `tu_mts_flag` does not exist in the residual coding syntax of the intra-coding unit. When `sps_mts_intra_enabled_flag` = 0, `tu_mts_flag` exists in the residual coding syntax of the intra-coding unit. Furthermore, `sps_mts_inter_enabled_flag` indicates whether `tu_mts_flag` exists in the residual coding syntax of the coding unit (inter-coding unit) where inter-frame prediction is applied. For example, when sps_mts_inter_enabled_flag = 0, tu_mts_flag does not exist in the residual coding syntax of the inter-frame coding unit. When sps_mts_inter_enabled_flag = 0, tu_mts_flag exists in the residual coding syntax of the inter-frame coding unit.

[0169] Decoder 200 can obtain tu_mts_flag based on sps_mts_intra_enabled_flag or sps_mts_inter_enabled_flag (S1120). For example, when sps_mts_intra_enabled_flag = 1 or sps_mts_inter_enabled_flag = 1, decoder 200 can obtain tu_mts_flag. In this case, tu_mts_flag indicates whether the MTS is applied to the residual samples of the luminance transformation unit. For example, when tu_mts_flag = 0, the MTS is not applied to the residual samples of the luminance transformation unit. When tu_mts_flag = 1, the MTS is applied to the residual samples of the luminance transformation unit. At least one of the embodiments described in this disclosure can be applied with respect to tu_mts_flag = 1.

[0170] Decoder 200 can obtain mts_idx based on tu_mts_flag (S1130). For example, when tu_mts_flag = 1, the decoder can obtain mts_idx. In this case, mts_idx indicates whether to apply which transform kernel to the luminance residual sample according to the horizontal and / or vertical direction of the current transform block. For example, at least one of the embodiments of this disclosure can be applied to mts_idx. As a detailed example, it can be applied to... Figure 6a and Figure 6b At least one of the implementation methods.

[0171] Decoder 200 can derive the transform kernel corresponding to mts_idx (S1140). For example, the transform kernel corresponding to mts_idx can be divided and defined as horizontal transform and vertical transform.

[0172] For another example, different transform kernels can be applied to horizontal and vertical transforms, but this disclosure is not limited thereto. The same transform kernel can be applied to both horizontal and vertical transforms.

[0173] In one implementation, mts_idx can be defined as shown in Table 1.

[0174] [Table 1]

[0175] mts_idx[x0][y0] trTypeHor trTypeVer 0 0 0 1 1 1 2 2 1 3 1 2 4 2 2

[0176] Furthermore, the decoder 200 can perform an inverse transform based on the transform kernel derived in step S1140 (S1150).

[0177] exist Figure 11The implementation of obtaining tu_mts_flag to determine whether to apply MTS, obtaining mts_idx based on the obtained tu_mts_flag value, and determining the transform kernel has been basically described, but this disclosure is not limited thereto. For example, the decoder 200 can determine the transform kernel by directly parsing mts_idx without parsing tu_mts_flag. In this case, Table 1 can be used. That is, when the mts_idx value indicates 0, DCT-2 can be applied in the horizontal / vertical direction. When the mts_idx value indicates a value other than 0, DST-7 or DCT-8 can be applied based on the mts_idx value.

[0178] In another embodiment of this disclosure, a decoding process for performing the transformation process is described.

[0179] Decoder 200 can examine the transform size (nTbS). In this case, the transform size (nTbS) can be a variable indicating the horizontal sample size of the scaling transform coefficients.

[0180] Decoder 200 can check the transform kernel type (trType). In this case, the transform kernel type (trType) can be a variable indicating the type of transform kernel, and various implementations of this disclosure can be applied. The transform kernel type (trType) can include a horizontal transform kernel type (trTypeHor) and a vertical transform kernel type (trTypeVer).

[0181] Referring to Table 1, the kernel type (trType) can indicate DCT-2 when it is 0, DST-7 when it is 1, and DCT-8 when it is 2.

[0182] Decoder 200 can perform transform matrix multiplication based on at least one of transform size (nTbS) or transform kernel type.

[0183] For another example, when the transformation kernel type is 1 and the transformation size is 4, the previously determined transformation matrix 1 can be applied when performing transformation matrix multiplication.

[0184] For another example, when the transformation kernel type is 1 and the transformation size is 8, the previously determined transformation matrix 2 can be applied when performing transformation matrix multiplication.

[0185] For another example, when the transformation kernel type is 1 and the transformation size is 16, the previously determined transformation matrix 3 can be applied when performing transformation matrix multiplication.

[0186] For another example, when the transformation kernel type is 1 and the transformation size is 32, the previously defined transformation matrix 4 can be applied.

[0187] Similarly, when the transformation kernel type is 2 and the transformation size is 4, 8, 16 and 32, the previously defined transformation matrices 5, 6, 7 and 8 can be applied respectively.

[0188] In this case, each of the previously defined transformation matrices 1 through 8 can correspond to any of the various types of transformation matrices. For example, transformation matrices of the type shown in Figure 6 can be applied.

[0189] Decoder 200 can derive transformed samples based on transform matrix multiplication.

[0190] Implementation methods may be used, but this disclosure is not limited thereto. The above-described embodiments and other embodiments of this disclosure may be combined and used.

[0191] Figure 12 This is an implementation of the present disclosure, and is a table showing the assignment of transform sets to each intra-prediction mode in NSST.

[0192] The auxiliary transform unit 122 can apply the auxiliary transform to the signal after the main transform. In this case, the auxiliary transform can be predefined in a table in the encoder 100 and / or decoder 200.

[0193] In one implementation, NSST can be conditionally applied to the auxiliary transform. For example, NSST is applied only in the case of intra-prediction blocks and can have a set of applicable transforms for each prediction mode group.

[0194] According to embodiments of this disclosure, when the size of the transform block is greater than or equal to 4×4, NSST can be applied to the upper left 4×4 or 8×8 region of the transform block based on the width and height of the transform block.

[0195] In this case, prediction mode groups can be configured based on the symmetry of the prediction direction. For example, prediction mode 52 and prediction mode 16 are symmetrical about each other with respect to prediction mode 34 (diagonal direction), and therefore can form a group such that the same set of transformations can be applied to prediction mode 52 and prediction mode 16. In this case, when the transformation of prediction mode 52 is applied, the input data is transposed and applied. This is because prediction mode 52 and prediction mode 16 have the same set of transformations.

[0196] Furthermore, the planar mode and DC mode have their own transformation sets because there is no symmetry regarding direction. Each transformation set can be configured with two transformations. For the other directional modes, three transformations can be configured for each transformation set, but this disclosure is not limited to this. Each transformation set can be configured with multiple transformations.

[0197] Figure 13This describes an implementation of the present disclosure, and a flowchart of the calculation of Givens rotation is shown.

[0198] In another implementation, the NSST is not applied to the entire main transform block, but may be applied only to its top-left 8×8 region. For example, when the block size is equal to or greater than 8×8, an 8×8 NSST is applied. When the block size is less than 8×8, a 4×4 NSST is applied. In this case, after dividing the block into 4×4 blocks, a 4×4 NSST can be applied to each of the 4×4 blocks. According to embodiments of this disclosure, when the transform block size is greater than or equal to 4×4, the NSST can be applied only to the top-left 4×4 region of the transform block.

[0199] As another implementation, even in the case of 4×N / N×4 (N>=16), 4×4NSST can be applied.

[0200] Since both the 8×8NSST and the 4×4NSST follow the transform combination configuration described in this disclosure and are non-separable transforms, the 8×8NSST receives 64 data and outputs 64 data, while the 4×4NSST has 16 inputs and 16 outputs.

[0201] Both the 8×8 NSST and 4×4 NSST are configured as hierarchical combinations of Givens rotations. Equation 1 below shows the matrix corresponding to a Givens rotation, and Equation 2 below shows the matrix product.

[0202] [Equation 1]

[0203]

[0204] [Equation 2]

[0205] t m =x m cosθ-x n sinθ

[0206] t n =x m sinθ+x n cosθ

[0207] As above Figure 13 As shown, since one Givens rotation rotates two data points, a total of 32 or 8 Givens rotations are needed to process 64 data points (for 8×8 NSST) or 16 data points (for 4×4 NSST).

[0208] Therefore, a bundle of 32 or 8 is used to form the Givens rotation layer. The output data of one Givens rotation layer is transmitted as the input data for the next Givens rotation layer through a defined permutation.

[0209] Figure 14 A round configuration of a 4×4 NSST consisting of Givens rotating layers and displacements is shown as an embodiment in which this disclosure is applied.

[0210] Refer to above Figure 14 This illustrates the sequential processing of four Givens rotating layers in a 4×4 NSST configuration. (See above.) Figure 14 As shown, the output data of one Givens rotation layer is transmitted to the input data of the next Givens rotation layer through the determined permutation (i.e., reorganization).

[0211] As above Figure 14 As shown, the pattern to be permuted is determined systematically. In the case of 4×4NSST, four Givens rotation layers and corresponding permutations are combined to form a round.

[0212] In the case of 8×8 NSST, six Givens rotation layers and their corresponding permutations form one round. 4×4 NSST goes through two rounds, while 8×8 NSST goes through four rounds. Different rounds use the same permutation pattern, but the applied Givens rotation angles differ. Therefore, it is necessary to store the angle data of all Givens rotations that constitute each transformation.

[0213] As a final step, a permutation is performed on the data output through the Givens rotation layer, and the corresponding permutation information is stored for each transformation. In the forward NSST, the corresponding permutation is performed last, while in the inverse NSST, the corresponding inverse permutation is applied first.

[0214] In the case of inverse NSST, Givens rotation layers and permutations applied to forward NSST are performed in reverse order, and rotations are performed by even taking negative values ​​for the angle of each Givens rotation.

[0215] Furthermore, according to one implementation, the reduced auxiliary transformation (RST), which will be described below, can be used instead of the NSST.

[0216] Figure 15 This is an embodiment of the present disclosure and a block diagram for describing the operations of the forward reduction transform and the inverse reduction transform.

[0217] Reduced auxiliary transformation (RST)

[0218] Assuming the orthogonal matrix representing a transformation has an N×N form, reducing the transformation (hereinafter referred to as "RT") leaves only R (R < N) of the N transformation fundamental vectors. The matrix used to generate the forward RT for the transformation coefficients is given in Equation 3 below.

[0219] [Equation 3]

[0220]

[0221] The inverse RT matrix is ​​the transpose of the forward RT matrix. Applications of forward RT and inverse RT include... Figure 15 As shown.

[0222] If we apply RT to the top left 8×8 block of a transform block that has undergone the main transform, then this RT can be called an 8×8 reduced auxiliary transform (8×8RST).

[0223] Assuming the value of R in Equation 3 is 16, the forward 8×8RST has a 16×64 matrix form, and the inverse 8×8RST has a 64×16 matrix form.

[0224] In addition, with Figure 12 The same transform set configuration is applied to 8×8 RST. That is, it can be based on... Figure 12 The transformation set in the code is used to apply the corresponding 8×8 RST.

[0225] As one implementation method, in Figure 12 In this context, assuming that a transform set consists of two or three transforms according to the intra-frame prediction mode, one of the maximum values ​​among the four transforms, including the case where no auxiliary transform is applied, can be configured to be selected. In this case, a transform can be treated as an identity matrix.

[0226] If indices 0, 1, 2, and 3 are assigned to the four transformations respectively, the corresponding transformation can be specified by signaling the syntax element called the NSST index for each transformation block. That is, in the case of NSST, an 8×8 NSST can be specified based on the NSST index relative to the top-left 8×8 block. In the RST configuration, an 8×8 RST can be specified. Furthermore, in this case, index 0 can be assigned to the identity matrix, i.e., the case where no auxiliary transformation is applied.

[0227] If a forward 8×8 RST is applied (e.g., Equation 3), 16 effective transform coefficients are generated. Therefore, it is possible to reduce the 64 input data points forming the 8×8 region to 16 output data points. From a two-dimensional region viewpoint, only 1 / 4 of the region is filled with effective transform coefficients. Therefore, Figure 16 The 4×4 top-left region can be filled with 16 output data obtained by applying a forward 8×8 RST.

[0228] Furthermore, as mentioned above, the low-frequency non-separable transform (LFNST) applied to the low-frequency domain (e.g., the upper left 4×4 region of the transform block) can be used as an auxiliary transform.

[0229] Figure 16 This is an embodiment of the present disclosure, and is a diagram illustrating the process of performing a reverse scan from the 64th to the 17th according to the reverse scan sequence.

[0230] Figure 16 This illustrates the assumption that the forward scan order starts from 1, and that the scan is performed on coefficients 17 through 64 (in the forward scan order). However, Figure 16 The inverse scan is shown, and the inverse scan is performed on coefficients 64 through 17.

[0231] refer to Figure 16 The top-left 4x4 region is the region of interest (ROI) with valid transform coefficients, while the remaining regions are cleared. In other words, by default, the value 0 can be assigned to the remaining regions.

[0232] If in Figure 16 The presence of valid non-zero transform coefficients within the ROI region means that 8×8 RST should not be applied. In this case, the corresponding NSST index encoding can be omitted.

[0233] Conversely, if in Figure 16 If there are no non-zero transform coefficients in regions outside the ROI (if 8×8 RST is applied, 0 is assigned to regions outside the ROI), then the NSST index can be encoded since 8×8 RST may have already been applied.

[0234] As mentioned above, since it is necessary to check for the existence of non-zero transform coefficients, conditional NSST index coding can be performed after the residual coding process.

[0235] According to embodiments of this disclosure, when the size of the transform block is greater than or equal to 4×4, NSST can be applied to the upper left 4×4 or 8×8 region of the transform block.

[0236] Figure 17 This describes an implementation of the present disclosure and illustrates a flowchart of encoding an image signal through a primary transform and an auxiliary transform.

[0237] Encoder 100 may determine (or select) a forward auxiliary transform based on at least one of the prediction mode, block shape, and / or block size of the current block (S1710). In this case, candidates for the forward auxiliary transform may include those shown in Figure 6 and / or Figure 12At least one of the embodiments. Furthermore, according to embodiments of this disclosure, encoder 100 can determine the forward auxiliary transform by taking into account an intra-prediction mode modified by applying a wide-angle intra-prediction mode as described later.

[0238] Encoder 100 can determine the optimal forward auxiliary transform through rate distortion (RD) optimization. The optimal forward auxiliary transform may correspond to one of multiple transform combinations. Multiple transform combinations may be defined by transform indices. For example, for RD optimization, encoder 100 may compare the results of all forward auxiliary transforms, quantization, residual coding, etc., performed on each candidate. In this case, equations such as cost = rate + λ·distortion or cost = distortion + λ·rate may be used, but this disclosure is not limited thereto.

[0239] The encoder 100 can signal the auxiliary transform index corresponding to the optimal forward auxiliary transform (S1720). In this case, other embodiments described in this disclosure can be applied to the auxiliary transform index.

[0240] For example, Figure 12 The transform set configuration can be used as an auxiliary transform index. Depending on the intra-prediction mode, a transform set consists of 2 or 3 transforms. Therefore, encoder 100 can be configured to select the largest of four transforms, including the case where no auxiliary transform is applied. Assuming indices 0, 1, 2, and 3 are assigned to the four transforms respectively, encoder 100 can specify the transform to be applied by signaling the auxiliary transform index for each transform coefficient block. In this case, encoder 100 can assign index 0 to the identity matrix, i.e., the case where no auxiliary transform is applied. Furthermore, as... Figures 27a to 30b As shown, for the application of wide-angle intra-frame prediction mode, the auxiliary transform index can be constructed like an index mapping table.

[0241] In another implementation, the signaling of the secondary transform index can be performed in any of the following steps: 1) before residual encoding, 2) during residual encoding (after encoding at the last non-zero transform coefficient position), or 3) after residual encoding. The implementation is described in detail below.

[0242] 1) A method for signaling the auxiliary transform index before residual coding.

[0243] Encoder 100 can determine the forward auxiliary transform.

[0244] Encoder 100 can encode the auxiliary transform index corresponding to the forward auxiliary transform.

[0245] Encoder 100 can encode the position of the last non-zero transform coefficient.

[0246] Encoder 100 can perform residual encoding on syntax elements except for the positions of the last non-zero transform coefficients.

[0247] 2) A method for signaling the auxiliary transform index during residual coding.

[0248] Encoder 100 can determine the forward auxiliary transform.

[0249] Encoder 100 can encode the position of the last non-zero transform coefficient.

[0250] If the non-zero transform coefficients are not located in a specific region, the encoder 100 can encode the auxiliary transform index corresponding to the forward auxiliary transform. In this case, if a reduced auxiliary transform is applied, when the transform coefficients are set according to the scan order, the specific region indicates the region other than the location where non-zero transform coefficients may exist, but this disclosure is not limited thereto.

[0251] Encoder 100 can perform residual encoding on syntax elements except for the positions of the last non-zero transform coefficients.

[0252] 3) A method for signaling the auxiliary transform index after residual encoding.

[0253] Encoder 100 can determine the forward auxiliary transform.

[0254] Encoder 100 can encode the position of the last non-zero transform coefficient.

[0255] Encoder 100 can perform residual encoding on syntax elements except for the positions of the last non-zero transform coefficients.

[0256] If the non-zero transform coefficients are not located in a specific region, the encoder 100 can encode the auxiliary transform index corresponding to the forward auxiliary transform after residual encoding. In this case, if a reduced auxiliary transform is applied, when the transform coefficients are set according to the scan order, the specific region indicates the remaining region other than the location where non-zero transform coefficients may exist, but this disclosure is not limited thereto.

[0257] Furthermore, encoder 100 can perform a forward master transform (S1730) on the current block (residual block). In this case, steps S1710 and / or S1720 can be similarly applied to the forward master transform.

[0258] The encoder 100 can perform a forward auxiliary transform on the current block using the optimal forward auxiliary transform (S1740). For example, the forward auxiliary transform can be a reduced auxiliary transform. A reduced auxiliary transform refers to a transform that takes N residual data (N×1 residual vectors) as input and outputs L (L<N) transform coefficient data (L×1 transform coefficient vectors).

[0259] In one implementation, the reduced auxiliary transform can be applied to a specific region of the current block. For example, when the current block is N×N, the specific region may refer to the top-left N / 2×N / 2 region, but this disclosure is not limited thereto. The specific region can be configured differently based on at least one of the prediction mode, block shape, or block size. For example, when the current block is N×N, the specific region may refer to the top-left M×M region (M≤N). Furthermore, an inseparable transform (low-frequency inseparable transform) can be applied to at least some regions (e.g., the top-left 4×4 region) corresponding to the low-frequency domain of the transformed block as an auxiliary transform.

[0260] In addition, encoder 100 can generate transform coefficient blocks by performing quantization on the current block (S1750).

[0261] Encoder 100 can generate a bit stream by performing entropy encoding on the transform coefficient block.

[0262] Figure 18 This describes an implementation of the present disclosure, and illustrates a flowchart of decoding an image signal through inverse auxiliary transformation and inverse main transformation.

[0263] Decoder 200 can obtain the secondary transform index from the bitstream (S1810). In this case, other embodiments described in this disclosure can be applied to the secondary transform index. For example, the secondary transform index may include... Figure 6a , Figure 6b and / or Figure 12 At least one of the implementation methods. Furthermore, it can be based on... Figures 27a to 30b The index mapping table shown determines the secondary transformation index.

[0264] In another implementation, the step of obtaining the secondary transform index can be performed in any of the following steps: 1) before residual decoding, 2) during residual decoding (after decoding the positions of transform coefficients that are not the last 0), or 3) after residual decoding.

[0265] Decoder 200 can derive the auxiliary transform corresponding to the auxiliary transform index (S1820). In this case, candidates for the auxiliary transform can include Figure 6 and / or Figure 12 At least one of the implementation methods.

[0266] In this context, steps S1810 and S1820 are implementation methods, and this disclosure is not limited thereto. For example, decoder 200 may derive the secondary transform based on at least one of the prediction mode, block shape, and / or block size of the current block without obtaining the secondary transform index.

[0267] Furthermore, the decoder 200 can obtain the transform coefficient block by entropy decoding of the bit stream, and can perform inverse quantization on the transform coefficient block (S1830).

[0268] Decoder 200 can perform an inverse auxiliary transform (S1840) on the inverse quantized transform coefficient block. For example, the inverse auxiliary transform can be a reduced auxiliary transform. A reduced auxiliary transform refers to a transform that takes N residual data (N×1 residual vectors) as input and outputs L (L<N) transform coefficient data (L×1 transform coefficient vectors).

[0269] In one implementation, the reduced auxiliary transformation can be applied to a specific region of the current block. For example, when the current block is N×N, the specific region may refer to the top-left N / 2×N / 2 region, but this disclosure is not limited thereto. The specific region can be configured differently based on at least one of the prediction mode, block shape, or block size. For example, when the current block is N×N, the specific region may refer to the top-left M×M region (M≤N) or M×L region (M≤N, L≤N).

[0270] In addition, the non-separable transform (low-frequency non-separable transform) applied to the low-frequency domain (e.g., the upper left 4×4 region) of the transform block including the inverse quantization transform coefficients can be used as the inverse auxiliary transform.

[0271] In addition, the decoder 200 can perform an inverse main transformation (S1850) on the result of the inverse auxiliary transformation.

[0272] Decoder 200 generates a residual block in step S1850 and generates a reconstructed block by adding the residual block and the prediction block.

[0273] The following describes a method and apparatus for performing an effective transformation while further improving prediction accuracy by considering the wide-angle intra-frame prediction mode in the above-described process of determining the auxiliary transform (or inverse auxiliary transform).

[0274] The embodiments of this disclosure, which will be described later below, relate to methods and apparatus for encoding / decoding still or moving images, and more specifically to auxiliary transform set mapping for wide-angle mode when wide-angle intra-frame prediction (WAIP) is applied, and methods and apparatus for applying auxiliary transforms when the mode changes to wide-angle due to WAIP.

[0275] Embodiments of this disclosure provide a method and apparatus for allocating an auxiliary transform set to a wide-angle mode when applying WAIP.

[0276] Furthermore, embodiments of this disclosure provide a method and apparatus for applying a corresponding auxiliary transformation by transposing (or not transposing) the transformed input data based on the changed prediction direction when the prediction direction changes due to the fulfillment of the conditions for applying WAIP.

[0277] The embodiments of this disclosure can improve compression performance when encoding or decoding still or moving images by applying a suitable auxiliary transform set to each wide-angle mode according to the application of the wide-angle intra-frame prediction mode and arranging input data based on the wide-angle mode.

[0278] In the following text, the inseparable transform described herein may correspond to the aforementioned low-frequency inseparable transform (LFNST), inseparable auxiliary transform (NSST), and / or reduced auxiliary transform (RST).

[0279] Figure 19 This is an implementation of the present disclosure, and an example of an intra-prediction mode configuration consisting of 67 modes is shown.

[0280] Intra-frame prediction methods (intra-prediction) include planar mode and DC mode, i.e., non-directional prediction mode, and multiple directional prediction modes with their own directions. Existing image compression standards (e.g., HEVC) use 35 intra-frame prediction modes. 33 of these 35 intra-frame prediction modes correspond to directional prediction modes. In the currently discussed JEM or Universal Image Coding (VVC) standards, 67 intra-frame prediction modes are now considered or used. These 67 intra-frame prediction modes include planar mode, DC mode, and 65 directional intra-frame prediction modes. The configuration of these 67 intra-frame prediction modes can be as follows: Figure 19 That's how it's expressed.

[0281] exist Figure 19 In the diagram, the portions indicated by solid arrows correspond to the existing 35 intra-prediction modes, while the portions indicated by dashed arrows correspond to the added directional intra-prediction modes.

[0282] Figure 20 This is an example of an implementation of the present disclosure, and an example of a mapping table for 67 intra-prediction modes for 35 intra-prediction modes is shown.

[0283] The mapping from the indices of the existing 35 intra-prediction modes to the indices of the 67 intra-prediction modes can be like... Figure 20 That configuration.

[0284] like Figure 20 As shown, in the 67 mode configurations, index 0 and index 1 correspond to the planar mode and DC mode, respectively. In the following text, the configuration of intra-prediction with 35 intra-prediction modes is referred to as "35 mode" or "35 mode configuration". The configuration of intra-prediction with 67 intra-prediction modes is referred to as "67 mode" or "67 mode configuration".

[0285] Figure 21This describes an implementation of the present disclosure and illustrates an example of configuring two additional wide-angle modes in 35 intra-frame prediction modes.

[0286] The examples of wide-angle intra-frame prediction described through the embodiments of this disclosure can be compared with... Figure 21 same. Figure 21 This example shows a case where two wide-angle modes, 35 and 36, are added to the right after mode 34 in a 35-mode configuration.

[0287] Figure 22 This describes an implementation of the present disclosure, and illustrates an example of configuring an additional 10 wide-angle modes among 35 intra-frame prediction modes.

[0288] Figure 22 This example illustrates the case where, in mode 35, five wide-angle modes are added in the lower direction and five wide-angle modes are added in the upper direction. The modes added in the lower direction have indices starting from the top: -1, -2, -3, -4, and -5, while the modes added in the upper direction have indices starting from the left: 35, 36, 37, 38, and 39. Including the modes added in mode 35, in mode 67, ten modes can be added on each side, both upper and lower. In this case, the indices of the ten lower modes can be -1, -2, ..., -10, and the indices of the ten upper modes can be 67, 68, ..., 76. (The example is missing from the original text.) Figure 23 As shown, a mapping table can be configured between modes 35 and 67 for the added modes.

[0289] Figure 23 This describes an implementation of the present disclosure and illustrates an example of an index mapping table between 35 intra-frame prediction modes and a wide-angle mode added in 67 intra-frame prediction modes.

[0290] In 67 mode, with Figure 23 The indices corresponding to the wide-angle modes not included in the index are -1, -3, -5, -7, -9, 67, 69, 71, 73, and 75. Based on the 67 mode, the indices can appear between 2 and -2, between -2 and -4, between -4 and -6, between -6 and -8, between -8 and -10, between 66 and 68, between 68 and 70, between 70 and 72, between 72 and 74, and between 74 and 76.

[0291] According to embodiments of this disclosure, a modified intra-prediction mode (wide-angle intra-prediction mode) can be used instead of a specific intra-prediction mode if certain conditions are met. For example, prediction can be performed using mode 35 instead of mode 2 based on mode 35.

[0292] If the width (horizontal length of the transform block (or transform unit)) is written as nWidth and its height (vertical length) is written as nHeight, then the index of the intra-prediction mode (predModeIntra) can be changed as shown in Table 2.

[0293] [Table 2]

[0294]

[0295] The conditions for using the modified intra-prediction mode based on the application of the wide-angle intra-prediction mode according to Table 2, and the configuration of the index of the modified intra-prediction mode are as follows.

[0296] (a) When the ratio of the width to the height of the transform block (nWidth / nHeight) is less than or equal to 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 2 and less than or equal to 4 (nWidth / nHeight <= 2 and 2 <= predModeIntra <= 4), the modified index of the intra-prediction mode is set to the value obtained by adding 33 to the index of the intra-prediction mode (predModeIntra = predModeIntra + 33).

[0297] (b) When the ratio of the width to the height of the transform block (nWidth / nHeight) is greater than 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 2 and less than or equal to 6 (nWidth / nHeight>2 and 2<=predModeIntra<=6), the modified index of the intra-prediction mode is set to the value obtained by adding 33 to the index of the intra-prediction mode (predModeIntra=predModeIntra+33).

[0298] (c) When the ratio of the height to the width of the transform block (nHeight / nWidth) is less than or equal to 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 32 and less than or equal to 34 (nHeight / nWidth <= 2 and 32 <= predModeIntra <= 34), the index of the modified intra-prediction mode is set to the value obtained by subtracting 35 from the index of the intra-prediction mode (predModeIntra = predModeIntra - 35).

[0299] (d) When the ratio of the height to the width of the transform block (nHeight / nWidth) is greater than 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 30 and less than or equal to 34 (nHeight / nWidth>2 and 30<=predModeIntra<=34), the index of the modified intra-prediction mode is set to the value obtained by subtracting 35 from the index of the intra-prediction mode (predModeIntra=predModeIntra-35).

[0300] In Table 2, the modified intra-prediction mode has a negative value compared to cases (c) and (d). Based on Figure 22 In the downward direction, -1 is assigned in mode 2, and the index values ​​decrease one after another towards the bottom.

[0301] Based on the 67 mode, the conditions for using the modified intra-prediction mode and the configuration of the index of the modified intra-prediction mode can be the same as in Table 3.

[0302] [Table 3]

[0303]

[0304] (a) When the ratio of the width to the height of the transform block (nWidth / nHeight) is less than or equal to 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 2 and less than or equal to 7 (nWidth / nHeight <= 2 and 2 <= predModeIntra <= 7), the modified index of the intra-prediction mode is set to the value obtained by adding 65 to the index of the intra-prediction mode (predModeIntra = predModeIntra + 65).

[0305] (b) When the ratio of the width to the height of the transform block (nWidth / nHeight) is greater than 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 2 and less than or equal to 11 (nWidth / nHeight>2 and 2<=predModeIntra<=11), the modified index of the intra-prediction mode is set to the value obtained by adding 65 to the index of the intra-prediction mode (predModeIntra=predModeIntra+65).

[0306] (c) When the ratio of the height to the width of the transform block (nHeight / nWidth) is less than or equal to 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 61 and less than or equal to 66 (nHeight / nWidth <= 2 and 61 <= predModeIntra <= 66), the index of the modified intra-prediction mode is set to the value obtained by subtracting 67 from the index of the intra-prediction mode (predModeIntra = predModeIntra - 67).

[0307] (d) When the ratio of the height to the width of the transform block (nHeight / nWidth) is greater than 2, and the index of the intra-prediction mode (predModeIntra) is greater than or equal to 57 and less than or equal to 66 (nHeight / nWidth>2 and 57<=predModeIntra<=66), the index of the modified intra-prediction mode is set to the value obtained by subtracting 67 from the index of the intra-prediction mode (predModeIntra=predModeIntra-67).

[0308] That is, in Tables 3(a) and (b), when the width of the transform block is greater than its height, and the intra-prediction mode is greater than or equal to 2 and less than the first reference value, the value obtained by adding 65 to the intra-prediction mode is determined as the modified intra-prediction mode. In this case, when the ratio of the width of the transform block to its height is less than or equal to 2, the first reference value can be set to 8 (case (a)), and when the ratio of the width of the transform block to its height is greater than 2, the first reference value can be set to 12 (case (b)).

[0309] Furthermore, in Tables 3(c) and (d), when the width of the transform block is less than its height, and the intra-prediction mode is greater than or equal to the second reference value and less than or equal to 66, the value obtained by subtracting 67 from the intra-prediction mode is determined as the modified intra-prediction mode. In this case, when the ratio of the transform block's height to its width is less than or equal to 2, the second reference value can be set to 61 (case (c)), and when the ratio of the transform block's width to its height is greater than 2, the second reference value can be set to 57 (case (d)).

[0310] refer to Figure 24 and Figure 25 Describe situations where the above-described wide-angle intra-frame prediction method can be advantageously applied.

[0311] Figure 24 and Figure 25 This describes an implementation of the present disclosure, and an example of intra-frame prediction for non-square blocks is shown.

[0312] If the width of the block is greater than its height, such as Figure 24 As shown, typically, the reference sample located at the top is closer to the block to be predicted than the reference sample located on the left. Therefore, a prediction performed in the lower left direction may be more accurate than a prediction performed in the upper right direction. Conversely, as... Figure 25 As shown, if the block height is greater than the block width, the reference sample on the left is typically closer to the position within the block to be predicted than the reference sample on the top. Therefore, a prediction performed in the upper right direction may be more accurate than a prediction performed in the lower left direction. Thus, transforming the index of the aforementioned intra-prediction mode can be more advantageous. Regarding modes 35 and 67, the intra-prediction modes (i.e., the targets of the transformation) can be summarized as shown in Tables 4 and 5.

[0313] [Table 4]

[0314] condition Replaced intra-prediction mode W / H==2 Modes 2, 3, and 4 W / H>2 Patterns 2, 3, 4, 5, 6 W / H==1 none H / W = 1 / 2 Patterns 32, 33, and 34 H / W<1 / 2 Patterns 30, 31, 32, 33, 34

[0315] [Table 5]

[0316] condition Replaced intra-prediction mode W / H==2 Patterns 2, 3, 4, 5, 6, 7 W / H>2 Patterns 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 W / H==1 none H / W = 1 / 2 Patterns 61, 62, 63, 64, 65, 66 H / W<1 / 2 Patterns 57, 58, 59, 60, 61, 62, 63, 64, 65, 66

[0317] Table 4 shows the intra-prediction modes to be transformed in the 35-mode configuration, and Table 5 shows the intra-prediction modes to be transformed in the 67-mode configuration. In Tables 4 and 5, W represents the width of the transform block, and H is the height of the transform block.

[0318] In one implementation, the index of the wide-angle intra-prediction mode is encoded before the index transformation. That is, in the 67-mode configuration, although intra-prediction mode 2 is transformed into intra-prediction mode 67, 2 (i.e., the index of the previous mode) is encoded.

[0319] Figure 26 illustrates an embodiment of the present disclosure and shows an example of the order in which pixels are arranged when a two-dimensional block is changed to a one-dimensional vector, wherein... Figure 26a An example of row precedence is shown, and Figure 26b An example of column priority order is shown.

[0320] As mentioned above, if the wide-angle intra-frame prediction mode is applied, such as Figure 22 As shown, a direction corresponding to the wide-angle intra-prediction mode is added to the existing intra-prediction. The prediction mode added on the lower side is represented as ML1 to ML. N The prediction patterns added on the upper side are represented as ML1 to ML. N For example, the pattern closest to pattern 2 is ML1, and the indices (1 to N) corresponding to the various prediction patterns in the lower direction increase (prediction pattern values ​​ML1 to ML). N (reduced), and the bottommost mode is represented as ML.N For patterns 35 and 67, the patterns closest to 34 and 66 are represented as MT1, with indices (1 to M) increasing to the right, and the rightmost pattern represented as MT1. M .

[0321] Regarding mode ML1 to ML N and ML1 to ML M , can be like Figure 12 The map shown is an auxiliary transformation set. Figure 12 In this context, the same auxiliary transform set is applied to directional pattern pairs (e.g., patterns 32 and 36) that are symmetrical to the diagonal direction (pattern 34 in pattern 67 and pattern 18 in pattern 35). The same or similar methods can be applied to ML1 through ML2. N and MT1 to MT M That is, wide-angle intra-frame prediction mode. For example, if M = N and MLa and MTa are symmetrical about each other about the diagonal direction (a = 1, ..., N), the same transform set can be applied to MLa and MTa.

[0322] In this case, relative to the MTa mode, after first transposing the two-dimensional input data, an auxiliary transformation is applied to MLa. That is, as follows: Figure 26a and Figure 26b As shown, if the transformation is applied to MLa after converting two-dimensional input data into one-dimensional input data by reading it along a row-first (or column-first) direction, then the same transformation is applied to MTa after converting the input data into one-dimensional data by reading it along a column-first (or row-first) direction. Figure 26a and Figure 26b The numerical indicator for each position is used to indicate the index of the pixel position, not the pixel value.

[0323] Figure 27 illustrates an embodiment of this disclosure and shows an example of a mapping table between 35 transform sets and intra-frame prediction modes, wherein... Figure 27a An example is shown where the transform set is symmetrically assigned to each wide-angle mode. Figure 27b An example is shown where transform set 2 is assigned to all wide-angle modes, and Figure 27c An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0324] exist Figure 27a In, with Figure 12 The same transform set index can be assigned to patterns 0 through 66. Different indices of the additional transform sets can be assigned to the pattern pairs corresponding to the wide-angle patterns (where the pattern pairs correspond to (MLa, MTa), a = 1, 2, ..., 10).

[0325] In addition, such as Figure 27b As shown, transform set index 2 can be reused in other wide-angle modes. (See figure.) Figure 27c As shown, a separate transform set index 35 can be assigned to other wide-angle modes.

[0326] Figure 28 illustrates an embodiment of the present disclosure and shows an example of a mapping table between 10 transform sets and intra-frame prediction modes, wherein... Figure 28a An example is shown where transform set 2 is assigned to each wide-angle mode, and Figure 28b An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0327] like Figure 28a and Figure 28b As shown, 10 transform sets can be mapped to intra-prediction modes from 0 to 66. Figure 28a and Figure 28b The indices shown are used to distinguish between transformation sets. Figures 27a to 27c The same index shown can correspond to the same transformation set or to different transformation sets.

[0328] like Figure 28a As shown, transform set 2 can be reused for other wide-angle intra-frame prediction modes, and transform set 35 can be assigned as follows: Figure 28b The individual transformation set shown.

[0329] Figure 29 illustrates an implementation of this disclosure and shows an example of a mapping table between six transform sets and intra-frame prediction modes, wherein... Figure 29a An example is shown where transform set 2 is assigned to each wide-angle mode, and Figure 29b An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0330] The six transform sets can be mapped to intra-prediction modes 0 through 66, such as Figure 29a and 29b As shown. Figure 29a and Figure 29b The indices shown are used to distinguish between transform sets. Figures 27a to 28b The same index can correspond to the same transformation set, or the index can correspond to different transformation sets.

[0331] like Figure 29a As shown, transform set 2 can be reused for other wide-angle intra-frame prediction modes, and as... Figure 29b As shown, transform set 35 can be assigned to other wide-angle intra-prediction modes.

[0332] Figure 30 illustrates an embodiment of the present disclosure and shows an example of a mapping table between four transform sets and intra-frame prediction modes, wherein... Figure 30a An example is shown where transform set 2 is assigned to each wide-angle mode, and Figure 30b An example is shown where the same set of additional transforms is assigned to all wide-angle modes.

[0333] The four transform sets can be mapped to intra-prediction modes 0 through 66, such as Figure 30a and Figure 30b As shown. Figure 30a and Figure 30b The indices shown are used to distinguish between transform sets. Figures 27a to 28b The same index can correspond to the same transformation set, or the index can correspond to different transformation sets.

[0334] like Figure 30a As shown, transform set 2 can be reused for other wide-angle intra-frame prediction modes, and as... Figure 30b As shown, transform set 35 can be assigned to other wide-angle intra-prediction modes.

[0335] That is, such as Figure 30a As shown, when the modified intra-prediction mode is greater than or equal to 0 and less than or equal to 1 due to the application of the wide-angle intra-prediction mode, the index of the inseparable auxiliary transform set can be determined as the first index value 0. When the modified intra-prediction mode is greater than or equal to 2 and less than or equal to 12, the index of the inseparable auxiliary transform set can be determined as the second index value 2. When the modified intra-prediction mode is greater than or equal to 13 and less than or equal to 23, the index of the inseparable auxiliary transform set can be determined as the third index value 18. When the modified intra-prediction mode is greater than or equal to 24 and less than or equal to 44, the index of the inseparable auxiliary transform set can be determined as the fourth index value 34. When the modified intra-prediction mode is greater than or equal to 45 and less than or equal to 55, the index of the inseparable auxiliary transform set can be determined as the third index value 18. When the modified intra-prediction mode is greater than or equal to 56, the index of the inseparable auxiliary transform set can be determined as the second index value 2. When the modified intra-prediction mode is less than 0, the index of the inseparable auxiliary transform set can be determined as the second index value 2.

[0336] Figure 31 This is an example of an implementation of the present disclosure, and illustrates a flowchart of a process by which an encoder performs a transformation by taking into account wide-angle intra-frame prediction (WAIP). Figure 31 An example of the operation of encoder 100 is shown. Figure 31 An example is shown of an auxiliary transformation step performed by encoder 100 after the prediction step and the main transformation step and before the quantization step.

[0337] Encoder 100 can determine the intra-frame prediction mode by taking WAIP into account (S3110). According to embodiments of this disclosure, such as... Figure 24 or Figure 25 As shown, WAIP can be applied to generate more accurate prediction samples for non-square prediction cells. If WAIP is applied, as shown in Tables 2 through 5, a modified intra-prediction mode can be used instead of a specific intra-prediction mode.

[0338] Furthermore, encoder 100 encodes the index corresponding to the applied intra-prediction mode (S1350). In this case, encoder 100 can encode the original intra-prediction mode before the index changes. For example, in the 67-mode configuration, although intra-prediction mode 2 is transformed into intra-prediction mode 67, 2 (i.e., the index of the previous intra-prediction mode) is encoded.

[0339] Encoder 100 uses a pre-configured map to determine the secondary transform set (S3120). In this case, the pre-configured map can be... Figures 27a to 30b The index mapping table shown.

[0340] Subsequently, encoder 100 can select the optimal auxiliary transform by rate-distortion cost comparison after applying the main transform (S3130), and can encode the index of the selected auxiliary transform (S3140). According to embodiments of this disclosure, the auxiliary transform can be a non-separable transform applied to the upper left region (low-frequency domain) of the transform block after applying the main transform. Regarding the transform block to which the auxiliary transform has been applied, encoder 100 can output a bitstream by quantization and entropy coding. Encoder 100 can apply the main transform to a transform block from which prediction samples have been excluded from the coding unit before the auxiliary transform. In this case, the main transform can correspond to a separable transform in the row and column directions.

[0341] Figure 32 This is an example of an implementation of the present disclosure, and illustrates a flowchart of a decoder performing a transformation by taking WAIP into account. Figure 32 An example of the operation of decoder 200 is shown. Figure 32 An example of an inverse auxiliary transformation step performed by decoder 200 after the inverse quantization step and before the inverse main transformation step is shown.

[0342] Decoder 200 parses the index of the intra-prediction mode for the transform block that has already undergone entropy decoding and inverse quantization from the image signal (S3210). Furthermore, decoder 200 parses the index of the auxiliary transform from the bitstream associated with the image signal. In this case, encoder 100 can transmit the index of the intra-prediction mode, the index of the auxiliary transform, and the image signal.

[0343] Decoder 200 determines the intra-frame prediction mode by considering WAIP (S3220). According to embodiments of this disclosure, such as... Figure 24 or Figure 25 As shown, WAIP can be applied to generate more accurate prediction samples for non-square prediction units. If WAIP is applied, as shown in Tables 2 to 5, a modified intra-prediction mode can be used instead of a specific intra-prediction mode. Furthermore, the index of the intra-prediction mode received from encoder 100 can be the original index of the intra-prediction mode before the change, rather than the index changed by applying WAIP. In this case, after performing a transformation on the original index with reference to Tables 2 to 5, decoder 200 can determine the index of the modified intra-prediction mode.

[0344] Decoder 200 selects an auxiliary transform set from a predetermined mapping (S3230). In this case, the pre-configured mapping may be... Figures 27a to 30b The index mapping table shown.

[0345] Decoder 200 selects an auxiliary transform from the chosen set of auxiliary transforms (S3240) and applies the inverse auxiliary transform to the transform block (S3250). In this case, the auxiliary transform can be a non-separable transform applied to the upper left region (low-frequency domain) of the transform block after the main transform is applied. Subsequently, decoder 200 can generate residual samples by applying the inverse main transform to the transform block to which the inverse auxiliary transform has been applied. In this case, the inverse main transform can correspond to a separable transform in both the row and column directions.

[0346] Figure 33 This is an implementation of the present disclosure, and another example of a flowchart showing the decoder performing a transformation by taking WAIP into account.

[0347] A method for decoding an image signal according to an embodiment of the present disclosure may include: step S3310, determining a modified intra-prediction mode for the current block with different widths and heights based on the ratio between the width and height of the current block and the intra-prediction mode; step S3320, determining an inverse inseparable transform set based on the modified intra-prediction mode; and step S3330, applying an inverse inseparable transform matrix selected from the inverse inseparable transform set to the upper left region of the current block determined based on the width and height of the current block.

[0348] More specifically, firstly, the decoder 200 can generate a block (the current block) consisting of coefficients in the frequency domain by performing entropy decoding and inverse quantization on the bitstream of the image signal generated by the encoder 100.

[0349] In step S3310, if the current block is non-square (the width and height of the current block are different), the decoder 200 can determine the modified intra-prediction mode by considering the conditions for the width and height of the current block and the conditions for the range of the encoded intra-prediction mode. The transformation between the index of the encoded intra-prediction mode and the index of the modified intra-prediction mode can be configured as shown in Tables 2 to 4.

[0350] For example, in Table 3, specifically in (a) and (b), when the width of the transform block is greater than its height, and the intra-prediction mode is greater than or equal to and less than the first reference value, the value obtained by adding 65 to the intra-prediction mode is determined as the modified intra-prediction mode. In this case, when the ratio of the width of the current block to its height is less than or equal to 2, the first reference value is set to 8 (case (a)). When the ratio of the width of the transform block to its height is greater than 2, the first reference value can be set to 12 (case (b)). Furthermore, in (c) and (d) of Table 3, when the width of the transform block is less than its height, and the intra-prediction mode is greater than or equal to the second reference value and less than or equal to 66, the value obtained by subtracting 67 from the intra-prediction mode is determined as the modified intra-prediction mode. In this case, when the ratio of the height of the transform block to its width is less than or equal to 2, the second reference value can be set to 61 (case (c)). When the ratio of the width to the height of the transform block is greater than 2, the second reference value can be set to 57 (case (d)).

[0351] In step S3320, decoder 200 determines the inverse inseparable transform set based on the modified intra-prediction mode. According to embodiments of this disclosure, decoder 200 can determine the index of the low-frequency inseparable transform matrix set corresponding to the modified intra-prediction mode. The mapping between the index of the modified intra-prediction mode and the inverse inseparable transform can be achieved by considering WAIP... Figures 27a to 30b As shown in the diagram.

[0352] For example, such as Figure 30aAs shown, when the modified intra-prediction mode is greater than or equal to 0 and less than or equal to 1 due to the application of the wide-angle intra-prediction mode, the index of the inseparable auxiliary transform set can be determined as the first index value 0. When the modified intra-prediction mode is greater than or equal to 2 and less than or equal to 12, the index of the inseparable auxiliary transform set can be determined as the second index value 2. When the modified intra-prediction mode is greater than or equal to 13 and less than or equal to 23, the index of the inseparable auxiliary transform set can be determined as the third index value 18. When the modified intra-prediction mode is greater than or equal to 24 and less than or equal to 44, the index of the inseparable auxiliary transform set can be determined as the fourth index value 34. When the modified intra-prediction mode is greater than or equal to 45 and less than or equal to 55, the index of the inseparable auxiliary transform set can be determined as the third index value 18. When the modified intra-prediction mode is greater than or equal to 56, the index of the inseparable auxiliary transform set can be determined as the second index value 2. When the modified intra-prediction mode is less than 0, the index of the inseparable auxiliary transform set can be determined as the second index value 2.

[0353] In step S3330, decoder 200 may apply an inverse inseparable transform matrix selected from the inverse inseparable transform set determined in step S3320 to the current block. For example, decoder 200 may apply an inverse inseparable transform matrix specified by the index of the inverse inseparable transform set determined in step S3320 to the current block. Thereafter, decoder 200 may generate a residual signal or residual block by applying an additional transform (or inverse master transform).

[0354] Figure 34 This is an example of an embodiment of the present disclosure, and a block diagram of an image processing apparatus is shown. The image processing apparatus may correspond to a decoding apparatus or an encoding apparatus.

[0355] The image processing apparatus 3400 for processing image signals includes a memory 3420 for storing image signals and a processor 3410 connected to the memory for processing image signals.

[0356] The processor 3410 according to embodiments of the present disclosure may consist of at least one processing circuit for processing image signals, and may process image signals by executing instructions for encoding or decoding image signals. That is, the processor 3410 may encode raw image data or decode encoded image signals by executing the above-described encoding or decoding methods.

[0357] Figure 35 An example of an image encoding system in which embodiments of the present disclosure are applied is shown.

[0358] An image encoding system may include a source device and a receiving device. The source device may transmit encoded video / image information or data to the receiving device in the form of a file or stream via digital storage media or over a network.

[0359] The source device may include an image source, an encoding device, and a transmitter. The receiving device may include a receiver, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display, and the display may be implemented as a separate device or an external component.

[0360] Image sources can acquire video / images through video / image capture, compositing, or generation processes. Image sources can include video / image capture devices and / or video / image generation devices. Video / image capture devices can include, for example, one or more cameras, video / image archives containing previously captured video / images, etc. Video / image generation devices can include, for example, computers, tablets, and smartphones, and can generate video / images (electronically). For example, virtual video / images can be generated by computers, etc., and in this case, the video / image capture process can be replaced by a process for generating related data.

[0361] An encoding device can encode input video / images. The encoding device can perform a series of processes for compression and encoding efficiency, including prediction, transform, quantization, etc. The encoded data (encoded video / image information) can be output as a bitstream.

[0362] A transmitter can transmit encoded video / image information or data output in a bitstream to a receiver in a receiving device via a digital storage medium or network, either as a file or a stream. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter can include elements for generating media files according to a predetermined file format and may include elements for transmission over a broadcast / communication network. The receiver can extract the bitstream and send it to a decoding device.

[0363] The decoding device can perform a series of processes (including dequantization, inverse transform, prediction, etc.) corresponding to the operation of the encoding device to decode video / images.

[0364] The renderer can render decoded video / images. The rendered video / images can then be displayed on a monitor.

[0365] Figure 36 An example of an image streaming system that applies an embodiment of the present disclosure is shown.

[0366] Reference Figure 36 A content streaming system using this disclosure can generally include an encoding server, a streaming server, a web server, a media storage device, a user device, and a multimedia input device.

[0367] The basic function of an encoding server is to generate a bitstream by compressing content input from multimedia input devices (such as smartphones, cameras, or camcorders) into digital data, and then send the bitstream to a streaming server. In another example, if multimedia input devices such as smartphones, cameras, or camcorders generate the bitstream directly, the encoding server can be omitted.

[0368] Bitstreams can be generated by applying the bitstream generation or encoding methods disclosed herein. A streaming server can temporarily store bitstreams during transmission or reception.

[0369] A streaming server sends multimedia data to a user's device based on a user request via a web server. The web server acts as an intermediary, informing the user which services are available. When a user requests a service from the web server, the web server forwards the request to the streaming server. The streaming server then sends the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server controls the commands / responses between devices within the content streaming system.

[0370] A streaming server can receive content from media storage and / or encoding servers. For example, if content is received from an encoding server, the streaming server can receive the content in real time. In this case, to provide a smooth streaming service, the streaming server can store a bitstream for a given period of time.

[0371] Examples of user equipment may include mobile phones, smartphones, laptops, terminals for digital broadcasting, personal digital assistants (PDAs), portable multimedia players (PMPs), navigators, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., watch-type terminals (smartwatches), glass-type terminals (smart glass), and head-mounted displays (HMDs)), digital televisions, desktop computers, and digital television signage.

[0372] Servers within a content streaming system can operate as distributed servers. In this case, data received from the servers can be distributed and processed.

[0373] The embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in each figure can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0374] Furthermore, decoders and encoders utilizing this disclosure can be included in multimedia broadcasting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, image chat devices, real-time communication devices (such as image communication), mobile streaming devices, storage media, portable cameras, on-demand video (VoD) service providers, OTT (Internet TV) image devices, Internet streaming service providers, three-dimensional (3D) image devices, image telephone image devices, transportation terminals (e.g., vehicle terminals, aircraft terminals, ship terminals, etc.), and medical imaging devices, and can be used to process image signals or data signals. For example, OTT image devices can include game consoles, Blu-ray players, Internet access televisions, home theater systems, smartphones, tablet computers, digital video recorders (DVRs), etc.

[0375] Furthermore, the processing method applying this disclosure can be generated in the form of a computer-executable program, and the processing method can be stored in a computer-readable recording medium. Multimedia data having the data structure according to this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distribution storage devices for storing computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Furthermore, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmission over the Internet). Additionally, bitstreams generated by encoding methods can be stored in a computer-readable recording medium or transmitted via wired / wireless communication networks.

[0376] Furthermore, the embodiments of this disclosure can be implemented as a computer program product using program code, which can be executed on a computer by the embodiments of this disclosure. The program code can be stored on a computer-readable medium.

[0377] As described above, the embodiments described in this disclosure can be implemented and executed on a processor, microprocessor, controller, or chip. For example, the functional units shown in the drawings can be implemented and executed on a computer, processor, microprocessor, controller, or chip.

[0378] Furthermore, decoders and encoders employing this disclosure can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, cameras for surveillance, image-interactive devices, real-time communication devices such as image communication, mobile streaming devices, storage media, portable video cameras, video-on-demand (VoD) service providers, over-the-top (OTT) imaging devices, internet streaming service providers, three-dimensional (3D) imaging devices, image telephone devices, and medical imaging devices, and can be used to process image signals or data signals. For example, OTT imaging devices can include game consoles, Blu-ray players, internet access televisions, home theater systems, smartphones, tablet computers, and digital video recorders (DVRs).

[0379] Furthermore, the processing methods applied according to this disclosure can be generated in the form of a computer-executable program and can be stored in a computer-readable recording medium. Multimedia data having the data structure according to this disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices storing computer-readable data. Computer-readable recording media can include, for example, Blu-ray discs (BD), Universal Serial Bus (USB), ROM, PROM, EPROM, EEPROM, RAM, CD-ROM, magnetic tape, floppy disks, and optical data storage devices. Furthermore, computer-readable recording media include media implemented in the form of a carrier (e.g., transmission via the Internet). Additionally, bitstreams generated using encoding methods can be stored in a computer-readable recording medium or transmitted over wired and wireless communication networks.

[0380] Furthermore, the embodiments of this disclosure can be implemented as a computer program product using program code. The program code according to the embodiments of this disclosure can be executed by a computer. The program code can be stored on a computer-readable medium.

[0381] The above-described embodiments are implemented by combinations of components and features of this disclosure in a predetermined form. Unless otherwise stated, each component or feature should be considered selectively. Each component or feature may be implemented without combination with another component or feature. Furthermore, some components and / or features may be combined with each other and may implement embodiments of this disclosure. The order of operations described in the embodiments of this disclosure may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced by corresponding components or features of another embodiment. It is apparent that some claims referencing a particular claim may be combined with other claims referencing claims other than the particular claim to constitute an embodiment, or new claims may be added by amendment after filing the application.

[0382] The embodiments of this disclosure can be implemented by various means, such as hardware, firmware, software, or a combination thereof. When an embodiment is implemented in hardware, an embodiment of this disclosure can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, etc.

[0383] When the implementation is carried out by firmware or software, one embodiment of this disclosure can be implemented by modules, processes, functions, etc., that perform the above-described functions or operations. Software code can be stored in memory and can be driven by a processor. The memory is located inside or outside the processor and can exchange data with the processor in various well-known ways.

[0384] It will be apparent to those skilled in the art that this disclosure may be embodied in other specific forms without departing from its essential features. Therefore, the foregoing detailed description should not be construed as restrictive in all respects, but rather as illustrative. The scope of this disclosure should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this disclosure are included within its scope.

[0385] Industrial applicability

[0386] The foregoing preferred embodiments of this disclosure have been disclosed for illustrative purposes, and those skilled in the art can improve, modify, substitute or add various other embodiments without departing from the technical spirit and scope of this disclosure as disclosed in the appended claims.

Claims

1. A method for decoding an image signal, the method comprising the following steps: Based on the ratio between the width and height of the current block and the intra prediction mode, a modified intra prediction mode with different widths and heights for the current block is determined according to the intra prediction mode. The prediction sample of the current block is obtained based on the modified intra-frame prediction mode; The inverse inseparable transform set is determined based on the modified intra-frame prediction mode; as well as The inversely inseparable transformation matrix selected from the set of inversely inseparable transformations is applied to the upper left region of the current block, determined based on the width and height of the current block. Obtain residual samples for the current block; and The image signal is reconstructed based on the residual samples. Determining the inversely inseparable transform set includes determining the index of the inversely inseparable transform set associated with the modified intra-prediction mode. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 0 and less than or equal to 1, the index is determined as the first index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 2 and less than or equal to 12, the index is determined as the second index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 13 and less than or equal to 23, the index is determined as the third index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 24 and less than or equal to 44, the index is determined as the fourth index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 45 and less than or equal to 55, the index is determined as the third index value. Wherein, based on the modified intra-frame prediction mode being greater than or equal to 56, the index is determined as the second index value, and Wherein, based on the modified intra-frame prediction mode being less than 0, the index is determined as the second index value.

2. The method according to claim 1, in, Determining the modified intra-prediction mode includes: based on the fact that the width is greater than the height and the intra-prediction mode is greater than or equal to 2 and less than a first reference value, determining the value obtained by adding 65 to the intra-prediction mode as the modified intra-prediction mode.

3. The method according to claim 2, in, Based on the fact that the ratio of the width to the height is less than or equal to 2, the first reference value is set to 8, and The first reference value is set to 12 based on the fact that the ratio of the width to the height is greater than 2.

4. The method according to claim 1, in, Determining the modified intra-prediction mode includes: based on the fact that the width is less than the height and the intra-prediction mode is greater than a second reference value and less than or equal to 66, determining the value obtained by subtracting 67 from the intra-prediction mode as the modified intra-prediction mode.

5. The method according to claim 4, in, Based on the ratio being less than or equal to 2, the second reference value is set to 61, and Wherein, based on the ratio being greater than 2, the second reference value is set to 57.

6. A method for encoding an image signal, the method comprising the following steps: Based on the ratio between the width and height of the current block and the intra prediction mode, a modified intra prediction mode with different widths and heights for the current block is determined according to the intra prediction mode. The prediction sample for the current block is generated based on the modified intra-frame prediction mode; Generate residual samples for the current block based on the predicted samples; Based on the modified intra-frame prediction mode, determine the set of non-separable transforms for the residual samples; as well as The inseparable transformation matrix selected from the inseparable transformation set is applied to the upper left region of the current block, determined based on the width and height of the current block. Determining the inseparable transform set includes: determining the index of the inseparable transform set related to the modified intra-prediction mode. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 0 and less than or equal to 1, the index is determined as the first index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 2 and less than or equal to 12, the index is determined as the second index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 13 and less than or equal to 23, the index is determined as the third index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 24 and less than or equal to 44, the index is determined as the fourth index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 45 and less than or equal to 55, the index is determined as the third index value. Wherein, based on the modified intra-frame prediction mode being greater than or equal to 56, the index is determined as the second index value, and Wherein, based on the modified intra-frame prediction mode being less than 0, the index is determined as the second index value.

7. A method for transmitting data, said data comprising a bitstream for an image, wherein, The bitstream is generated by performing the following methods: determining a modified intra-prediction mode for the current block with different widths and heights based on the ratio between the width and height of the current block and the intra-prediction mode; generating prediction samples for the current block based on the modified intra-prediction mode; generating residual samples for the current block based on the prediction samples; determining an inseparable transform set for the residual samples based on the modified intra-prediction mode; applying an inseparable transform matrix selected from the inseparable transform set to the upper left region of the current block determined based on the width and height of the current block; and encoding image information related to the inseparable transform matrix. as well as Send the data, which includes the bitstream of the image information. Determining the inseparable transform set includes determining the index of the inseparable transform set associated with the modified intra-prediction mode. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 0 and less than or equal to 1, the index is determined as the first index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 2 and less than or equal to 12, the index is determined as the second index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 13 and less than or equal to 23, the index is determined as the third index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 24 and less than or equal to 44, the index is determined as the fourth index value. Specifically, based on the modified intra-frame prediction mode being greater than or equal to 45 and less than or equal to 55, the index is determined as the third index value. Wherein, based on the modified intra-frame prediction mode being greater than or equal to 56, the index is determined as the second index value, and Wherein, based on the modified intra-frame prediction mode being less than 0, the index is determined as the second index value.

Images