Image Symbolization Method and Apparatus

Abstract

To provide a method and apparatus for video coding based on an improved implicit transform approach.SOLUTION: In a method, transform block signaling information is acquired from a coded video bitstream. In addition, a determination is made to determine whether the transform block signaling information indicates an implicit transform scheme, and at least one of a low-frequency non-separable transform (LFNST) and a matrix-based intra predication mode (MIP) is invalid. In response to the determination that the transform block signaling information indicates the implicit transform scheme, and at least one of the LFNST and MIP is signaled as invalid, a primary transform type is determined based on a size of a coding block unit (CU), and a primary transform is performed for a transform block that is partitioned from the CU in accordance with the determined primary transform type.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 62 / 858,887, filed on June 7, 2019, entitled “IMPLICIT TRANSFORM SELECTION” and to U.S. Patent Application No. 16 / 883,545, filed on May 26, 2020, entitled “METHOD AND APPARATUS FOR IMPROVED IMPLICIT TRANSFORM SELECTION”. The entire disclosures of these prior applications are hereby incorporated by reference in their entirety.

[0002] The present disclosure presents a set of advanced video coding techniques. More specifically, an improved implicit transform approach is proposed.

Background Art

[0003] The background description provided herein is for the purpose of generally presenting the context of the disclosure. The work of the inventors named herein, to the extent described in this background section, as well as aspects of the description that may not otherwise qualify as prior art at the time of filing, are not, either expressly or impliedly, admitted as prior art to the present disclosure.

[0004] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video includes a series of pictures, each picture having spatial dimensions of, for example, 1920×1080 luminance (luminance) samples and associated color (chrominance) samples. The series of pictures can have a fixed or variable picture rate (informally also known as the frame rate), for example, 60 pictures per second, i.e., a picture rate of 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920×1080 luminance sample resolution at a frame rate of 60 Hz) requires a bandwidth close to 1.5 Gbit / s. One hour of such video requires storage space exceeding 600 gigabytes (GB).

[0005] One purpose of video encoding and decoding can be the reduction of redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth requirements or storage space requirements, sometimes by two or more orders of magnitude. Both reversible compression and irreversible compression, as well as combinations thereof, can be used. Reversible compression refers to techniques that can reconstruct an exact replica of the original signal from the compressed original signal. When using irreversible compression, the reconstructed signal may not be the same as the original signal, but the distortion between the original signal and the reconstructed signal is small enough to be useful for the intended use of the reconstructed signal. In the case of video, irreversible compression is widely used. The amount of acceptable distortion depends on the application. For example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio reflects this, and higher acceptable / tolerable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transformation, quantization, and entropy encoding.

[0007] Video codec technology can include techniques known as intra coding. In intra coding, sample values are represented without referring to samples or other data from previously reconstructed reference pictures. In some video codecs, a picture is spatially subdivided into multiple block samples. When the samples of all blocks are coded in intra mode, that picture can be an intra picture. Intra pictures, and their derivatives such as, for example, independent decoder refresh pictures, can be used to reset the decoder state and thus can be used as the first picture in an encoded video bitstream and video session or as a still image. The samples of an intra block can be subjected to a transform, and the transform coefficients can be quantized before entropy coding. Intra prediction can be considered a technique that minimizes the sample values in the pre-transform domain. In some cases, the smaller the post-transform DC value and the smaller the AC coefficients, the fewer bits are required at a given quantization step size to represent the block after entropy coding.

[0008] Traditional intra coding, such as that known from MPEG-2 generation coding techniques, does not use intra prediction. However, some newer video compression techniques include techniques that attempt, for example, from the surrounding sample data and / or metadata obtained during the encoding / decoding of spatially adjacent and previously decoded blocks. Such techniques will hereinafter be referred to as "intra prediction" techniques. Note that in at least some cases, intra prediction uses only reference data from the currently reconstructed picture and does not use reference data from reference pictures.

[0009] Multiple different forms of intra prediction may exist. If two or more of such techniques can be used in a given video coding technique, the technique in use can be coded in an intra prediction mode. In certain cases, the mode can have sub - modes and / or parameters, which can be coded individually or can be included in the mode codeword. Which codeword to use for a given mode / sub - mode / parameter combination can affect the coding efficiency gain through intra prediction, and so can the entropy coding technique used to convert the codeword into the bitstream.

[0010] A particular intra prediction mode was introduced in H.264, improved in H.265, and further improved in more recent coding techniques such as, for example, the Joint Exploration Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). A predictor block can be formed using adjacent sample values belonging to already available samples. The sample values of the adjacent samples are replicated into the predictor block according to a direction. The reference to the direction in use can be coded into the bitstream or can itself be predicted. SUMMARY OF THE INVENTION

[0011] Aspects of the present disclosure provide methods and apparatuses for video coding / decoding. In some examples, an apparatus for video decoding includes a receiving circuit and a processing circuit.

[0012] According to one aspect of the present disclosure, a method for video decoding of a decoder is provided. In this method, conversion block signaling information is obtained from an encoded video bitstream. Further, a determination is made as to whether the conversion block signaling information indicates an implicit conversion scheme and whether at least one of a low-frequency non-separable transform (LFNST) and a matrix-based intra predication mode (MIP) is invalid. In response to the determination that the conversion block signaling information indicates an implicit conversion scheme and at least one of LFNST and MIP is invalid, a primary conversion type is determined based on the size of a coding block unit (CU), and a primary conversion is performed on a conversion block divided from the CU according to the determined primary conversion type.

[0013] In some embodiments, to determine the primary conversion type, a determination may be made as to whether a conversion skip mode is enabled. In response to the determination that the conversion skip mode is enabled, depending on the width of the CU being greater than or equal to T1 and less than or equal to T2, a conversion type DST-7 can be determined for the horizontal conversion of the conversion block. Depending on the width of the CU being less than T1 or greater than T2, a conversion type DCT-2 can be determined for the horizontal conversion of the conversion block. Depending on the height of the CU being greater than or equal to T1 and less than or equal to T2, a conversion type DST-7 can be determined for the vertical conversion of the conversion block. Depending on the height of the CU being less than T1 or greater than T2, a conversion type DCT-2 can be determined for the vertical conversion of the conversion block.

[0014] In some embodiments, T1 can be equal to one of 2 pixels, 4 pixels, or 8 pixels, and T2 can be equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels.

[0015] In response to a determination that the transform block signaling information indicates an implicit transform scheme and that at least one of LFNST or MIP is signaled as true, in a first example, a first transform type DCT-2 can be determined for the transform block. In a second example, a second transform type other than DCT-7 can be determined for the transform block, the second transform type including at least one of DST-1, DCT-5, compound orthonormal transform (COT), or Karhunen–Loeve transform.

[0016] In response to a determination that the transform block signaling information indicates an implicit transform scheme and that MIP is signaled as false indicating that MIP is not applicable to the transform block, depending on the width of the CU being greater than or equal to T1 and less than or equal to T2, a transform type DST-7 can be determined for the horizontal transform of the transform block. Depending on the width of the CU being less than T1 or greater than T2, a transform type DCT-2 can be determined for the horizontal transform of the transform block. Depending on the height of the CU being greater than or equal to T1 and less than or equal to T2, a transform type DST-7 can be determined for the vertical transform of the transform block. Depending on the height of the CU being less than T1 or greater than T2, a transform type DCT-2 can be determined for the vertical transform of the transform block.

[0017] In a first example, T1 can be equal to 2 pixels and T2 can be equal to one of 4 pixels or 8 pixels. In a second example, T1 can be equal to 4 pixels and T2 can be equal to one of 4 pixels or 8 pixels. In a third example, T1 can be equal to 8 pixels and T2 can be equal to one of 8 pixels, 16 pixels, or 32 pixels. In a fourth example, T1 can be equal to 16 pixels and T2 can be equal to one of 16 pixels or 32 pixels.

[0018] In some embodiments, in response to a determination that the transform block signaling information indicates an implicit transform scheme and that both LFNST and MIP are signaled as false such that neither LFNST nor MIP is applied to the transform block, in accordance with the width of the CU being greater than or equal to T1 and less than or equal to T2, transform type DST-7 can be determined for the horizontal transform of the transform block. In accordance with the width of the CU being less than T1 or greater than T2, transform type DCT-2 can be determined for the horizontal transform of the transform block. In accordance with the height of the CU being greater than or equal to T1 and less than or equal to T2, transform type DST-7 can be determined for the vertical transform of the transform block. In accordance with the height of the CU being less than T1 or greater than T2, transform type DCT-2 can be determined for the vertical transform of the transform block.

[0019] According to another aspect of the present disclosure, a method for video decoding of a decoder is provided. In the method, transform block signaling information is obtained from an encoded video bitstream. A primary transform type is determined based on the transform block signaling information and the size of the coded block unit (CU). In accordance with the determined primary transform type, a primary transform is performed on the transform block divided from the CU.

[0020] In some embodiments, in order to determine the primary transform type, in accordance with the width of the CU being greater than or equal to T1 and less than or equal to T2, transform type DST-7 can be determined for the horizontal transform of the transform block. In accordance with the width of the CU being greater than T2 and less than or equal to T3, for the horizontal transform of the transform block, the transform type can be determined based on the signaled index from the transform block signaling information. The signaled index indicates that the transform type is one of DCT-2 or DST-7. In accordance with the width of the CU being less than T1 or greater than T3, transform type DCT-2 can be determined for the horizontal transform of the transform block.

[0021] In some embodiments, to determine the primary conversion type, depending on the height of the CU being greater than or equal to T1 and less than or equal to T2, the conversion type DST-7 can be determined for the vertical conversion of the conversion block. Depending on the height of the CU being greater than T2 and less than or equal to T3, for the vertical conversion of the conversion block, the conversion type can be determined based on the signaling index from the conversion block signaling information. The signaled index indicates that the conversion type is one of DCT-2 or DST-7. Depending on the height of the CU being less than T1 or greater than T3, the conversion type DCT-2 can be determined for the vertical conversion of the conversion block.

[0022] In some embodiments, T1 can be equal to one of 2 pixels, 4 pixels, or 8 pixels. T2 can be equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels. T3 can be equal to one of 8 pixels, 16 pixels, 32 pixels, or 64 pixels.

[0023] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform a method for video decoding.

Brief Description of the Drawings

[0024] Further features, properties, and various advantages of the matters related to the disclosure will become even more apparent from the following detailed description and the accompanying drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 7C

Figure 7D

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 13A

Figure 13B

Figure 14

Figure 15

Figure 16

Embodiments for Carrying Out the Invention

[0025] FIG. 1 illustrates a simplified block diagram of a communication system (100) according to an embodiment of the present disclosure. The communication system (100) includes a plurality of terminal devices that can communicate with each other, for example, via a network (150). For example, the communication system (100) includes a first pair of terminal devices (110) and (120) interconnected via a network (150). In the example of FIG. 1, the first pair of terminal devices (110) and (120) perform unidirectional data transmission. For example, the terminal device (110) may encode video data (e.g., a stream of video pictures captured by the terminal device (110)) for transmission to another terminal device (120) via the network (150). The encoded video data can be transmitted in the form of one or more encoded video bitstreams. The terminal device (120) may receive the encoded video data from the network (150), decode the encoded video data to restore the video pictures, and display the video pictures according to the restored video data. Unidirectional data transmission may be common in media service providing applications and the like.

[0026] In another example, the communication system (100) includes, for example, a second pair of terminal devices (130) and (140) that perform bidirectional transmission of encoded video data that may occur during a video conference. In the bidirectional transmission of data, in one example, each of the terminal devices (130) and (140) may encode video data (e.g., a stream of video pictures captured by that terminal device) for transmission to the other of the terminal devices (130) and (140) via the network (150). Each of the terminal devices (130) and (140) may also receive the encoded video data transmitted by the other of the terminal devices (130) and (140), decode the encoded video data to restore the video pictures, and display the video pictures on an accessible display device according to the restored video data.

[0027] In FIG. 1, the terminal devices (110), (120), (130) and (140) are shown as a server, a personal computer, and a smartphone, but the principles of the present disclosure may not be so limited. Embodiments of the present disclosure find use in laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. The network (150) represents any number of networks that transmit encoded video data between the terminal devices (110), (120), (130) and (140), including, for example, a wired (wired) communication network and / or a wireless communication network. The communication network 150 may exchange data on a circuit-switched channel and / or a packet-switched channel. Representative networks include wide area communication networks, local area networks, wide area networks, and / or the Internet. For the purposes of this description, the architecture and topology of the network (150) may not be important for the operation of the present disclosure, unless otherwise described below.

[0028] FIG. 2 illustrates an example of the placement of a video encoder and a video decoder in a streaming environment as an example of an application related to the disclosed matter. The disclosed matter can be equally applied to other applications where video can be used, including, for example, video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, and the like.

[0029] The streaming system may include a capture subsystem (213) that may include a video source (201), such as a digital camera, that creates, for example, a stream of uncompressed video pictures (202). In one example, the stream of video pictures (202) includes samples taken by a digital camera. The stream of video pictures (202) is drawn as a thick line to emphasize that it has a high data volume compared to the encoded video data (204) (or encoded video bitstream), and may be processed by an electronics device (220) that includes a video encoder (203) coupled to the video source (201). The video encoder (203) can include hardware, software, or a combination thereof to enable or implement aspects of the disclosure described in more detail hereinafter. The encoded video data (204) (or encoded video bitstream (204)) is drawn as a thin line to emphasize that it has a low data volume compared to the stream of video pictures (202), and can be stored in a streaming server (205) for later use. One or more streaming client subsystems, such as the client subsystems (206) and (208) of FIG. 2, can access the streaming server (205) to retrieve copies (207) and (209) of the encoded video data (204). The client subsystem (206) can include, for example, a video decoder (210) within an electronics device (230). The video decoder (210) can decode an incoming copy (207) of the encoded video data and create an outgoing stream of video pictures (211), and the outgoing stream of video pictures (211) can be rendered on a display (212) (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data (204), (207), and (209) (e.g., video bitstreams) can be encoded according to a particular video encoding / compression standard. Examples of those standards include ITU-T Recommendation H.265.In one example, a certain video coding standard under development is informally known as Versatile Video Coding (VVC). The matters disclosed herein may be used in the context of VVC.

[0030] Note that the electronic devices (220) and (230) can include other components (not shown). For example, the electronic device (220) can include a video decoder (not shown), and the electronic device (230) can also include a video encoder (not shown).

[0031] FIG. 3 shows a block diagram of a video decoder (310) according to an embodiment of the present disclosure. The video decoder (310) can be included in an electronic device (330). The electronic device (330) can include a receiver (331) (e.g., a receiving circuit). The video decoder (310) can be used in place of the video decoder (210) in the example of FIG. 2.

[0032] In the same or another embodiment, the receiver (331) can receive one or more encoded video sequences that will be decoded by the video decoder (310) with one encoded video sequence at a time, and the decoding of each encoded video sequence is independent of other encoded video sequences. The encoded video sequence can be received from a channel (301) that can be a hardware / software link to a storage device storing the encoded video data. The receiver (331) may receive the encoded video data together with other data such as, for example, encoded audio data and / or auxiliary data streams, and those data can be transferred to their respective using entities (not shown). The receiver (331) can separate the encoded video sequence from other data. To counter network jitter, a buffer memory (315) can be coupled between the receiver (331) and the entropy decoder / parser 320 (hereinafter, “parser (320)”). In a particular application, the buffer memory (315) is part of the video decoder (310). In others, it may be external to the video decoder (310) (not shown). In still others, for example, to counter network jitter, a buffer memory (not shown) can exist external to the video decoder (310), and further, for example, to handle playback timing, another buffer memory (315) can exist inside the video decoder (310). When the receiver (331) is receiving data from a storage / transfer device with sufficient bandwidth and controllability or from a synchronous network, the buffer memory (315) may not be needed or can be made small. For use on a best-effort packet network such as the Internet, for example, the buffer memory (315) can be needed, made relatively large, and advantageously made of an adaptable size, and can also be implemented, at least in part, by an operating system or similar element (not shown) external to the video decoder (310).

[0033] The video decoder (310) may include a parser (320) for reconstructing symbols (321) from the encoded video sequence. The categories of those symbols include information used to manage the operation of the video decoder (310) and may also include information for controlling a rendering device, such as a renderer device (312) (e.g., a display screen), which is not an integrated part of the electronics device (330) but can be coupled to the electronics device (330), as shown in FIG. 3. The control information for the (one or more) rendering devices may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may perform syntax analysis / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence can be according to a video encoding technique or standard and can follow various principles, including variable length encoding, Huffman encoding, arithmetic encoding with or without context dependence. The parser (320) can extract a set of subgroup parameters regarding at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to a group. The subgroups can include a Group of Pictures (GOP), a picture, a tile, a slice, a macroblock, a Coding Unit (CU), a block, a Transform Unit (TU), a Prediction Unit (PU), etc. The parser (320) can also extract information such as transform coefficients, quantization parameter values, motion vectors, etc. from the encoded video sequence information.

[0034] The parser (320) may perform entropy decoding / syntax analysis processing on the video sequence received from the buffer memory (315) to produce the symbols (321).

[0035] For the reconstruction of symbol (321), multiple different units may be involved according to the type of the encoded video picture or its part and other factors (e.g., inter-picture and intra-picture, inter-block and intra-block, etc.). Which units are involved and how they are involved can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (320). Such a flow of subgroup control information between the parser (320) and the following multiple units is not shown for clarity.

[0036] Beyond the aforementioned functional blocks, the video decoder (310) can conceptually be subdivided into a number of functional units as described later. In a practical implementation operating under commercial constraints, many of these units can interact closely with each other and can be at least partially integrated with each other. However, for the purpose of explaining the matters related to the disclosure, a conceptual subdivision into the following functional units is appropriate.

[0037] The first unit is the scaler / inverse transform unit (351). The scaler / inverse transform unit (351) receives the quantized transform coefficients together with control information including which transform to use, block size, quantization coefficient, quantization scaling matrix, etc. as the symbol(s) (321) from the parser (320). The scaler / inverse transform unit (351) can output a block with sample values that can be input to the aggregator (355).

[0038] In some cases, the output samples of the scaler / inverse transform (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from previously reconstructed pictures but can use prediction information from previously reconstructed parts of the current picture. Such prediction information can be provided by the intra-picture prediction unit (352). Optionally, the intra-picture prediction unit (352) generates a block of the same size and shape as the block being reconstructed, using surrounding already reconstructed information fetched from the current picture buffer (358). The current picture buffer (358) buffers, for example, a partially reconstructed current picture and / or a fully reconstructed current picture. The aggregator (355) optionally adds, sample by sample, the prediction information generated by the intra prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0039] In other cases, the output samples of the scaler / inverse transform unit (351) may relate to inter-coded, potentially motion-compensated blocks. In such cases, the motion-compensation prediction unit (353) can access the reference picture memory (357) to fetch the samples used for prediction. After motion-compensating the fetched samples according to the symbols (321) related to the block, these samples can be added by the aggregator (355) to the output of the scaler / inverse transform unit (351) (in this case, called the residual samples or residual signal) to generate the output sample information. The address in the reference picture memory (357) from which the motion-compensation prediction unit (353) fetches the prediction samples can be controlled by the motion vector and is available to the motion-compensation unit (353) in the form of, for example, a symbol (321) having X, Y, and reference picture components. Motion compensation can also include interpolation of the sample values fetched from the reference picture memory (357) when an accurate sub-sample motion vector is used, a motion vector prediction mechanism, etc.

[0040] The output samples of the aggregator (355) can be subjected to various loop filtering techniques in the loop filter unit (356). The video compression technology can include in-loop filter technology, which is controlled by parameters made available to the loop filter unit (356) as symbols (321) from the parser (320) included in the encoded video sequence (also referred to as the encoded video bitstream), but can also respond to meta information obtained during the decoding of the preceding part (in decoding order) of the encoded picture or encoded video sequence, and can also respond to previously reconstructed and loop-filtered sample values.

[0041] The output of the loop filter unit (356) can be made into a sample stream that can be output to the renderer device (312), and this can also be stored in the reference picture memory (357) for use in future inter-picture prediction.

[0042] When a particular encoded picture is completely reconstructed, it can be used as a reference picture for future prediction. For example, when the encoded picture corresponding to the current picture is completely reconstructed and that encoded picture is specified as a reference picture (e.g., by the parser (320)), the current picture buffer (358) can become part of the reference picture memory (357), and a new current picture buffer can be reallocated before starting the reconstruction of the next encoded picture.

[0043] The video decoder (310) may perform decoding processing according to a predetermined video compression technique in a standard such as, for example, ITU-T Recommendation H.265. The encoded video sequence may conform to the syntax defined by the video compression technique or standard used, in the sense of faithfully adhering to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, the profile can select specific tools from all the tools available in the video compression technique or standard such that only those tools available for use under that profile are selected. Also, for compliance, it is also necessary that the complexity of the encoded video sequence be within the range defined by the level of the video compression technique or standard. Optionally, the level may restrict the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The restrictions set by the level may optionally be further restricted through the Hypothetical Reference Decoder (HRD) specification and the metadata for HRD buffer management signaled in the encoded video sequence.

[0044] In one embodiment, the receiver (331) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of the (one or more) encoded video sequences. The additional data may be used by the video decoder (310) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0045] Figure 4 shows a block diagram of a video encoder (403) according to an embodiment of the present disclosure. The video encoder (403) is included in an electronics device (420). For example, the electronics device (420) includes a transmitter (440) (e.g., a transmission circuit). The video encoder (403) can be used in place of the video encoder (203) in the example of FIG. 2.

[0046] The video encoder (403) can receive video samples from a video source (401) (not part of the electronics device (420) in the example of FIG. 4) that can capture (one or more) video images to be encoded by the encoder (403). In another example, the video source (401) is part of the electronics device (420).

[0047] The video source (401) can provide a source video sequence to be encoded by the video encoder (403) in the form of a digital video sample stream having any suitable bit depth (e.g., 8 bits, 10 bits, 12 bits,...), any color space (e.g., BT.601 Y CrCB, RGB,...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service providing system, the video source (401) can be a storage device storing pre-prepared videos. In a video conferencing system, the video source (401) can be a camera that captures local image information as a video sequence. The video data can be provided as a plurality of individual pictures that convey motion when viewed in sequence. The pictures themselves can be organized as a spatial array of pixels, and each pixel can have one or more samples depending on the sampling structure, color space, etc. used. Those skilled in the art can immediately understand the relationship between pixels and samples. The following description focuses on samples.

[0048] According to one embodiment, a video encoder (403) can encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under other time constraints required by an application. Enforcing an appropriate encoding speed is one function of a controller (450). In some embodiments, the controller (450) controls and is functionally coupled to other functional units as described below. The coupling is not shown for clarity. Parameters set by the controller (450) can include rate control related parameters (picture skip, quantizer, lambda value of rate distortion optimization techniques, …), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (450) can be configured to have other suitable functions related to the video encoder (403) that are optimized for a particular system design.

[0049] In some embodiments, the video encoder (403) is configured to operate in an encoding loop. As an overly simplified explanation, in one example, the encoding loop can include a source coder (430) (which is responsible for creating symbols such as a symbol stream based on, for example, an input picture to be encoded and the (one or more) reference pictures) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs the symbols to generate sample data in the same manner as a (remote) decoder would also do when the compression between the symbols and the encoded video bitstream is reversible in the video compression techniques considered in the matters related to the disclosure. The reconstructed sample stream (sample data) is input into the reference picture memory (434). Since the decoding of the symbol stream results in a bit-exact result that is independent of the decoder location (local or remote), the content in the reference picture memory (434) is also bit-exact between the local encoder and the remote encoder. In other words, the prediction part of the encoder "sees" the same sample values as the reference picture samples that the decoder "sees" when the decoder uses prediction during decoding. This basic principle of reference picture synchronization (and the resulting drift in case synchronization cannot be maintained, for example, due to channel errors) is also used in some related technologies.

[0050] The operation of the "local" decoder (433) can be considered to be the same as that of a "remote" decoder such as, for example, the video decoder (310), which has already been described in detail above in relation to FIG. 3. However, referring briefly to FIG. 3 as well, since the symbols are available and the encoding / decoding of the symbols into the encoded video sequence by the entropy coder (445) and the parser (320) can be reversible, the entropy decoding part of the video decoder (310) including the buffer memory (315) and the parser (320) does not have to be fully implemented in the local decoder (433).

[0051] What can be noticed at this point is that any decoder technology, except for the syntax analysis / entropy decoding existing in the decoder, must necessarily exist in the corresponding encoder in a substantially the same functional form. Since the description of the encoder technology is the reverse of the thoroughly described decoder technology, it can be omitted. Only in specific fields is a more detailed description required and is provided below.

[0052] During operation, in some examples, the source coder (430) can perform motion-compensated predictive coding that predictively encodes an input picture with respect to one or more previously encoded pictures designated as "reference pictures" from a video sequence. Thus, the coding engine (432) encodes the difference between a pixel block of the input picture and a pixel block of one or more reference pictures that can be selected as the (one or more) prediction criteria for the input picture.

[0053] The local video decoder (433) can decode the coded video data of a picture that can be designated as a reference picture based on the symbols created by the source coder (430). The operation of the coding engine (432) can advantageously be an irreversible process. When the coded video data can be decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence can typically be a replica of the source video sequence with some error. The local video decoder (433) can replicate the decoding process that can be performed by the video decoder on the reference picture and cause the reconstructed reference picture to be stored in the reference picture cache (434). Thus, the video encoder (403) can locally store a copy of the reconstructed reference picture having the same content as the reconstructed reference picture that would be obtained by a far-end video decoder.

[0054] Predictor (435) can perform a prediction search for the encoding engine (432). That is, for a new picture to be encoded, the predictor (435) can search the reference picture memory (434) for sample data (as candidate reference pixel blocks) that can serve as an appropriate prediction criterion for the new picture or for specific metadata such as, for example, reference picture motion vectors and block shapes. The predictor (435) can operate on a per-pixel-block basis to find an appropriate prediction reference. Optionally, the input picture can have prediction criteria drawn from a plurality of reference pictures stored in the reference picture memory (434) as determined by the search results obtained by the predictor (435).

[0055] The controller (450) can manage the encoding process of the source coder (430), including, for example, setting parameters and subgroup parameters used to encode video data.

[0056] The outputs of all the aforementioned functional units can be subjected to entropy encoding in the entropy coder (445). The entropy coder (445) converts the symbols generated by the various functional units into an encoded video sequence by reversibly compressing the symbols according to techniques such as, for example, Huffman coding, variable-length coding, arithmetic coding, etc.

[0057] The transmitter (440) can buffer the (one or more) encoded video sequences generated by the entropy coder (445) and prepare them for transmission via the communication channel (460). The communication channel (460) can be a hardware / software link to a storage device that stores the encoded video data. The transmitter (440) can merge the encoded video data from the video coder (403) with other data to be transmitted, such as, for example, encoded audio data and / or an auxiliary data stream (the source is not shown).

[0058] The controller (450) may manage the operation of the video encoder (403). In encoding, the controller (450) may assign to each encoded picture a specific encoded picture type that may affect the encoding technique applicable to that picture. For example, a picture may often be assigned one of the following picture types.

[0059] An intra picture (I picture) may be encoded and decoded without using any other picture in the sequence as a source of prediction. Some video codecs allow multiple different types of intra pictures, for example including an Independent Decoder Refresh (IDR) picture. Those skilled in the art know those variants of I pictures, as well as their respective uses and characteristics.

[0060] A predicted picture (P picture) may be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index to predict the sample values of each block.

[0061] A bi-directionally predicted picture (B picture) may be encoded and decoded using intra prediction or inter prediction using at most two motion vectors and a reference index to predict the sample values of each block. Similarly, a multi-predicted picture can use three or more reference pictures and associated metadata for the reconstruction of a single block.

[0062] The source picture can generally be spatially subdivided into a plurality of sample blocks (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples each) and encoded block by block. The blocks can be encoded predictively with reference to other (already encoded) blocks determined by the encoding assignment applied to each of those blocks in their respective pictures. For example, blocks of an I picture can be encoded non-predictively, or they can be encoded predictively with reference to already encoded blocks of the same picture (spatial prediction or intra prediction). Pixel blocks of a P picture can be encoded non-predictively or via spatial or temporal prediction with reference to a reference picture encoded one ahead. Blocks of a B picture can be encoded non-predictively or via spatial or temporal prediction with reference to one or two reference pictures encoded ahead.

[0063] The video encoder (403) can perform an encoding process according to a predetermined video encoding technique or standard such as ITU-T Recommendation H.265. In its operation, the video encoder (403) can perform various compression processes including a predictive encoding process that exploits the temporal and spatial redundancies in the input video sequence. The encoded video data can thus conform to the syntax defined by the video encoding technique or standard being used.

[0064] In one embodiment, the transmitter (440) can transmit additional data together with the encoded video. The source coder (430) can include such data as part of the encoded video sequence. The additional data can have temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, and the like.

[0065] The video can be captured as a plurality of source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra prediction) utilizes the spatial correlation within a given picture, while inter-picture prediction utilizes the (temporal or other) correlation between pictures. In one example, a particular picture being encoded / decoded, referred to as the current picture, is divided into a plurality of blocks. When a block within the current picture is similar to a reference block within a reference picture that has been previously encoded and is still being buffered in the video, that block within the current picture can be encoded by a vector called a motion vector. The motion vector points to the reference block within the reference picture and can have a third dimension that specifies the reference picture when multiple reference pictures are being used.

[0066] In some embodiments, dual prediction techniques can be used in inter-picture prediction. According to the dual prediction technique, for example, two reference pictures are used, such as a first reference picture and a second reference picture that are both earlier than the current picture in the decoding order (however, in the display order, they can be in the past and future respectively). A block within the current picture can be encoded by a first motion vector pointing to a first reference block within the first reference picture and a second motion vector pointing to a second reference block within the second reference picture. The block can be predicted by a combination of the first reference block and the second reference block.

[0067] Furthermore, merge mode techniques can be used to improve the encoding efficiency in inter-picture prediction.

[0068] According to some embodiments of the present disclosure, predictions such as, for example, inter-picture prediction and intra-picture prediction are performed in units of blocks. For example, according to the HEVC standard, pictures in a sequence of video pictures are divided into a plurality of coding tree units (CTUs) for compression, and those CTUs in a picture have the same size, such as, for example, 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs) which are one luma CTB and two chroma CTBs. Each CTU can be recursively quad-tree divided into one or more coding units (CUs). For example, a 64×64 pixel CTU can be divided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In one example, each CU is analyzed to determine the prediction type of that CU, such as, for example, an inter prediction type or an intra prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, the prediction operation during coding (encoding / decoding) is performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values (e.g., luma values), such as, for example, 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, and the like.

[0069] FIG. 5 shows a diagram of a video encoder (503) according to another embodiment of the present disclosure. The video encoder (503) is configured to receive sample values of a processing block (e.g., a prediction block) in a current video picture in a sequence of video pictures and encode the processing block into an encoded picture that is part of an encoded video sequence. In one example, the video encoder (503) is used in place of the video encoder (203) in the example of FIG. 2.

[0070] In an example of HEVC, a video encoder (503) receives a matrix of sample values for a processing block, such as an 8×8 sample of a prediction block. The video encoder (503) determines whether the processing block is best encoded using an intra mode, an inter mode, or a bi-prediction mode, for example, using rate-distortion optimization. If the processing block is encoded in the intra mode, the video encoder (503) can encode the processing block into an encoded picture using intra prediction techniques, and if the processing block is encoded in the inter mode or the bi-prediction mode, the video encoder (503) can encode the processing block into an encoded picture using inter prediction techniques or bi-prediction techniques, respectively. In certain video encoding techniques, the merge mode can be an inter-picture prediction sub-mode in which a motion vector is derived from one or more motion vector predictors without the benefit of an outer encoded motion vector component of the predictor. In certain other video encoding techniques, there may be motion vector components applicable to the target block. In one example, the video encoder (503) includes other components, such as a mode decision module (not shown) for determining, for example, the mode of the processing block.

[0071] In the example of FIG. 5, the video encoder (503) includes an inter encoder (530), an intra encoder (522), a residual calculator (523), a switch (526), a residual encoder (524), a general controller (521), and an entropy encoder (525) coupled together as shown in FIG. 5.

[0072] The inter-encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block with one or more reference blocks (e.g., blocks in a previous picture and a subsequent picture) in a reference picture, generate inter-prediction information (e.g., a description of redundant information according to an inter-coding technique, a motion vector, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using some suitable technique. In some examples, the reference picture is a reference picture decoded based on the encoded video information.

[0073] The intra-encoder (522) is configured to receive samples of a current block (e.g., a processing block), in some cases, compare the block with an already encoded block in the same picture, generate quantized coefficients after transformation, and in some cases, also generate intra-prediction information (e.g., intra-prediction direction information according to one or more intra-coding techniques). In one example, the intra-encoder (522) also calculates an intra-prediction result (e.g., a predicted block) based on the intra-prediction information and a reference block in the same picture.

[0074] The overall controller (521) is configured to determine overall control data and control other components of the video encoder (503) based on the overall control data. In one example, the overall controller (521) determines the mode of a block and provides a control signal to the switch (526) based on that mode. For example, when the mode is the intra mode, the overall controller (521) controls the switch (526) to select the intra mode result for use by the residual calculator (523), and controls the entropy encoder (525) to select the intra prediction information and include the intra prediction information in the bitstream. When the mode is the inter mode, the overall controller (521) controls the switch (526) to select the inter prediction result for use by the residual calculator (523), and controls the entropy encoder (525) to select the inter prediction information and include the inter prediction information in the bitstream.

[0075] The residual calculator (523) is configured to calculate the difference (residual data) between the received block and the prediction result selected from the intra encoder (522) or the inter encoder (530). The residual encoder (524) operates based on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (524) is configured to convert the residual data from the spatial domain to the frequency domain and generate transform coefficients. Then, the transform coefficients are subjected to quantization processing to obtain quantized transform coefficients. In various embodiments, the video encoder (503) also includes a residual decoder (528). The residual decoder (528) is configured to perform inverse transformation and generate decoded residual data. The decoded residual data can be preferably used by the intra encoder (522) and the inter encoder (530). For example, the inter encoder (530) can generate a decoded block based on the decoded residual data and the inter prediction information, and the intra encoder (522) can generate a decoded block based on the decoded residual data and the intra prediction information. The decoded block is preferably processed to generate a decoded picture, and the decoded picture can be buffered in a memory circuit (not shown) and can be used as a reference picture in some examples.

[0076] The entropy encoder (525) is configured to format the bitstream to include the encoded block. The entropy encoder (525) is configured to include various information according to a suitable standard such as the HEVC standard. In one example, the entropy encoder (525) is configured to include general control data, selected prediction information (e.g., intra prediction information or inter prediction information), residual information, and other suitable information in the bitstream. Note that according to the matters related to the present disclosure, when encoding a block in either the inter mode or the merge submode of the bi-prediction mode, there is no residual information.

[0077] FIG. 6 shows a diagram of a video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive an encoded picture that is part of an encoded video sequence and decode the encoded picture to generate a reconstructed picture. In one example, the video decoder (610) is used in place of the video decoder (210) in the example of FIG. 2.

[0078] In the example of FIG. 6, the video decoder (610) includes an entropy decoder (671), an inter decoder (680), a residual decoder (673), a reconstruction module (674), and an intra decoder (672) that are coupled together as shown in FIG. 6.

[0079] The entropy decoder (671) may be configured to reconstruct from the encoded picture specific symbols that represent syntax elements that make up the encoded picture. Such symbols can include, for example, the mode in which a block is encoded (e.g., intra mode, inter mode, bi-prediction mode, the latter two in merge sub-modes or other sub-modes), prediction information (e.g., intra prediction information or inter prediction information, etc.) that can identify specific samples or metadata used for prediction by the intra decoder (672) or the inter decoder (680) respectively, residual information in the form of, for example, quantized transform coefficients, and the like. In one example, when the prediction mode is inter mode or bi-prediction mode, inter prediction information is provided to the inter decoder (680), and when the prediction type is intra prediction type, intra prediction information is provided to the intra decoder (672). The residual information can be inverse quantized and provided to the residual decoder (673).

[0080] The inter decoder (680) is configured to receive inter prediction information and generate an inter prediction result based on the inter prediction information.

[0081] The intra decoder (672) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0082] The residual decoder (673) is configured to perform inverse quantization to extract the inverse quantized transform coefficients, and process the inverse quantized transform coefficients to convert the residual from the frequency domain to the spatial domain. The residual decoder (673) may also require specific control information (including quantization parameter (QP)), and such information may be provided by the entropy decoder (671) (since this may be only low-volume control information, the data path is not shown).

[0083] The reconstruction module (674) is configured to combine the residual output by the residual decoder (673) and the prediction result (output by the inter or intra prediction module as appropriate) in the spatial domain to form a reconstruction block. The reconstruction block can be part of a reconstructed picture, or alternatively, the reconstructed picture can be part of a reconstructed video. Note that in order to improve visual quality, other suitable processes can be performed, such as deblocking processing and the like.

[0084] Note that the video encoders (203), (403), and (503), and the video decoders (210), (310), and (610) may be implemented using any suitable technology. In one embodiment, the video encoders (203), (403), and (503), and the video decoders (210), (310), and (610) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (203), (403), and (503), and the video decoders (210), (310), and (610) may be implemented using one or more processors that execute software instructions.

[0085] Aspects of the present disclosure provide a set of advanced video coding techniques. More specifically, an improved implicit transform approach is proposed.

[0086] In the video coding community, ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC1 / SC29 / WG11) issued the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Since then, ITU-T and ISO / IEC have been studying the potential need for standardization of future video coding technologies with compression capabilities significantly exceeding those of the HEVC standard (including its extensions). In October 2017, ITU-T and ISO / IEC jointly called for proposals for video compression with capabilities beyond HEVC (CfP). By February 15, 2018, a total of 22 CfP responses for standard dynamic range (SDR), 12 CfP responses for high dynamic range (HDR), and 12 CfP responses for the 360 video category were submitted respectively. In April 2018, all received CfP responses were evaluated at the 122nd MPEG / 10th JVET (Joint Video Exploration Team - Joint Video Expert Team) meeting. JVET officially started the standardization of next-generation video coding beyond HEVC (i.e., Versatile Video Coding (VVC)) and the latest version of the VVC Test Model (i.e., VTM3).

[0087] In HEVC, the primary transforms can be assumed to be DCT-2 of 4 points, 8 points, 16 points, and 32 points, and the transform core matrix can be represented using 8-bit integers, i.e., an 8-bit transform core. The transform core matrix of the smaller DCT-2 is a part of the larger DCT-2, as shown below: 4×4 transform

Number

Number

Number

Number

[0088] The DCT-2 core exhibits symmetry / antisymmetry characteristics. Therefore, the so-called "partial butterfly" implementation is supported to reduce the number of arithmetic counts (multiplication, addition / subtraction, shift), and equal matrix multiplication results can be obtained using the partial butterfly.

[0089] In VVC, two sub-block transforms are provided. The first sub-block transform is SVT or SBT. In JVET-J0024, JVET-K0139, and JVET-L0358, a spatially varying transform (SVT) scheme has been proposed. In SVT, for the inter-prediction residual, only the residual block may exist within the coded block. Since the residual block is smaller than the coded block, the transform size in SVT is smaller than the coded block size. For the regions not covered by the residual block or the transform, zero residuals can be assumed.

[0090] More specifically, in JVET-L0358, SVT is also called Sub-block Transform (SBT). The sub-block types (SVT-H, SVT-V), sizes, and positions (left half, left quarter, right half, right quarter, upper half, upper quarter, lower half, lower quarter) supported by SBT are shown in FIGS. 7A-7D. FIGS. 7A-7D respectively show the sub-block types (SVT-H, SVT-V) and positions (left half, right half, upper half, lower half) supported by SBT. The shaded areas labeled with the letter "A" are the residual blocks with transform, and the other areas are assumed to have zero residuals without transform.

[0091] The second sub-block transform is Intra Sub-Partitions (ISP). As shown in Table 1, the ISP coding mode divides the luma intra prediction block into two or four sub-partitions in the vertical or horizontal direction according to the block size dimension. FIGS. 8 and 9 show examples of two possibilities. FIG. 8 shows an exemplary division of a 4×8 block or an 8×4 block. FIG. 9 shows an exemplary division of a block that is neither a 4×8 block, an 8×4 block, nor a 4×4 block. All sub-partitions satisfy the condition of having at least 16 samples. For the chroma component, ISP is not applied.

Table 1

[0092] In some embodiments, for each of these sub-partitions, the coefficients transmitted by the encoder can be entropy decoded, and then the residual signal can be generated by inverse quantization and inverse transformation of the coefficients. Then, the sub-partition is intra predicted, and finally, the corresponding reconstructed samples are obtained by adding the residual signal to the prediction signal. Therefore, the reconstructed values of each sub-partition can be made available to generate the prediction of the next sub-partition, which allows the process to be repeated successively. All sub-partitions share the same intra mode.

[0093] In some embodiments, the ISP algorithm is only tested in the intra mode that is part of the MPM list. For this reason, when a block uses ISP, the MPM flag can be presumed to be 1. Additionally, when ISP is used for a particular block, the MPM list can be modified to exclude the DC mode and to prioritize the horizontal intra mode for ISP horizontal division and the vertical intra mode for ISP vertical division.

[0094] In the ISP, since the conversion and reconstruction are performed individually for each sub - partition, each sub - partition can be regarded as a sub - TU.

[0095] In the current VVC, in addition to the 4 - point, 8 - point, 16 - point, and 32 - point DCT - 2 transforms which are the same as in HEVC, additional 2 - point and 64 - point DCT - 2s are also added for the primary transform. The 64 - point DCT - 2 core defined in VVC can be shown as a 64×64 matrix as follows: { {aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa} {bf,bg,bh,bi,bj,bk,bl,bm,bn,bo,bp,bq,br,bs,bt,bu,bv,bw,bx,by,bz,ca,cb,cc,cd,ce,cf,cg,ch,ci,cj,ck,-ck,-cj,-ci,-ch,-cg,-cf,-ce,-cd,-cc,-cb,-ca,-bz,-by,-bx,-bw,-bv,-bu,-bt,-bs,-br,-bq,-bp,-bo,-bn,-bm,-bl,-bk,-bj,-bi,-bh,-bg,-bf} {ap,aq,ar,as,at,au,av,aw,ax,ay,az,ba,bb,bc,bd,be,-be,-bd,-bc,-bb,-ba,-az,-ay,-ax,-aw,-av,-au,-at,-as,-ar,-aq,-ap,-ap,-aq,-ar,-as,-at,-au,-av,-aw,-ax,-ay,-az,-ba,-bb,-bc,-bd,-be,be,bd,bc,bb,ba,az,ay,ax,aw,av,au,at,as,ar,aq,ap} {bg,bj,bm,bp,bs,bv,by,cb,ce,ch,ck,-ci,-cf,-cc,-bz,-bw,-bt,-bq,-bn,-bk,-bh,-bf,-bi,-bl,-bo,-br,-bu,-bx,-ca,-cd,-cg,-cj,cj,cg,cd,ca,bx,bu,br,bo,bl,bi,bf,bh,bk,bn,bq,bt,bw,bz,cc,cf,ci,-ck,-ch,-ce,-cb,-by,-bv,-bs,-bp,-bm,-bj,-bg} {ah,ai,aj,ak,al,am,an,ao,-ao,-an,-am,-al,-ak,-aj,-ai,-ah,-ah,-ai,-aj,-ak,-al,-am,-an,-ao,ao,an,am,al,ak,aj,ai,ah,ah,ai,aj,ak,al,am,an,ao,-ao,-an,-am,-al,-ak,-aj,-ai,-ah,-ah,-ai,-aj,-ak,-al,-am,-an,-ao,ao,an,am,al,ak,aj,ai,ah} {bh,bm,br,bw,cb,cg,-ck,-cf,-ca,-bv,-bq,-bl,-bg,-bi,-bn,-bs,-bx,-cc,-ch,cj,ce,bz,bu,bp,bk,bf,bj,bo,bt,by,cd,ci,-ci,-cd,-by,-bt,-bo,-bj,-bf,-bk,-bp,-bu,-bz,-ce,-cj,ch,cc,bx,bs,bn,bi,bg,bl,bq,bv,ca,cf,ck,-cg,-cb,-bw,-br,-bm,-bh} {aq,at,aw,az,bc,-be,-bb,-ay,-av,-as,-ap,-ar,-au,-ax,-ba,-bd,bd,ba,ax,au,ar,ap,as,av,ay,bb,be,-bc,-az,-aw,-at,-aq,-aq,-at,-aw,-az,-bc,be,bb,ay,av,as,ap,ar,au,ax,ba,bd,-bd,-ba,-ax,-au,-ar,-ap,-as,-av,-ay,-bb,-be,bc,az,aw,at,aq} {bi,bp,bw,cd,ck,-ce,-bx,-bq,-bj,-bh,-bo,-bv,-cc,-cj,cf,by,br,bk,bg,bn,bu,cb,ci,-cg,-bz,-bs,-bl,-bf,-bm,-bt,-ca,-ch,ch,ca,bt,bm,bf,bl,bs,bz,cg,-ci,-cb,-bu,-bn,-bg,-bk,-br,-by,-cf,cj,cc,bv,bo,bh,bj,bq,bx,ce,-ck,-cd,-bw,-bp,-bi} {ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad,ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad,ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad,ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad} {bj,bs,cb,ck,-cc,-bt,-bk,-bi,-br,-ca,-cj,cd,bu,bl,bh,bq,bz,ci,-ce,-bv,-bm,-bg,-bp,-by,-ch,cf,bw,bn,bf,bo,bx,cg,-cg,-bx,-bo,-bf,-bn,-bw,-cf,ch,by,bp,bg,bm,bv,ce,-ci,-bz,-bq,-bh,-bl,-bu,-cd,cj,ca,br,bi,bk,bt,cc,-ck,-cb,-bs,-bj} {ar,aw,bb,-bd,-ay,-at,-ap,-au,-az,-be,ba,av,aq,as,ax,bc,-bc,-ax,-as,-aq,-av,-ba,be,az,au,ap,at,ay,bd,-bb,-aw,-ar,-ar,-aw,-bb,bd,ay,at,ap,au,az,be,-ba,-av,-aq,-as,-ax,-bc,bc,ax,as,aq,av,ba,-be,-az,-au,-ap,-at,-ay,-bd,bb,aw,ar} {bk,bv,cg,-ce,-bt,-bi,-bm,-bx,-ci,cc,br,bg,bo,bz,ck,-ca,-bp,-bf,-bq,-cb,cj,by,bn,bh,bs,cd,-ch,-bw,-bl,-bj,-bu,-cf,cf,bu,bj,bl,bw,ch,-cd,-bs,-bh,-bn,-by,-cj,cb,bq,bf,bp,ca,-ck,-bz,-bo,-bg,-br,-cc,ci,bx,bm,bi,bt,ce,-cg,-bv,-bk} {ai,al,ao,-am,-aj,-ah,-ak,-an,an,ak,ah,aj,am,-ao,-al,-ai,-ai,-al,-ao,am,aj,ah,ak,an,-an,-ak,-ah,-aj,-am,ao,al,ai,ai,al,ao,-am,-aj,-ah,-ak,-an,an,ak,ah,aj,am,-ao,-al,-ai,-ai,-al,-ao,am,aj,ah,ak,an,-an,-ak,-ah,-aj,-am,ao,al,ai} {bl,by,-ck,-bx,-bk,-bm,-bz,cj,bw,bj,bn,ca,-ci,-bv,-bi,-bo,-cb,ch,bu,bh,bp,cc,-cg,-bt,-bg,-bq,-cd,cf,bs,bf,br,ce,-ce,-br,-bf,-bs,-cf,cd,bq,bg,bt,cg,-cc,-bp,-bh,-bu,-ch,cb,bo,bi,bv,ci,-ca,-bn,-bj,-bw,-cj,bz,bm,bk,bx,ck,-by,-bl} {as,az,-bd,-aw,-ap,-av,-bc,ba,at,ar,ay,-be,-ax,-aq,-au,-bb,bb,au,aq,ax,be,-ay,-ar,-at,-ba,bc,av,ap,aw,bd,-az,-as,-as,-az,bd,aw,ap,av,bc,-ba,-at,-ar,-ay,be,ax,aq,au,bb,-bb,-au,-aq,-ax,-be,ay,ar,at,ba,-bc,-av,-ap,-aw,-bd,az,as} {bm,cb,-cf,-bq,-bi,-bx,cj,bu,bf,bt,ci,-by,-bj,-bp,-ce,cc,bn,bl,ca,-cg,-br,-bh,-bw,ck,bv,bg,bs,ch,-bz,-bk,-bo,-cd,cd,bo,bk,bz,-ch,-bs,-bg,-bv,-ck,bw,bh,br,cg,-ca,-bl,-bn,-cc,ce,bp,bj,by,-ci,-bt,-bf,-bu,-cj,bx,bi,bq,cf,-cb,-bm} {ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab} {bn,ce,-ca,-bj,-br,-ci,bw,bf,bv,-cj,-bs,-bi,-bz,cf,bo,bm,cd,-cb,-bk,-bq,-ch,bx,bg,bu,-ck,-bt,-bh,-by,cg,bp,bl,cc,-cc,-bl,-bp,-cg,by,bh,bt,ck,-bu,-bg,-bx,ch,bq,bk,cb,-cd,-bm,-bo,-cf,bz,bi,bs,cj,-bv,-bf,-bw,ci,br,bj,ca,-ce,-bn} {at,bc,-ay,-ap,-ax,bd,au,as,bb,-az,-aq,-aw,be,av,ar,ba,-ba,-ar,-av,-be,aw,aq,az,-bb,-as,-au,-bd,ax,ap,ay,-bc,-at,-at,-bc,ay,ap,ax,-bd,-au,-as,-bb,az,aq,aw,-be,-av,-ar,-ba,ba,ar,av,be,-aw,-aq,-az,bb,as,au,bd,-ax,-ap,-ay,bc,at} {bo,ch,-bv,-bh,-ca,cc,bj,bt,-cj,-bq,-bm,-cf,bx,bf,by,-ce,-bl,-br,-ck,bs,bk,cd,-bz,-bg,-bw,cg,bn,bp,ci,-bu,-bi,-cb,cb,bi,bu,-ci,-bp,-bn,-cg,bw,bg,bz,-cd,-bk,-bs,ck,br,bl,ce,-by,-bf,-bx,cf,bm,bq,cj,-bt,-bj,-cc,ca,bh,bv,-ch,-bo} {aj,ao,-ak,-ai,-an,al,ah,am,-am,-ah,-al,an,ai,ak,-ao,-aj,-aj,-ao,ak,ai,an,-al,-ah,-am,am,ah,al,-an,-ai,-ak,ao,aj,aj,ao,-ak,-ai,-an,al,ah,am,-am,-ah,-al,an,ai,ak,-ao,-aj,-aj,-ao,ak,ai,an,-al,-ah,-am,am,ah,al,-an,-ai,-ak,ao,aj} {bp,ck,-bq,-bo,-cj,br,bn,ci,-bs,-bm,-ch,bt,bl,cg,-bu,-bk,-cf,bv,bj,ce,-bw,-bi,-cd,bx,bh,cc,-by,-bg,-cb,bz,bf,ca,-ca,-bf,-bz,cb,bg,by,-cc,-bh,-bx,cd,bi,bw,-ce,-bj,-bv,cf,bk,bu,-cg,-bl,-bt,ch,bm,bs,-ci,-bn,-br,cj,bo,bq,-ck,-bp} {au,-be,-at,-av,bd,as,aw,-bc,-ar,-ax,bb,aq,ay,-ba,-ap,-az,az,ap,ba,-ay,-aq,-bb,ax,ar,bc,-aw,-as,-bd,av,at,be,-au,-au,be,at,av,-bd,-as,-aw,bc,ar,ax,-bb,-aq,-ay,ba,ap,az,-az,-ap,-ba,ay,aq,bb,-ax,-ar,-bc,aw,as,bd,-av,-at,-be,au} {bq, -ci, -bl, -bv, cd, bg, ca, -by, -bi, -cf, bt, bn, ck, -bo, -bs, cg, bj, bx, -cb, -bf, -cc, bw, bk, ch, -br, -bp, cj, bm, bu, -ce, -bh, -bz, bz, bh, ce, -bu, -bm, -cj, bp, br, -ch, -bk, -bw, cc, bf, cb, -bx, -bj, -cg, bs, bo, -ck, -bn, -bt, cf, bi, by, -ca, -bg, -cd, bv, bl, ci, -bq} {ae, -ag, -ad, -af, af, ad, ag, -ae, -ae, ag, ad, af, -af, -ad, -ag, ae, ae, -ag, -ad, -af, af, ad, ag, -ae, -ae, ag, ad, af, -af, -ad, -ag, ae, ae, -ag, -ad, -af, af, ad, ag, -ae, -ae, ag, ad, af, -af, -ad, -ag, ae, ae, -ag, -ad, -af, af, ad, ag, -ae, -ae, ag, ad, af, -af, -ad, -ag, ae} {br, -cf, -bg, -cc, bu, bo, -ci, -bj, -bz, bx, bl, ck, -bm, -bw, ca, bi, ch, -bp, -bt, cd, bf, ce, -bs, -bq, cg, bh, cb, -bv, -bn, cj, bk, by, -by, -bk, -cj, bn, bv, -cb, -bh, -cg, bq, bs, -ce, -bf, -cd, bt, bp, -ch, -bi, -ca, bw, bm, -ck, -bl, -bx, bz, bj, ci, -bo, -bu, cc, bg, cf, -br} {av, -bb, -ap, -bc, au, aw, -ba, -aq, -bd, at, ax, -az, -ar, -be, as, ay, -ay, -as, be, ar, az, -ax, -at, bd, aq, ba, -aw, -au, bc, ap, bb, -av, -av, bb, ap, bc, -au, -aw, ba, aq, bd, -at, -ax, az, ar, be, -as, -ay, ay, as, -be, -ar, -az, ax, at, -bd, -aq, -ba, aw, au, -bc, -ap, -bb, av} {bs,-cc,-bi,-cj,bl,bz,-bv,-bp,cf,bf,cg,-bo,-bw,by,bm,-ci,-bh,-cd,br,bt,-cb,-bj,-ck,bk,ca,-bu,-bq,ce,bg,ch,-bn,-bx,bx,bn,-ch,-bg,-ce,bq,bu,-ca,-bk,ck,bj,cb,-bt,-br,cd,bh,ci,-bm,-by,bw,bo,-cg,-bf,-cf,bp,bv,-bz,-bl,cj,bi,cc,-bs} {ak,-am,-ai,ao,ah,an,-aj,-al,al,aj,-an,-ah,-ao,ai,am,-ak,-ak,am,ai,-ao,-ah,-an,aj,al,-al,-aj,an,ah,ao,-ai,-am,ak,ak,-am,-ai,ao,ah,an,-aj,-al,al,aj,-an,-ah,-ao,ai,am,-ak,-ak,am,ai,-ao,-ah,-an,aj,al,-al,-aj,an,ah,ao,-ai,-am,ak} {bt,-bz,-bn,cf,bh,ck,-bi,-ce,bo,by,-bu,-bs,ca,bm,-cg,-bg,-cj,bj,cd,-bp,-bx,bv,br,-cb,-bl,ch,bf,ci,-bk,-cc,bq,bw,-bw,-bq,cc,bk,-ci,-bf,-ch,bl,cb,-br,-bv,bx,bp,-cd,-bj,cj,bg,cg,-bm,-ca,bs,bu,-by,-bo,ce,bi,-ck,-bh,-cf,bn,bz,-bt} {aw,-ay,-au,ba,as,-bc,-aq,be,ap,bd,-ar,-bb,at,az,-av,-ax,ax,av,-az,-at,bb,ar,-bd,-ap,-be,aq,bc,-as,-ba,au,ay,-aw,-aw,ay,au,-ba,-as,bc,aq,-be,-ap,-bd,ar,bb,-at,-az,av,ax,-ax,-av,az,at,-bb,-ar,bd,ap,be,-aq,-bc,as,ba,-au,-ay,aw} {bu, -bw, -bs, by, bq, -ca, -bo, cc, bm, -ce, -bk, cg, bi, -ci, -bg, ck, bf, cj, -bh, -ch, bj, cf, -bl, -cd, bn, cb, -bp, -bz, br, bx, -bt, -bv, bv, bt, -bx, -br, bz, bp, -cb, -bn, cd, bl, -cf, -bj, ch, bh, -cj, -bf, -ck, bg, ci, -bi, -cg, bk, ce, -bm, -cc, bo, ca, -bq, -by, bs, bw, -bu} {aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa, aa, -aa, -aa, aa} {bv, -bt, -bx, br, bz, -bp, -cb, bn, cd, -bl, -cf, bj, ch, -bh, -cj, bf, -ck, -bg, ci, bi, -cg, -bk, ce, bm, -cc, -bo, ca, bq, -by, -bs, bw, bu, -bu, -bw, bs, by, -bq, -ca, bo, cc, -bm, -ce, bk, cg, -bi, -ci, bg, ck, -bf, cj, bh, -ch, -bj, cf, bl, -cd, -bn, cb, bp, -bz, -br, bx, bt, -bv} {ax, -av, -az, at, bb, -ar, -bd, ap, -be, -aq, bc, as, -ba, -au, ay, aw, -aw, -ay, au, ba, -as, -bc, aq, be, -ap, bd, ar, -bb, -at, az, av, -ax, -ax, av, az, -at, -bb, ar, bd, -ap, be, aq, -bc, -as, ba, au, -ay, -aw, aw, ay, -au, -ba, as, bc, -aq, -be, ap, -bd, -ar, bb, at, -az, -av, ax} {bw,-bq,-cc,bk,ci,-bf,ch,bl,-cb,-br,bv,bx,-bp,-cd,bj,cj,-bg,cg,bm,-ca,-bs,bu,by,-bo,-ce,bi,ck,-bh,cf,bn,-bz,-bt,bt,bz,-bn,-cf,bh,-ck,-bi,ce,bo,-by,-bu,bs,ca,-bm,-cg,bg,-cj,-bj,cd,bp,-bx,-bv,br,cb,-bl,-ch,bf,-ci,-bk,cc,bq,-bw} {al,-aj,-an,ah,-ao,-ai,am,ak,-ak,-am,ai,ao,-ah,an,aj,-al,-al,aj,an,-ah,ao,ai,-am,-ak,ak,am,-ai,-ao,ah,-an,-aj,al,al,-aj,-an,ah,-ao,-ai,am,ak,-ak,-am,ai,ao,-ah,an,aj,-al,-al,aj,an,-ah,ao,ai,-am,-ak,ak,am,-ai,-ao,ah,-an,-aj,al} {bx,-bn,-ch,bg,-ce,-bq,bu,ca,-bk,-ck,bj,-cb,-bt,br,cd,-bh,ci,bm,-by,-bw,bo,cg,-bf,cf,bp,-bv,-bz,bl,cj,-bi,cc,bs,-bs,-cc,bi,-cj,-bl,bz,bv,-bp,-cf,bf,-cg,-bo,bw,by,-bm,-ci,bh,-cd,-br,bt,cb,-bj,ck,bk,-ca,-bu,bq,ce,-bg,ch,bn,-bx} {ay,-as,-be,ar,-az,-ax,at,bd,-aq,ba,aw,-au,-bc,ap,-bb,-av,av,bb,-ap,bc,au,-aw,-ba,aq,-bd,-at,ax,az,-ar,be,as,-ay,-ay,as,be,-ar,az,ax,-at,-bd,aq,-ba,-aw,au,bc,-ap,bb,av,-av,-bb,ap,-bc,-au,aw,ba,-aq,bd,at,-ax,-az,ar,-be,-as,ay} {by,-bk,cj,bn,-bv,-cb,bh,-cg,-bq,bs,ce,-bf,cd,bt,-bp,-ch,bi,-ca,-bw,bm,ck,-bl,bx,bz,-bj,ci,bo,-bu,-cc,bg,-cf,-br,br,cf,-bg,cc,bu,-bo,-ci,bj,-bz,-bx,bl,-ck,-bm,bw,ca,-bi,ch,bp,-bt,-cd,bf,-ce,-bs,bq,cg,-bh,cb,bv,-bn,-cj,bk,-by} {af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af,af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af,af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af,af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af} {bz,-bh,ce,bu,-bm,cj,bp,-br,-ch,bk,-bw,-cc,bf,-cb,-bx,bj,-cg,-bs,bo,ck,-bn,bt,cf,-bi,by,ca,-bg,cd,bv,-bl,ci,bq,-bq,-ci,bl,-bv,-cd,bg,-ca,-by,bi,-cf,-bt,bn,-ck,-bo,bs,cg,-bj,bx,cb,-bf,cc,bw,-bk,ch,br,-bp,-cj,bm,-bu,-ce,bh,-bz} {az,-ap,ba,ay,-aq,bb,ax,-ar,bc,aw,-as,bd,av,-at,be,au,-au,-be,at,-av,-bd,as,-aw,-bc,ar,-ax,-bb,aq,-ay,-ba,ap,-az,-az,ap,-ba,-ay,aq,-bb,-ax,ar,-bc,-aw,as,-bd,-av,at,-be,-au,au,be,-at,av,bd,-as,aw,bc,-ar,ax,bb,-aq,ay,ba,-ap,az} {ca,-bf,bz,cb,-bg,by,cc,-bh,bx,cd,-bi,bw,ce,-bj,bv,cf,-bk,bu,cg,-bl,bt,ch,-bm,bs,ci,-bn,br,cj,-bo,bq,ck,-bp,bp,-ck,-bq,bo,-cj,-br,bn,-ci,-bs,bm,-ch,-bt,bl,-cg,-bu,bk,-cf,-bv,bj,-ce,-bw,bi,-cd,-bx,bh,-cc,-by,bg,-cb,-bz,bf,-ca} {am,-ah,al,an,-ai,ak,ao,-aj,aj,-ao,-ak,ai,-an,-al,ah,-am,-am,ah,-al,-an,ai,-ak,-ao,aj,-aj,ao,ak,-ai,an,al,-ah,am,am,-ah,al,an,-ai,ak,ao,-aj,aj,-ao,-ak,ai,-an,-al,ah,-am,-am,ah,-al,-an,ai,-ak,-ao,aj,-aj,ao,ak,-ai,an,al,-ah,am} {cb,-bi,bu,ci,-bp,bn,-cg,-bw,bg,-bz,-cd,bk,-bs,-ck,br,-bl,ce,by,-bf,bx,cf,-bm,bq,-cj,-bt,bj,-cc,-ca,bh,-bv,-ch,bo,-bo,ch,bv,-bh,ca,cc,-bj,bt,cj,-bq,bm,-cf,-bx,bf,-by,-ce,bl,-br,ck,bs,-bk,cd,bz,-bg,bw,cg,-bn,bp,-ci,-bu,bi,-cb} {ba,-ar,av,-be,-aw,aq,-az,-bb,as,-au,bd,ax,-ap,ay,bc,-at,at,-bc,-ay,ap,-ax,-bd,au,-as,bb,az,-aq,aw,be,-av,ar,-ba,-ba,ar,-av,be,aw,-aq,az,bb,-as,au,-bd,-ax,ap,-ay,-bc,at,-at,bc,ay,-ap,ax,bd,-au,as,-bb,-az,aq,-aw,-be,av,-ar,ba} {cc,-bl,bp,-cg,-by,bh,-bt,ck,bu,-bg,bx,ch,-bq,bk,-cb,-cd,bm,-bo,cf,bz,-bi,bs,-cj,-bv,bf,-bw,-ci,br,-bj,ca,ce,-bn,bn,-ce,-ca,bj,-br,ci,bw,-bf,bv,cj,-bs,bi,-bz,-cf,bo,-bm,cd,cb,-bk,bq,-ch,-bx,bg,-bu,-ck,bt,-bh,by,cg,-bp,bl,-cc} {ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac,ac,-ab,ab,-ac,-ac,ab,-ab,ac} {cd,-bo,bk,-bz,-ch,bs,-bg,bv,-ck,-bw,bh,-br,cg,ca,-bl,bn,-cc,-ce,bp,-bj,by,ci,-bt,bf,-bu,cj,bx,-bi,bq,-cf,-cb,bm,-bm,cb,cf,-bq,bi,-bx,-cj,bu,-bf,bt,-ci,-by,bj,-bp,ce,cc,-bn,bl,-ca,-cg,br,-bh,bw,ck,-bv,bg,-bs,ch,bz,-bk,bo,-cd} {bb,-au,aq,-ax,be,ay,-ar,at,-ba,-bc,av,-ap,aw,-bd,-az,as,-as,az,bd,-aw,ap,-av,bc,ba,-at,ar,-ay,-be,ax,-aq,au,-bb,-bb,au,-aq,ax,-be,-ay,ar,-at,ba,bc,-av,ap,-aw,bd,az,-as,as,-az,-bd,aw,-ap,av,-bc,-ba,at,-ar,ay,be,-ax,aq,-au,bb} {ce,-br,bf,-bs,cf,cd,-bq,bg,-bt,cg,cc,-bp,bh,-bu,ch,cb,-bo,bi,-bv,ci,ca,-bn,bj,-bw,cj,bz,-bm,bk,-bx,ck,by,-bl,bl,-by,-ck,bx,-bk,bm,-bz,-cj,bw,-bj,bn,-ca,-ci,bv,-bi,bo,-cb,-ch,bu,-bh,bp,-cc,-cg,bt,-bg,bq,-cd,-cf,bs,-bf,br,-ce} {an,-ak,ah,-aj,am,ao,-al,ai,-ai,al,-ao,-am,aj,-ah,ak,-an,-an,ak,-ah,aj,-am,-ao,al,-ai,ai,-al,ao,am,-aj,ah,-ak,an,an,-ak,ah,-aj,am,ao,-al,ai,-ai,al,-ao,-am,aj,-ah,ak,-an,-an,ak,-ah,aj,-am,-ao,al,-ai,ai,-al,ao,am,-aj,ah,-ak,an} {cf,-bu,bj,-bl,bw,-ch,-cd,bs,-bh,bn,-by,cj,cb,-bq,bf,-bp,ca,ck,-bz,bo,-bg,br,-cc,-ci,bx,-bm,bi,-bt,ce,cg,-bv,bk,-bk,bv,-cg,-ce,bt,-bi,bm,-bx,ci,cc,-br,bg,-bo,bz,-ck,-ca,bp,-bf,bq,-cb,-cj,by,-bn,bh,-bs,cd,ch,-bw,bl,-bj,bu,-cf} {bc,-ax,as,-aq,av,-ba,-be,az,-au,ap,-at,ay,-bd,-bb,aw,-ar,ar,-aw,bb,bd,-ay,at,-ap,au,-az,be,ba,-av,aq,-as,ax,-bc,-bc,ax,-as,aq,-av,ba,be,-az,au,-ap,at,-ay,bd,bb,-aw,ar,-ar,aw,-bb,-bd,ay,-at,ap,-au,az,-be,-ba,av,-aq,as,-ax,bc} {cg,-bx,bo,-bf,bn,-bw,cf,ch,-by,bp,-bg,bm,-bv,ce,ci,-bz,bq,-bh,bl,-bu,cd,cj,-ca,br,-bi,bk,-bt,cc,ck,-cb,bs,-bj,bj,-bs,cb,-ck,-cc,bt,-bk,bi,-br,ca,-cj,-cd,bu,-bl,bh,-bq,bz,-ci,-ce,bv,-bm,bg,-bp,by,-ch,-cf,bw,-bn,bf,-bo,bx,-cg} {ag,-af,ae,-ad,ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag,ag,-af,ae,-ad,ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag,ag,-af,ae,-ad,ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag,ag,-af,ae,-ad,ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag} {ch,-ca,bt,-bm,bf,-bl,bs,-bz,cg,ci,-cb,bu,-bn,bg,-bk,br,-by,cf,cj,-cc,bv,-bo,bh,-bj,bq,-bx,ce,ck,-cd,bw,-bp,bi,-bi,bp,-bw,cd,-ck,-ce,bx,-bq,bj,-bh,bo,-bv,cc,-cj,-cf,by,-br,bk,-bg,bn,-bu,cb,-ci,-cg,bz,-bs,bl,-bf,bm,-bt,ca,-ch} {bd,-ba,ax,-au,ar,-ap,as,-av,ay,-bb,be,bc,-az,aw,-at,aq,-aq,at,-aw,az,-bc,-be,bb,-ay,av,-as,ap,-ar,au,-ax,ba,-bd,-bd,ba,-ax,au,-ar,ap,-as,av,-ay,bb,-be,-bc,az,-aw,at,-aq,aq,-at,aw,-az,bc,be,-bb,ay,-av,as,-ap,ar,-au,ax,-ba,bd} {ci,-cd,by,-bt,bo,-bj,bf,-bk,bp,-bu,bz,-ce,cj,ch,-cc,bx,-bs,bn,-bi,bg,-bl,bq,-bv,ca,-cf,ck,cg,-cb,bw,-br,bm,-bh,bh,-bm,br,-bw,cb,-cg,-ck,cf,-ca,bv,-bq,bl,-bg,bi,-bn,bs,-bx,cc,-ch,-cj,ce,-bz,bu,-bp,bk,-bf,bj,-bo,bt,-by,cd,-ci} {ao,-an,am,-al,ak,-aj,ai,-ah,ah,-ai,aj,-ak,al,-am,an,-ao,-ao,an,-am,al,-ak,aj,-ai,ah,-ah,ai,-aj,ak,-al,am,-an,ao,ao,-an,am,-al,ak,-aj,ai,-ah,ah,-ai,aj,-ak,al,-am,an,-ao,-ao,an,-am,al,-ak,aj,-ai,ah,-ah,ai,-aj,ak,-al,am,-an,ao} {cj,-cg,cd,-ca,bx,-bu,br,-bo,bl,-bi,bf,-bh,bk,-bn,bq,-bt,bw,-bz,cc,-cf,ci,ck,-ch,ce,-cb,by,-bv,bs,-bp,bm,-bj,bg,-bg,bj,-bm,bp,-bs,bv,-by,cb,-ce,ch,-ck,-ci,cf,-cc,bz,-bw,bt,-bq,bn,-bk,bh,-bf,bi,-bl,bo,-br,bu,-bx,ca,-cd,cg,-cj} {be,-bd,bc,-bb,ba,-az,ay,-ax,aw,-av,au,-at,as,-ar,aq,-ap,ap,-aq,ar,-as,at,-au,av,-aw,ax,-ay,az,-ba,bb,-bc,bd,-be,-be,bd,-bc,bb,-ba,az,-ay,ax,-aw,av,-au,at,-as,ar,-aq,ap,-ap,aq,-ar,as,-at,au,-av,aw,-ax,ay,-az,ba,-bb,bc,-bd,be} {ck,-cj,ci,-ch,cg,-cf,ce,-cd,cc,-cb,ca,-bz,by,-bx,bw,-bv,bu,-bt,bs,-br,bq,-bp,bo,-bn,bm,-bl,bk,-bj,bi,-bh,b ,bf,-bg,bh,-bi,bj,-bk,bl,-bm,bn,-bo,bp,-bq,br,-bs,bt,-bu,bv,-bw,bx,-by,bz,-ca,cb,-cc,cd,-ce,cf,-cj,ch,-ck,ci, } ただし、 {aa,ab,ac,ad,ae,af,ag,ah,ai,aj,ak,al,am,an,ao,ap,aq,ar,as,at,au,av,aw,ax,ay,z,ba,bb,bc,bd,be,b f,bg,bm,bn,bo,bp,bq,br,b,bt,bu,bv,bw,bx,bx,ca,cb,cc,cd,ce,cf,cg,ch,ci,cj,ck}= {64,83,36,89,75,50,18,90,87,80,70,57,43,25,9,90,90,88,85,82,78,73,67,61,54,46,38,31,22,13,4,9 1,90,90,90,88,87,86,84,83,81,79,77,73,71,69,65,62,59,56,52,48,44,41,37,33,28,24,20,15,11,7,2} である。

[0096] In addition to DCT-2 and 4×4 DST-7 that have been adopted in HEVC, in VVC, for residual coding of both inter-coded blocks and intra-coded blocks, an Adaptive Multiple Transform (AMT, also known as Enhanced Multiple Transform; EMT, or known as Multiple Transform Selection; MTS) scheme is used in VVC. MTS uses multiple selected transforms from the DCT / DST family other than the current transform in HEVC. The newly introduced transform matrices are DST-7 and DCT-8. Table 2 shows the basic functions of the selected DST / DCTs.

Table 2

[0097] All primary transform matrices in VVC can be used in 8-bit representation. AMT is applied to CUs where both the width and height are 32 or less, and whether to apply AMT is controlled by a flag called mts_flag. When mts_flag is equal to 0, only DCT-2 can be applied to encode the residual. When mts_flag is equal to 1, an additional index mts_idx is signaled using two bins to specify the horizontal and vertical transforms used according to Table 3, where a value of 1 means using DST-7 and a value of 2 means using DCT-8.

Table 3

[0098] In VVC Draft 4, when the signaling-based MTS above (i.e., explicit MTS) is not used, implicit MTS can also be applied. In implicit MTS, conversion selection is made according to the block width and height instead of signaling. More specifically, in the implicit MTS proposed in JVET-M0303, DST-7 is selected for the short side of the block, and DCT-2 is selected for the long side of the block. The transform core of DST-7, which is a matrix composed of basis vectors, can also be expressed as follows: 4-point DST-7: {a, b, c, d} {c, c, 0, -c} {d, -a, -c, b} {b, -d, c, -a} However, {a, b, c, d} = {29, 55, 74, 84}. 8-point DST-7: {a, b, c, d, e, f, g, h,} {c, f, h, e, b, -a, -d, -g,} {e, g, b, -c, -h, -d, a, f,} {g, c, -d, -f, a, h, b, -e,} {h, -a, -g, b, f, -c, -e, d,} {f, -e, -a, g, -d, -b, h, -c,} {d, -h, e, -a, -c, g, -f, b,} {b, -d, f, -h, g, -e, c, -a,} However, {a, b, c, d, e, f, g, h} = {17, 32, 46, 60, 71, 78, 85, 86}. 16-point DST-7: {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p,} {c, f, i, l, o, o, l, i, f, c, 0, -c, -f, -i, -l, -o,} {e, j, o, m, h, c, -b, -g, -l, -p, -k, -f, -a, d, i, n,} {g, n, l, e, -b, -i, -p, -j, -c, d, k, o, h, a, -f, -m,} {i,o,f,-c,-l,-l,-c,f,o,i,0,-i,-o,-f,c,l,} {k,k,0,-k,-k,0,k,k,0,-k,-k,0,k,k,0,-k,} {m,g,-f,-n,-a,l,h,-e,-o,-b,k,i,-d,-p,-c,j,} {o,c,-l,-f,i,i,-f,-l,c,o,0,-o,-c,l,f,-i,} {p,-a,-o,b,n,-c,-m,d,l,-e,-k,f,j,-g,-i,h,} {n,-e,-i,j,d,-o,a,m,-f,-h,k,c,-p,b,l,-g,} {l,-i,-c,o,-f,-f,o,-c,-i,l,0,-l,i,c,-o,f,} {j,-m,c,g,-p,f,d,-n,i,a,-k,l,-b,-h,o,-e,} {h,-p,i,-a,-g,o,-j,b,f,-n,k,-c,-e,m,-l,d,} {f,-l,o,-i,c,c,-i,o,-l,f,0,-f,l,-o,i,-c,} {d,-h,l,-p,m,-i,e,-a,-c,g,-k,o,-n,j,-f,b,} {b,-d,f,-h,j,-l,n,-p,o,-m,k,-i,g,-e,c,-a,} However, {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p} = {9,17,25,33,41,49,56,62,66,72,77,81,83,87,89,90}. 32 - point DST - 7: {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,A,B,C,D,E,F,} {c,f,i,l,o,r,u,x,A,D,F,C,z,w,t,q,n,k,h,e,b,-a,-d,-g,-j,-m,-p,-s,-v,-y,-B,-E,} {e,j,o,t,y,D,D,y,t,o,j,e,0,-e,-j,-o,-t,-y,-D,-D,-y,-t,-o,-j,-e,0,e,j,o,t,y,D,} {g,n,u,B,D,w,p,i,b,-e,-l,-s,-z,-F,-y,-r,-k,-d,c,j,q,x,E,A,t,m,f,-a,-h,-o,-v,-C,} {i,r,A,C,t,k,b,-g,-p,-y,-E,-v,-m,-d,e,n,w,F,x,o,f,-c,-l,-u,-D,-z,-q,-h,a,j,s,B,} {k,v,F,u,j,-a,-l,-w,-E,-t,-i,b,m,x,D,s,h,-c,-n,-y,-C,-r,-g,d,o,z,B,q,f,-e,-p,-A,} {m,z,z,m,0,-m,-z,-z,-m,0,m,z,z,m,0,-m,-z,-z,-m,0,m,z,z,m,0,-m,-z,-z,-m,0,m,z,} {o,D,t,e,-j,-y,-y,-j,e,t,D,o,0,-o,-D,-t,-e,j,y,y,j,-e,-t,-D,-o,0,o,D,t,e,-j,-y,} {q,E,n,-c,-t,-B,-k,f,w,y,h,-i,-z,-v,-e,l,C,s,b,-o,-F,-p,a,r,D,m,-d,-u,-A,-j,g,x,} {s,A,h,-k,-D,-p,c,v,x,e,-n,-F,-m,f,y,u,b,-q,-C,-j,i,B,r,-a,-t,-z,-g,l,E,o,-d,-w,} {u,w,b,-s,-y,-d,q,A,f,-o,-C,-h,m,E,j,-k,-F,-l,i,D,n,-g,-B,-p,e,z,r,-c,-x,-t,a,v,} {w,s,-d,-A,-o,h,E,k,-l,-D,-g,p,z,c,-t,-v,a,x,r,-e,-B,-n,i,F,j,-m,-C,-f,q,y,b,-u,} {y,o,-j,-D,-e,t,t,-e,-D,j,o,y,0,-y,-o,j,D,e,-t,-t,e,D,j,-o,-y,0,y,o,-j,-D,-e,t,} {A,k,-p,-v,e,F,f,-u,-q,j,B,a,-z,-l,o,w,-d,-E,-g,t,r,-i,-C,-b,y,m,-n,-x,c,D,h,-s,} {C,g,-v,-n,o,u,-h,-B,a,D,f,-w,-m,p,t,-i,-A,b,E,e,-x,-l,q,s,-j,-z,c,F,d,-y,-k,r,} {E,c,-B,-f,y,i,-v,-l,s,o,-p,-r,m,u,-j,-x,g,A,-d,-D,a,F,b,-C,-e,z,h,-w,-k,t,n,-q,} {F,-a,-E,b,D,-c,-C,d,B,-e,-A,f,z,-g,-y,h,x,-i,-w,j,v,-k,-u,l,t,-m,-s,n,r,-o,-q,p,} {D,-e,-y,j,t,-o,-o,t,j,-y,-e,D,0,-D,e,y,-j,-t,o,o,-t,-j,y,e,-D,0,D,-e,-y,j,t,-o,} {B,-i,-s,r,j,-A,-a,C,-h,-t,q,k,-z,-b,D,-g,-u,p,l,-y,-c,E,-f,-v,o,m,-x,-d,F,-e,-w,n,} {z,-m,-m,z,0,-z,m,m,-z,0,z,-m,-m,z,0,-z,m,m,-z,0,z,-m,-m,z,0,-z,m,m,-z,0,z,-m,} {x,-q,-g,E,-j,-n,A,-c,-u,t,d,-B,m,k,-D,f,r,-w,-a,y,-p,-h,F,-i,-o,z,-b,-v,s,e,-C,l,} {v,-u,-a,w,-t,-b,x,-s,-c,y,-r,-d,z,-q,-e,A,-p,-f,B,-o,-g,C,-n,-h,D,-m,-i,E,-l,-j,F,-k,} {t,-y,e,o,-D,j,j,-D,o,e,-y,t,0,-t,y,-e,-o,D,-j,-j,D,-o,-e,y,-t,0,t,-y,e,o,-D,j,} {r,-C,k,g,-y,v,-d,-n,F,-o,-c,u,-z,h,j,-B,s,-a,-q,D,-l,-f,x,-w,e,m,-E,p,b,-t,A,-i,} {p,-F,q,-a,-o,E,-r,b,n,-D,s,-c,-m,C,-t,d,l,-B,u,-e,-k,A,-v,f,j,-z,w,-g,-i,y,-x,h,} {n,-B,w,-i,-e,s,-F,r,-d,-j,x,-A,m,a,-o,C,-v,h,f,-t,E,-q,c,k,-y,z,-l,-b,p,-D,u,-g,} {l,-x,C,-q,e,g,-s,E,-v,j,b,-n,z,-A,o,-c,-i,u,-F,t,-h,-d,p,-B,y,-m,a,k,-w,D,-r,f,} {j,-t,D,-y,o,-e,-e,o,-y,D,-t,j,0,-j,t,-D,y,-o,e,e,-o,y,-D,t,-j,0,j,-t,D,-y,o,-e,} {h,-p,x,-F,y,-q,i,-a,-g,o,-w,E,-z,r,-j,b,f,-n,v,-D,A,-s,k,-c,-e,m,-u,C,-B,t,-l,d,} {f,-l,r,-x,D,-C,w,-q,k,-e,-a,g,-m,s,-y,E,-B,v,-p,j,-d,-b,h,-n,t,-z,F,-A,u,-o,i,-c,} {d,-h,l,-p,t,-x,B,-F,C,-y,u,-q,m,-i,e,-a,-c,g,-k,o,-s,w,-A,E,-D,z,-v,r,-n,j,-f,b,} {b,-d,f,-h,j,-l,n,-p,r,-t,v,-x,z,-B,D,-F,E,-C,A,-y,w,-u,s,-q,o,-m,k,-i,g,-e,c,-a,} However, {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, A, B, C, D, E, F} = {4, 9, 13, 17, 21, 26, 30, 34, 38, 42, 45, 50, 53, 56, 60, 63, 66, 68, 72, 74, 77, 78, 80, 82, 84, 85, 86, 88, 88, 89, 90, 90}. 4 - point DCT - 8: {a, b, c, d,} {b, 0, -b, -b,} {c, -b, -d, a,} {d, -b, a, -c,} However, {a, b, c, d} = {84, 74, 55, 29}. 8 - point DCT - 8: {a, b, c, d, e, f, g, h,} {b, e, h, -g, -d, -a, -c, -f,} {c, h, -e, -a, -f, g, b, d,} {d, -g, -a, -h, c, e, -f, -b,} {e, -d, -f, c, g, -b, -h, a,} {f, -a, g, e, -b, h, d, -c,} {g, -c, b, -f, -h, d, -a, e,} {h, -f, d, -b, a, -c, e, -g,} However, {a, b, c, d, e, f, g, h} = {86, 85, 78, 71, 60, 46, 32, 17}. 16 - point DCT - 8: {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p,} {b, e, h, k, n, 0, -n, -k, -h, -e, -b, -b, -e, -h, -k, -n,} {c, h, m, -p, -k, -f, -a, -e, -j, -o, n, i, d, b, g, l,} {d, k, -p, -i, -b, -f, -m, n, g, a, h, o, -l, -e, -c, -j,} {e, n, -k, -b, -h, 0, h, b, k, -n, -e, -e, -n, k, b, h,} {f,0,-f,-f,0,f,f,0,-f,-f,0,f,f,0,-f,-f,} {g,-n,-a,-m,h,f,-o,-b,-l,i,e,-p,-c,-k,j,d,} {h,-k,-e,n,b,0,-b,-n,e,k,-h,-h,k,e,-n,-b,} {i,-h,-j,g,k,-f,-l,e,m,-d,-n,c,o,-b,-p,a,} {j,-e,-o,a,-n,-f,i,k,-d,-p,b,-m,-g,h,l,-c,} {k,-b,n,h,-e,0,e,-h,-n,b,-k,-k,b,-n,-h,e,} {l,-b,i,o,-e,f,-p,-h,c,-m,-k,a,-j,-n,d,-g,} {m,-e,d,-l,-n,f,-c,k,o,-g,b,-j,-p,h,-a,i,} {n,-h,b,-e,k,0,-k,e,-b,h,-n,-n,h,-b,e,-k,} {o,-k,g,-c,b,-f,j,-n,-p,l,-h,d,-a,e,-i,m,} {p,-n,l,-j,h,-f,d,-b,a,-c,e,-g,i,-k,m,-o,} However, {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p} = {90,89,87,83,81,77,72,66,62,56,49,41,33,25,17,9}. 32 - point DCT - 8: {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,A,B,C,D,E,F,} {b,e,h,k,n,q,t,w,z,C,F,-E,-B,-y,-v,-s,-p,-m,-j,-g,-d,-a,-c,-f,-i,-l,-o,-r,-u,-x,-A,-D,} {c,h,m,r,w,B,0,-B,-w,-r,-m,-h,-c,-c,-h,-m,-r,-w,-B,0,B,w,r,m,h,c,c,h,m,r,w,B,} {d,k,r,y,F,-A,-t,-m,-f,-b,-i,-p,-w,-D,C,v,o,h,a,g,n,u,B,-E,-x,-q,-j,-c,-e,-l,-s,-z,} {e,n,w,F,-y,-p,-g,-c,-l,-u,-D,A,r,i,a,j,s,B,-C,-t,-k,-b,-h,-q,-z,E,v,m,d,f,o,x,} {f,q,B,-A,-p,-e,-g,-r,-C,z,o,d,h,s,D,-y,-n,-c,-i,-t,-E,x,m,b,j,u,F,-w,-l,-a,-k,-v,} {g,t,0,-t,-g,-g,-t,0,t,g,g,t,0,-t,-g,-g,-t,0,t,g,g,t,0,-t,-g,-g,-t,0,t,g,g,t,} {h,w,-B,-m,-c,-r,0,r,c,m,B,-w,-h,-h,-w,B,m,c,r,0,-r,-c,-m,-B,w,h,h,w,-B,-m,-c,-r,} {i,z,-w,-f,-l,-C,t,c,o,F,-q,-a,-r,E,n,d,u,-B,-k,-g,-x,y,h,j,A,-v,-e,-m,-D,s,b,p,} {j,C,-r,-b,-u,z,g,m,F,-o,-e,-x,w,d,p,-E,-l,-h,-A,t,a,s,-B,-i,-k,-D,q,c,v,-y,-f,-n,} {k,F,-m,-i,-D,o,g,B,-q,-e,-z,s,c,x,-u,-a,-v,w,b,t,-y,-d,-r,A,f,p,-C,-h,-n,E,j,l,} {l,-E,-h,-p,A,d,t,-w,-a,-x,s,e,B,-o,-i,-F,k,m,-D,-g,-q,z,c,u,-v,-b,-y,r,f,C,-n,-j,} {m,-B,-c,-w,r,h,0,-h,-r,w,c,B,-m,-m,B,c,w,-r,-h,0,h,r,-w,-c,-B,m,m,-B,-c,-w,r,h,} {n,-y,-c,-D,i,s,-t,-h,E,d,x,-o,-m,z,b,C,-j,-r,u,g,-F,-e,-w,p,l,-A,-a,-B,k,q,-v,-f,} {o,-v,-h,C,a,D,-g,-w,n,p,-u,-i,B,b,E,-f,-x,m,q,-t,-j,A,c,F,-e,-y,l,r,-s,-k,z,d,} {p,-s,-m,v,j,-y,-g,B,d,-E,-a,-F,c,C,-f,-z,i,w,-l,-t,o,q,-r,-n,u,k,-x,-h,A,e,-D,-b,} {q,-p,-r,o,s,-n,-t,m,u,-l,-v,k,w,-j,-x,i,y,-h,-z,g,A,-f,-B,e,C,-d,-D,c,E,-b,-F,a,} {r,-m,-w,h,B,-c,0,c,-B,-h,w,m,-r,-r,m,w,-h,-B,c,0,-c,B,h,-w,-m,r,r,-m,-w,h,B,-c,} {s,-j,-B,a,-C,-i,t,r,-k,-A,b,-D,-h,u,q,-l,-z,c,-E,-g,v,p,-m,-y,d,-F,-f,w,o,-n,-x,e,} {t,-g,0,g,-t,-t,g,0,-g,t,t,-g,0,g,-t,-t,g,0,-g,t,t,-g,0,g,-t,-t,g,0,-g,t,t,-g,} {u,-d,B,n,-k,-E,g,-r,-x,a,-y,-q,h,-F,-j,o,A,-c,v,t,-e,C,m,-l,-D,f,-s,-w,b,-z,-p,i,} {v,-a,w,u,-b,x,t,-c,y,s,-d,z,r,-e,A,q,-f,B,p,-g,C,o,-h,D,n,-i,E,m,-j,F,l,-k,} {w,-c,r,B,-h,m,0,-m,h,-B,-r,c,-w,-w,c,-r,-B,h,-m,0,m,-h,B,r,-c,w,w,-c,r,B,-h,m,} {x, -f, m, -E, -q, b, -t, -B, j, -i, A, u, -c, p, F, -n, e, -w, -y, g, -l, D, r, -a, s, C, -k, h, -z, -v, d, -o,} {y, -i, h, -x, -z, j, -g, w, A, -k, f, -v, -B, l, -e, u, C, -m, d, -t, -D, n, -c, s, E, -o, b, -r, -F, p, -a, q,} {z, -l, c, -q, E, u, -g, h, -v, -D, p, -b, m, -A, -y, k, -d, r, -F, -t, f, -i, w, C, -o, a, -n, B, x, -j, e, -s,} {A, -o, c, -j, v, F, -t, h, -e, q, -C, -y, m, -a, l, -x, -D, r, -f, g, -s, E, w, -k, b, -n, z, B, -p, d, -i, u,} {B, -r, h, -c, m, -w, 0, w, -m, c, -h, r, -B, -B, r, -h, c, -m, w, 0, -w, m, -c, h, -r, B, B, -r, h, -c, m, -w,} {C, -u, m, -e, d, -l, t, -B, -D, v, -n, f, -c, k, -s, A, E, -w, o, -g, b, -j, r, -z, -F, x, -p, h, -a, i, -q, y,} {D, -x, r, -l, f, -a, g, -m, s, -y, E, C, -w, q, -k, e, -b, h, -n, t, -z, F, B, -v, p, -j, d, -c, i, -o, u, -A,} {E, -A, w, -s, o, -k, g, -c, b, -f, j, -n, r, -v, z, -D, -F, B, -x, t, -p, l, -h, d, -a, e, -i, m, -q, u, -y, C,} {F, -D, B, -z, x, -v, t, -r, p, -n, l, -j, h, -f, d, -b, a, -c, e, -g, i, -k, m, -o, q, -s, u, -w, y, -A, C, -E,} However, {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, A, B, C, D, E, F} = {90, 90, 89, 88, 88, 86, 85, 84, 82, 80, 78, 77, 74, 72, 68, 66, 63, 60, 56, 53, 50, 45, 42, 38, 34, 30, 26, 21, 17, 13, 9, 4}.

[0099] In VVC, in some cases, when both the height and width of the coding block are 64 or less, the transform size can always be the same as the coding block size. When either the height or width of the coding block is greater than 64, when performing transform or intra prediction, the coding block is further divided into a plurality of sub-blocks so that the width and height of each sub-block are 64 or less, and one transform is executed for each sub-block.

[0100] In VVC draft v5, the syntax in Table 4 below can be used to enable or disable MTS in the SPS. [Table 4]

[0101] In VVC draft v5, in some cases, DST-7 and / or DCT-8 can be used without being explicitly signaled, that is, DST-7 and / or DCT-8 can be used implicitly based on information available to both the encoder and the decoder. These cases include the following: (a) Intra Sub-Partitioning (ISP): In the ISP mode, as long as the block width is 4 or more and 16 or less, the horizontal transform is selected as DST-7, and as long as the block height is 4 or more and 16 or less, the vertical transform is selected as DST-7. (b) Sub-Block Transform (SBT): In the SBT mode, for the sub-TUs located in the left half (or quarter) and the right half (or quarter) of the current CU, the horizontal transforms are DCT-8 and DST-7 respectively. Otherwise, if the sub-TU has the same width as the current CU, DCT-2 is used. For the sub-TUs located in the upper half (or quarter) and the lower half (or quarter) of the current CU, the vertical transforms are DCT-8 and DST-7 respectively. Otherwise, if the sub-TU has the same height as the current CU, DCT-2 is used. (c) MTS is disabled in SPS: When sps_mts_enabled_flag is signaled as true while both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false, for the intra prediction residual, as long as the block width is 4 or more and 16 or less, the horizontal transform is selected as DST-7, and as long as the block height is 4 or more and 16 or less, the vertical transform is selected as DST-7.

[0102] In VVC, a mode-dependent non-separable secondary transform (NSST) can be applied between the forward core transform and quantization (in the encoder) and between the inverse quantization and inverse core transform (in the decoder). To maintain low complexity, NSST is applied only to the low-frequency coefficients after the primary transform. When both the width (W) and height (H) of the transform coefficient block are 8 or more, an 8×8 non-separable secondary transform can be applied to the upper-left 8×8 region of the transform coefficient block. Otherwise, when either W or H of the transform coefficient block is equal to 4, a 4×4 non-separable secondary transform is applied to the upper-left min(8, W)×min(8, H) region of the transform coefficient block, and a 4×4 non-separable transform can be executed. The above transform selection rules can be applied to both the luma and chroma components.

[0103] When using the matrix multiplication implementation of non-separable transformation with a 4×4 input block as an example, it is described as follows in Equation (1). To apply non-separable transformation, the 4×4 input block X:

Number

Number

[0104] The non-separable transformation is

Number

[0105] In one design of NSST, for both 4×4 and 8×8 block sizes, a total of 35×3 non-separable second-order transformations can exist, where 35 is the number of transformation sets defined by the intra prediction modes represented as a set, and 3 is the number of NSST candidates for each intra prediction mode. The mapping from intra prediction modes to transformation sets is defined in Table 5. The transformation set applied to the luma / chroma transformation coefficients is specified by the corresponding luma / chroma intra prediction mode according to Table 5. For intra prediction modes greater than 34 (diagonal prediction direction), the transformation coefficient block is transposed before / after the second-order transformation in the encoder / decoder.

[0106] For each transform set, the selected non-separable secondary transform candidate is further specified by a CU-level NSST index that is explicitly signaled. This index can be signaled in the bitstream once for each CU after the transform coefficients and truncated unary binarization are used. The truncation value is 2 for the planar mode or DC mode, and 3 for the angular intra prediction mode. This NSST index can be signaled only if there are two or more non-zero coefficients within the CU. The default value can be zero if not signaled. A zero value of this syntax element can indicate that no secondary transform is applied to the current CU, and values from 1 to 3 indicate which secondary transform from the set should be applied.

[0107] In some embodiments, for each transform set, the selected non-separable secondary transform candidate can be further specified by a CU-level NSST index that is explicitly signaled. This index can be signaled in the bitstream once for each CU after the transform coefficients and truncated unary binarization are used. The truncation value is 2 for the planar mode or DC mode, and 3 for the angular intra prediction mode. This NSST index can be signaled only if there are two or more non-zero coefficients within the CU. The default value is zero if not signaled. A zero value of this syntax element indicates that no secondary transform is applied to the current CU, and values from 1 to 3 indicate which secondary transform from the set should be applied.

Table 5

[0108] In JVET-N0193, a modification of NSST using a conversion zero-out scheme, i.e., a Reduced Size Transform (RST), also known as Low-Frequency Non-Separable Secondary Transform (LFNST) in VVC Draft 5, has been proposed. JVET-N0193 checks whether the intra prediction mode is planar or DC in order to entropy code the conversion index of NSST. In JVET-N0193, four conversion sets are applied, and each conversion set includes three RST conversion cores. Those three RST conversion cores can be either of size 16×48 (or 16×64) (applied to a conversion coefficient block where both the height and width are 8 or more) or 16×16 (applied to a conversion coefficient block where either the height or the width is equal to 4). For notation convenience, the 16×48 (or 16×64) conversion is denoted as RST8×8, and the 16×16 conversion is denoted as RST4×4. For RST8×8, two alternatives using a 16×64 conversion core and a 16×48 conversion core are shown in FIGS. 10 and 11, respectively. FIG. 10 shows a reduced secondary transform (RST) using a 16×64 secondary conversion core. FIG. 11 shows a reduced secondary transform (RST) using a 16×48 secondary conversion core. In VVC Draft 5, the conversion using a 16×48 conversion core is adopted.

[0109] An index indicating the selection of the LFNST kernel, i.e., lfnst_idx, is signaled at the end of the CU-level syntax as shown in Table 6. Table 6 provides the syntax at the CU level.

Table 6

[0110] In some examples, a Reduced Transform (RT) maps an N-dimensional vector to an R-dimensional vector in a different space, where R / N (R < N) is the reduction factor. The RST matrix is given by Equation (3):

Number

[0111] An RST8×8 with a reduction factor of 4 (1 / 4 size) can be applied. Therefore, instead of the conventional 64×64 non-separable transformation matrix size, a 16×64 direct matrix is used. In other words, on the decoder side, a 64×16 inverse RST matrix is used to generate the core (primary) transformation coefficients in the upper left 8×8 region. The forward RST8×8 uses a 16×64 (or 8×64 for an 8×8 block) matrix so that the forward RST8×8 generates non-zero coefficients only in the upper left 4×4 region within a given 8×8 region. In other words, when RST is applied, the 8×8 region except for the upper left 4×4 region can have only zero coefficients. For RST4×4, a 16×16 (or 8×16 for a 4×4 block) direct matrix multiplication can be applied.

[0112] Also, in RST8×8, instead of using all the upper left 8×8 coefficients as the input for calculating the secondary transformation to further reduce the transformation matrix size, the upper left three 4×4 coefficients are used as the input for calculating the secondary transformation. FIGS. 13A-13B show different alternatives of RST8×8. FIG. 13A shows an example of a 16×64 transformation matrix, and all the upper left 8×8 coefficients are applied as the input for calculating the secondary transformation. FIG. 13B shows an example of a 16×46 transformation matrix, and the upper left three 4×4 coefficients are used as the input for calculating the secondary transformation.

[0113] In some embodiments, the inverse RST can be conditionally applied when two conditions are met: (a) the block size is greater than or equal to a given threshold (W>=4 && H>=4); (b) the transformation skip mode flag is equal to zero.

[0114] If both the width (W) and height (H) of the transform coefficient block are greater than 4, RST8×8 is applied to the upper left 8×8 region of the transform coefficient block. Otherwise, RST4×4 is applied to the upper left min(8, W)×min(8, H) region of the transform coefficient block.

[0115] When the RST index is equal to 0, RST is not applied. Otherwise, when the RST index is equal to 1, RST is applied and its kernel is selected by the RST index.

[0116] Furthermore, RST is applied to both intra and inter-slice intra CUs, and to both luma and chroma. When dual tree is enabled, the RST indices for luma and chroma are signaled separately. For inter-slice (when dual tree is disabled), a single RST index is signaled and used for both luma and chroma components. When the ISP mode is selected, RST is disabled and the RST index is not signaled.

[0117] In some embodiments, the RST matrix can be selected from four transform sets where each transform set consists of two transforms. Which transform set is applied is determined from the intra prediction mode as follows: (a) If one of the three CCLM modes is indicated, transform set 0 is selected, (b) Otherwise, the transform set selection is made according to Table 7.

Table 7

[0118] VVC also includes a matrix-based intra prediction (MIP) mode. To predict the samples of a rectangular block of width W and height H, MIP takes, as input, a column of H reconstructed adjacent boundary samples located to the left of the block and a column of W reconstructed adjacent boundary samples located above the block. If these reconstructed samples are not available, they are generated as they are done in conventional intra prediction.

[0119] The generation of the prediction signal is based on the following three steps: (a) From the boundary samples, 4 samples are extracted by averaging when W = H = 4, and 8 samples are extracted by averaging in all other cases. (b) Using the averaged samples as input, matrix-vector multiplication and subsequent addition of an offset are performed. The result is a reduced prediction signal for the subsampled set of samples within the original block. (c) The prediction signal at the remaining positions is generated by linear interpolation, which is a single-step linear interpolation in each direction, from the prediction signal for the subsampled set.

[0120] The matrices and offset vectors required to generate the prediction signal are taken from three sets of matrices S0, S1, S2. Set S0 consists of 18 matrices A0 i , i ∈ {0, …, 17}, each of these matrices having 16 rows, 4 columns, and 18 offset vectors b0 i , i ∈ {0, …, 17}. The offset vectors b0 i each have a size of 16. The matrices and offset vectors of set S0 are used for 4×4 blocks. Set S1 consists of 10 matrices A1 i , i ∈ {0, …, 9}, each of these matrices having 16 rows, 8 columns, and 10 offset vectors b1 i , i ∈ {0, …, 9}. The offset vectors b1 iEach has a size of 16. The matrices and offset vectors of set S1 are used for blocks of sizes 4×8, 8×4, and 8×8. Finally, set S2 consists of six matrices A2 i , i ∈ {0, …, 5}, where each of these matrices has 64 rows, 8 columns, and six offset vectors b2 i , i ∈ {0, …, 5} of size 64. The matrices and offset vectors of set S2, or parts of these matrices and offset vectors, are used for all other block shapes.

[0121] Figure 14 shows an example of a MIP for an 8×8 block. As shown in Figure 14, given an 8×8 block, the MIP takes four averages along each axis of the boundary. The resulting eight input samples enter a matrix-vector multiplication. Matrices are taken from set S1. This produces 16 samples at the odd positions of the prediction block. Thus, a total of (8·16) / (8·8) = 2 multiplications are performed per sample. After adding the offsets, these samples are interpolated vertically by using the reduced upper boundary. Horizontal interpolation follows by using the original left boundary. In this case, the interpolation process does not require any multiplications.

[0122] Regarding the signaling of the MIP mode, for each intra-mode coding unit (CU), a flag indicating whether the MIP mode is applied to the corresponding prediction unit (PU) is sent in the bitstream. If the MIP mode is applied, the index predmode of the MIP mode is signaled using a MPM list that includes three MPMs.

[0123] Here, the derivation of the MPM is done as follows, using the intra-modes of the upper and left PUs. There are three fixed mapping tables map_angular_to_mip idx , idx ∈ {0, 1, 2}, and each table is given by Equation (4):

Equation

Equation

[0124] To generate the MPM list for the current block encoded by the MIP mode, first, the upper MIP mode, i.e., mode MIP above , and the left MIP mode, i.e., mode MIP left are derived. The value of mode MIP above is derived as follows: (a) If the upper PU, i.e., PU above is available, and belongs to the same CTU as the current PU, and PU above is encoded by MIP using the MIP mode predmode MIP above and idx(PU)=idx(PU above ),

Equation

Equation

Equation

[0125] Finally, given the derived mode MIP above and mode MIP left and three predefined fixed default MPM list list idx , idx ∈ {0, 1, 2} (each of these MPM lists contains three different MIP modes), an MPM list is constructed. The MPM list is constructed by replacing the default values with -1 and removing duplicate MIP modes based on these given default list list idx(PU) and mode MIP above and mode MIP left and.

[0126] The flag for signaling the MIP mode can be shown in Table 8, which is the syntax table at the CU level.

Table 8

[0127] In some embodiments, the MIP mode can be reconciled with the MPM-based coding of the conventional intra prediction mode as follows. The luma and chroma MPM list derivation processes for the conventional intra prediction mode map the MIP mode predmode MIP to one of the conventional intra prediction modes:

Number

[0128] Various methods are provided above, but there are several drawbacks to the above methods. For example, currently, when LFNST is applied, sps_mts_enabled_flag is signaled as true, but both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false. For small block sizes, the primary transform will be selected using an implicit transform scheme as introduced above, which means that DST-7 can always be enabled. However, the LFNST kernel may not operate very efficiently with DST-7.

[0129] In addition, currently, when MIP is applied, sps_mts_enabled_flag is signaled as true, but both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false. For small block sizes, the primary transform will be selected using an implicit transform scheme as introduced above, which means that DST-7 can always be enabled. However, DST-7 may not operate very efficiently in the MIP mode.

[0130] According to embodiments of the present disclosure, methods for improved implicit transform selection are provided. Also, each of those methods (or embodiments), encoders, and decoders can be implemented by a processing circuit (e.g., one or more processors, or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transitory computer-readable medium. In the following disclosure, the term block may be interpreted as a prediction block, a coded block, or a coding unit i.e., CU.

[0131] According to embodiments of the present disclosure, the term NSST may also refer to a reduced secondary transform (RST), which is an alternative design of a non-separable secondary transform, as described, for example, in JVET-M0292 or JVET-N0193, and which may also refer to a low-frequency non-separable secondary transform (LFNST) adopted in VVC draft v5.

[0132] According to embodiments of the present disclosure, DST-7 may also be replaced by DST-4.

[0133] According to embodiments of the present disclosure, "implicit transform" refers to a transform scheme that selects a group of non-DCT2 transforms (e.g., DST-1, DCT-5, DST-7, DCT-8, DST-4, DCT-4, etc.) without transform index signaling. In this regard, the group of non-DCT2 transforms can be selected using already encoded information available to both the encoder and the decoder, including but not limited to intra prediction modes (planar mode, DC mode, angular mode), block size, block width, block height, block aspect ratio, block area size, intra coding mode (whether MRL, ISP, MIP are used), position of selected spatial merge candidates (top merge candidate, left merge candidate), inter prediction modes (inter PDPC mode, CIIP mode, etc.).

[0134] In the following disclosure, "explicit conversion" refers to a conversion scheme that selects one conversion from a group of conversion type candidates (e.g., DCT-2, DST-1, DCT-5, DST-7, DCT-8, DST-4, DCT-4, etc.) using an index that is signaled to indicate which conversion type is selected.

[0135] In a first embodiment, the disclosed method includes obtaining conversion block signaling information from an encoded video bitstream and determining whether an implicit conversion scheme is applied for primary conversion type selection. When an implicit conversion scheme is selected, it means that the flag of sps_mts_enabled_flag is signaled as true, and both the flags of sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false. Also, for a W×H block, when the LFNST index (lfnst_idx) is signaled as 0 (i.e., disabled), and when TSM is not enabled, the primary conversion is selected using the following algorithm: (a) When W>=T1 and W<=T2, the horizontal conversion is DST-7. Otherwise, the horizontal conversion is DCT-2. Examples of values for T1 include 2 pixels, 4 pixels, or 8 pixels. Examples of values for T2 include 4, 8, 16, or 32. (b) When H>=T1 and H<=T2, the vertical conversion is DST-7. Otherwise, the vertical conversion is DCT-2. Examples of values for T1 include 2, 4, or 8. Examples of values for T2 include 4, 8, 16, or 32.

[0136] In contrast, in some embodiments, when the LFNST index is not signaled as 0 (i.e., LFNST is applied), the primary conversion can be selected as follows: (a) In one example, DCT-2 is always selected. (b) In another example, a predetermined transform type other than DCT-7 is selected, such as Hadamard transform, DST-1, DCT-5, Composite Orthogonal Transform (COT), Karhunen–Loeve transform (KLT), etc.

[0137] In addition to VVC draft v5, the changes to the method proposed in the first embodiment are shown below, and the changes are emphasized with underlines.

[0138] In one example, the input to the method disclosed above in the first embodiment is as follows: (a) The top-left sample of the current luma transform block, at the luma position (xTbY, yTbY) defined with respect to the top-left luma sample of the current picture, (b) A variable nTbW defining the width of the current transform block, (c) A variable nTbH defining the height of the current transform block, (d) A variable cIdx defining the color component of the current block, (e) An (nTbW)×(nTbH) array d[x][y] of scaled transform coefficients for x = 0...nTbW−1, y = 0...nTbH−1 The corresponding output of the above method in the first embodiment can be an (nTbW)×(nTbH) array rd[x][y] of residual samples for x = 0...nTbW−1, y = 0...nTbH−1.

[0139] The variable implicitMtsEnabled in the first embodiment can be derived as follows: (a) When sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, implicitMtsEnabled is set equal to 1. (b) IntraSubpartitionsSplitType is not equal to ISP_NO_SPLIT. (c) cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is 32 or less. (d) Both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, and lfnst_idx[x0][y0] is equal to 0 。 (e) Otherwise, implicitMtsEnabled is set equal to 0.

[0140] In the first embodiment, the variable trTypeHor that defines the horizontal transform kernel and the variable trTypeVer that defines the vertical transform kernel can be derived as follows: (a) If cIdx is greater than 0, trTypeHor and trTypeVer are set equal to 0. (b) Otherwise, if implicitMtsEnabled is equal to 1, the following applies: (i) If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, or both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, trTypeHor and trTypeVer are derived as follows: trTypeHor=(nTbW>=4 && nTbW<=16)? 1:0 trTypeVer=(nTbH>=4 && nTbH<=16)? 1:0 (ii) Otherwise (when cu_sbt_flag is equal to 1), trTypeHor and trTypeVer are defined in Table 10 below according to cu_sbt_horizontal_flag and cu_sbt_pos_flag. (iii) Otherwise, trTypeHor and trTypeVer are defined in Table 9 below according to tu_mts_idx[xTbY][yTbY].

[0141] Tables 9 - 10 give the specifications of trTypeHor and trTypeVer related to the method in the first embodiment. [Table 9] [Table 10]

[0142] In the second embodiment, when an implicit conversion scheme is applied to the primary conversion type selection (e.g., sps_mts_enabled_flag is signaled as true, but both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false), for a W×H block having an MIP flag (intr_mip_flag) signaled as 0 (i.e., MIP is not applied), if TSM is not enabled, the primary conversion can be selected using the following algorithm: (a) When W >= T1 and W <= T2, the horizontal conversion is DST - 7. Otherwise, the horizontal conversion is DCT - 2. Examples of values for T1 include 2, 4, or 8. Examples of values for T2 include 4, 8, 16, or 32. (b) When H >= T1 and H <= T2, the vertical conversion is DST - 7. Otherwise, the vertical conversion is DCT - 2. Examples of values for T1 include 2, 4, or 8. Examples of values for T2 include 4, 8, 16, or 32.

[0143] When the MIP flag is not signaled as 0 (i.e., MIP is applied), the primary conversion uses DCT - 2. In another example, the primary conversion uses a non - DST - 7 such as, for example, Hadamard transform, DST - 1, DCT - 5, COT, KLT, etc.

[0144] Alternatively, when the MIP flag (intra_mip_flag) is not equal to 0 (i.e., MIP is applied), the implicit conversion scheme may still be applied. However, the threshold values T1 and T2 may be different from those used for implicit conversion as introduced above. In one example, T1 is equal to 2 and T2 is equal to 4 or 8. In another example, T1 is equal to 4 and T2 is equal to 4 or 8. In yet another example, T1 is equal to 8 and T2 is equal to 8, 16, or 32. T1 may also be equal to 16 and T2 is equal to 16 or 32.

[0145] An example of a change to the method proposed in the second embodiment in addition to VVC draft v5 is shown below, with the changes emphasized in underline.

[0146] The input to the method in the second embodiment is as follows: (a) The top-left sample of the current luma conversion block, at the luma position (xTbY, yTbY) defined with respect to the top-left luma sample of the current picture, (b) A variable nTbW defining the width of the current conversion block, (c) A variable nTbH defining the height of the current conversion block, (d) A variable cIdx defining the color component of the current block, (e) An (nTbW)×(nTbH) array d[x][y] of scaled conversion coefficients for x = 0...nTbW−1, y = 0...nTbH−1 The output of this process can be an (nTbW)×(nTbH) array rd[x][y] of residual samples for x = 0...nTbW−1, y = 0...nTbH−1.

[0147] The variable implicitMtsEnabled is derived as follows: (a) When sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, implicitMtsEnabled is set equal to 1. (b) IntraSubpartitionsSplitType is not equal to ISP_NO_SPLIT. (c) The cu_sbt_flag is equal to 1, and Max(nTbW, nTbH) is 32 or less. (d) Both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, and intra_mip_flag[x0][y0] is equal to 0. (e) In other cases, implicitMtsEnabled is set to 0.

[0148] In the second embodiment, the variable trTypeHor that defines the horizontal transform kernel and the variable trTypeVer that defines the vertical transform kernel can be derived as follows: (a) If cIdx is greater than 0, trTypeHor and trTypeVer are set to 0. (b) In other cases, when implicitMtsEnabled is equal to 1, the following applies: (i) If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, or both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, trTypeHor and trTypeVer are derived as follows: trTypeHor = (nTbW >= 4 && nTbW <= 16)? 1 : 0 trTypeVer = (nTbH >= 4 && nTbH <= 16)? 1 : 0 (ii) In other cases (when cu_sbt_flag is equal to 1), trTypeHor and trTypeVer are defined in Table 10 according to cu_sbt_horizontal_flag and cu_sbt_pos_flag. (iii) In other cases, trTypeHor and trTypeVer are defined in Table 9 according to tu_mts_idx[xTbY][yTbY].

[0149] In the third embodiment, the first embodiment and the second embodiment can be combined. An example of the specification change of the proposed method in addition to VVC draft v5 is shown below, and the changes are emphasized with underlines.

[0150] The input to the method in the third embodiment is as follows: (a) The top - left sample of the current luma transform block, at the luma position (xTbY, yTbY) defined with respect to the top - left luma sample of the current picture. (b) The variable nTbW that defines the width of the current transform block. (c) The variable nTbH that defines the height of the current transform block. (d) The variable cIdx that defines the color component of the current block. (e) The (nTbW)×(nTbH) array d[x][y] of scaled transform coefficients for x = 0...nTbW - 1, y = 0...nTbH - 1. The output of this process can be the (nTbW)×(nTbH) array rd[x][y] of residual samples for x = 0...nTbW - 1, y = 0...nTbH - 1.

[0151] The variable implicitMtsEnabled in the third embodiment can be derived as follows: (a) When sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, implicitMtsEnabled is set equal to 1. (b) IntraSubpartitionsSplitType is not equal to ISP_NO_SPLIT. (c) cu_sbt_flag is equal to 1 and Max(nTbW,nTbH) is 32 or less. (d) Both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, and lfnst_idx[x0][y0] is equal to 0, and intra_mip_flag[x0][y0] is equal to 0. (e) Otherwise, implicitMtsEnabled is set equal to 0.

[0152] In the third embodiment, the variable trTypeHor defining the horizontal transform kernel and the variable trTypeVer defining the vertical transform kernel are derived as follows: (a) When cIdx is greater than 0, trTypeHor and trTypeVer are set equal to 0. (b) Otherwise, when implicitMtsEnabled is equal to 1, the following applies: (i) When IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, or both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, trTypeHor and trTypeVer are derived as follows: trTypeHor=(nTbW>=4 && nTbW<=16)? 1:0 trTypeVer=(nTbH>=4 && nTbH<=16)? 1:0 (ii) Otherwise (when cu_sbt_flag is equal to 1), trTypeHor and trTypeVer are defined in Table 10 according to cu_sbt_horizontal_flag and cu_sbt_pos_flag. (iii) In other cases, trTypeHor and trTypeVer are defined in Table 9 according to tu_mts_idx[xTbY][yTbY].

[0153] In the fourth embodiment of the present disclosure, a combination of implicit conversion and explicit conversion signaling can be applied. Regarding horizontal (or vertical) conversion, either implicit conversion or explicit conversion can be used.

[0154] In one example, for a W×H block, when W is greater than or equal to T1 and W is less than or equal to T2, the horizontal conversion type is DST-7. Otherwise, when W is greater than T2 and less than or equal to T3, the horizontal conversion type can be either DCT-2 or DST-7, and the selection is signaled. Otherwise, when W is greater than T3 or less than T1, a default conversion type such as DCT-2 is applied. Examples of values for T1 include 2, 4, or 8. Examples of values for T2 include 4, 8, 16, or 32. Examples of values for T3 include 8, 16, 32, or 64. The combined setting of T1, T2, and T3 can be such that T1 is equal to 4 (or 2), T2 is equal to 16, and T3 is equal to 32 (or 64).

[0155] In another example, for a W×H block, when H is greater than or equal to T1 and H is less than or equal to T2, the vertical conversion type is DST-7. Otherwise, when H is greater than T2 and less than or equal to T3, the vertical conversion type can be either DCT-2 or DST-7, and the selection is signaled. Otherwise, when H is greater than T3 or less than T1, a default conversion type such as DCT-2 is applied. Examples of values for T1 include 2, 4, or 8. Examples of values for T2 include 4, 8, 16, or 32. Examples of values for T3 include 8, 16, 32, or 64. The combined setting of T1, T2, and T3 can be such that T1 is equal to 4 (or 2), T2 is equal to 16, and T3 is equal to 32 (or 64).

[0156] FIG. 15 shows a flowchart outlining a process (1500) according to an embodiment of the present disclosure. The process (1500) can be used for the reconstruction of blocks encoded in the intra mode and can thus be used to generate prediction blocks for blocks being reconstructed. In various embodiments, the process (1500) can be executed by a processing circuit, such as, for example, the processing circuits of terminal devices (110), (120), (130), and (140), the processing circuit executing the functions of video encoder (203), the processing circuit executing the functions of video decoder (210), the processing circuit executing the functions of video decoder (310), the processing circuit executing the functions of video encoder (403), and the like. In some embodiments, the process (1500) can be implemented by software instructions, such that when the processing circuit executes the software instructions, the processing circuit executes the process (1500). The process starts at (S1501) and proceeds to (S1510).

[0157] (S1510), conversion block signaling information is obtained from the encoded video bitstream. The conversion block signaling information can include at least one of a flag of sps_mts_enabled_flag, a flag of sps_explicit_mts_intra_enabled_flag, or a flag of the transform skip mode (TSM), or the like. The conversion block signaling information can also include an LFNST index (lfnst_idx) or an intra MIP flag (intra_mip_flag). When the flag of sps_mts_enabled_flag is signaled as true while the flags of both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false, an implicit conversion scheme is applied to the primary conversion type selection.

[0158] At (S1520), a decoder may determine whether the transform block signaling information indicates an implicit transform scheme and whether there is a problem with at least one of low-frequency non-separable transform (LFNST) and matrix-based intra prediction mode (MIP). If the implicit transform scheme is applied and there is a problem with at least one of low-frequency non-separable transform (LFNST) and matrix-based intra prediction mode (MIP), process 1500 proceeds to (S1530).

[0159] At (S1530), in response to the determination that the transform block signaling information indicates an implicit transform scheme and that at least one of LFNST and MIP is signaled as having a problem, a primary transform type is determined based on the size of the coded block unit (CU). Then, process 1500 proceeds to (S1540), where a primary transform is performed on the transform block split from the CU according to the determined primary transform type.

[0160] At (S1520), if the determination is that the implicit transform scheme is applied but at least one of low-frequency non-separable transform (LFNST) and matrix-based intra prediction mode (MIP) is true, process 1500 proceeds to (S1550), where, in response to this determination, a transform type other than DCT-2 or DCT-7 may be selected. Transform types other than DCT-7 may include Hadamard transform, DST-1, DCT-5, COT, and KLT.

[0161] The above technology can be implemented as computer software using computer-readable instructions physically stored on one or more computer-readable media. For example, FIG. 16 shows a computer system (1600) suitable for implementing a particular embodiment of the disclosed matter.

[0162] Computer software can be coded using any suitable machine code or computer language such that, when subjected to assembly, compilation, linking, or similar mechanisms, it can produce code having instructions that can be executed directly or via interpretation, microcode execution, and the like, by one or more computer central processing units (CPUs), graphics processing units (GPUs), and the like.

[0163] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0164] The components shown in FIG. 16 with respect to computer system (1600) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. Also, the configuration of the components should not be construed as having any dependency or requirement with respect to any one or combination of the components shown in this exemplary embodiment of computer system (1600).

[0165] Computer system (1600) may include a specific human interface input device. Such a human interface input device can respond to input by one or more human users via, for example, tactile input (e.g., keystrokes, swipes, moving a data glove, etc.), audio input (e.g., voice, clapping, etc.), visual input (e.g., gestures, etc.), olfactory input (not shown). The human interface device may also be used to capture certain media that are not necessarily directly related to conscious input by humans, such as, for example, audio (e.g., conversation, music, ambient sound, etc.), images (e.g., scanned images, photographic images obtained from a still camera, etc.), video (e.g., two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0166] The input human interface device may include one or more of a keyboard (1601), a mouse (1602), a trackpad (1603), a touch screen (1610), a data glove (not shown), a joystick (1605), a microphone (1606), a scanner (1607), and a camera (1608) (only one of each is shown).

[0167] The computer system (1600) may also include certain human interface output devices. Such human interface output devices can stimulate the senses of one or more human users, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices include tactile output devices (e.g., tactile feedback by a touch screen (1610), a data glove (not shown), or a joystick (1605), although there may also be a tactile feedback device that does not function as an input device), audio output devices (e.g., speakers (1609), headphones (not shown), etc.), visual output devices (e.g., a screen (1610) including a CRT screen, an LCD screen, a plasma screen, an OLED screen (each may or may not have a touch screen input function, each may or may not have a tactile feedback function. Some of these can output four-dimensional or higher-dimensional output through means such as two-dimensional visual output or, for example, stereoscopic output)), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), etc.), and a printer (not shown).

[0168] The computer system (1600) may also include human-accessible storage devices and their associated media such as, for example, an optical medium including a CD / DVD ROM / RW (1620) having a CD / DVD or similar medium (1621), a thumb drive (1622), a removable hard drive or solid state drive (1623), legacy magnetic media such as tapes and floppy disks (trademark, not shown), specialized ROM / ASIC / PLD-based devices such as security dongles (not shown), and the like.

[0169] Those skilled in the art will also understand that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include a transmission medium, a carrier wave, or other transient signals.

[0170] The computer system (1600) may also include an interface to one or more communication networks. The network can be, for example, wireless, wired, or optical. The network can further be local, wide area, metropolitan, vehicle and industrial, real-time, delay-tolerant, etc. Examples of networks include, for example, local area networks such as Ethernet (registered trademark), wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE and the like, and TV wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, and vehicle and industrial including CANBus and the like. A particular network generally requires an external network interface adapter attached to a particular general-purpose data port or peripheral bus (1649) (e.g., a USB port of the computer system (1600)), and others are generally integrated into the core of the computer system (1600) by attachment to the system bus described later (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using any of these networks, the computer system (1600) can communicate with other entities. Such communication may be only unidirectional reception (e.g., broadcast TV), only unidirectional transmission (e.g., CANbus to a specific CANbus device), or bidirectional, for example, to another computer system using a local or wide area digital network. Particular protocols and protocol stacks can be used on each of the networks and network interfaces as described above.

[0171] The aforementioned human interface device, human-accessible storage device, and network interface can be attached to the core (1640) of the computer system (1600).

[0172] The core (1640) may include one or more central processing units (CPUs) (1641), graphics processing units (GPUs) (1642), field programmable gate arrays (FPGAs) (1643) in the form of special programmable processing units, hardware accelerators (1644) for specific tasks, and the like. These devices can be connected via a system bus (1648) together with read-only memory (ROM) (1645), random access memory (1646), internal mass storage such as an internal hard drive, SSD, etc. (1647) that is not accessible to internal users, and the like (1647). In some computer systems, the system bus (1648) can be made accessible in the form of one or more physical plugs to allow for expansion by additional CPUs, GPUs, and the like. Peripheral devices may be attached either directly to the system bus (1648) of the core or via a peripheral bus (1649). Peripheral bus architectures include PCI, USB, and the like.

[0173] The CPU (1641), GPU (1642), FPGA (1643), and accelerator (1644) can execute specific instructions that can be combined to form the aforementioned computer code. The computer code can be stored in the ROM (1645) or RAM (1646). Transient data can also be stored in the RAM (1646), and permanent data can be stored, for example, in the internal mass storage (1647). Fast storage and retrieval to any of the memory devices can be enabled by the use of cache memory that may be associated near one or more CPUs (1641), GPUs (1642), mass storage (1647), ROM (1645), RAM (1646), and the like.

[0174] A computer-readable medium can have computer code thereon for performing various computer-implemented processes. The medium and the computer code may be specially designed and constructed for the purposes of this disclosure, or they may be of the kind well known and available to those having skill in the computer software arts.

[0175] As an example, and not by way of limitation, a computer system having an architecture (1600), particularly a core (1640), can provide functionality as a result of software embodied on one or more tangible computer-readable media being executed by one or more processors (including CPUs, GPUs, FPGAs, accelerators, and the like). Such computer-readable media can be, for example, specific storage of the core (1640) that is non-transitory in nature, such as a mass storage (1647) inside the core or a ROM (1645), and media associated with user-accessible mass storage as introduced above. The software implementing various embodiments of the present disclosure can be stored in such an apparatus and executed by the core (1640). The computer-readable media can include one or more memory devices or chips according to specific needs. The software can cause the core (1640) and particularly the processors (including CPUs, GPUs, FPGAs, and the like) therein to execute the specific processes described herein or specific portions of the specific processes, including defining data structures stored in the RAM (1646) and modifying such data structures according to processes defined by the software. Additionally, or alternatively, the computer system can provide functionality as a result of logic wired or otherwise embodied in a circuit (e.g., an accelerator (1644)) that operates instead of or in conjunction with software to execute the specific processes described herein or specific portions of the specific processes. References to software include logic, and vice versa where appropriate. References to computer-readable media can include a circuit (e.g., an integrated circuit (IC), etc.) storing software for execution, a circuit embodying logic for execution, or both where appropriate. The present disclosure includes suitable combinations of hardware and software. Appendix A: Acronyms JEM: joint exploration model VVC: Versatile Video Coding (Versatile Video Coding) BMS: Benchmark Set (Benchmark Set) MV: Motion Vector (Motion Vector) HEVC: High Efficiency Video Coding (High Efficiency Video Coding) SEI: Supplementary Enhancement Information (Supplementary Enhancement Information) VUI: Video Usability Information (Video Usability Information) GOPs: Groups of Pictures (Groups of Pictures) TUs: Transform Units, (Transform Units) PUs: Prediction Units (Prediction Units) CTUs: Coding Tree Units (Coding Tree Units) CTBs: Coding Tree Blocks (Coding Tree Blocks) PBs: Prediction Blocks (Prediction Blocks) HRD: Hypothetical Reference Decoder (Hypothetical Reference Decoder) SNR: Signal Noise Ratio (Signal Noise Ratio) CPUs: Central Processing Units (Central Processing Units) GPUs: Graphics Processing Units (Graphics Processing Units) CRT: Cathode Ray Tube (Cathode Ray Tube) LCD: Liquid-Crystal Display (Liquid-Crystal Display) OLED: Organic Light-Emitting Diode (Organic Light-Emitting Diode) CD: Compact Disc (Compact Disc) DVD: Digital Video Disc (Digital Video Disk) ROM: Read-Only Memory (Read-Only Memory) RAM: Random Access Memory (Random Access Memory) ASIC: Application-Specific Integrated Circuit (Application-Specific Integrated Circuit) PLD: Programmable Logic Device (Programmable Logic Device) LAN: Local Area Network (Local Area Network) GSM: Global System for Mobile communications (Global System for Mobile Communications) LTE: Long-Term Evolution (Long-Term Evolution) CANBus: Controller Area Network Bus (Controller Area Network Bus) USB: Universal Serial Bus (Universal Serial Bus) PCI: Peripheral Component Interconnect (Peripheral Component Interconnect) FPGA: Field Programmable Gate Areas (Field Programmable Gate Array) SSD: solid-state drive (Solid State Drive) IC: Integrated Circuit (Integrated Circuit) CU: Coding Unit (Coding Unit)

[0176] Although this disclosure describes several exemplary embodiments, there are changes, substitutions, and various equivalent alternatives that fall within the scope of the disclosure. Accordingly, it is understood that those skilled in the art, although not explicitly illustrated or described herein, can embody the principles of the disclosure and, therefore, devise numerous systems and methods that are within its spirit and scope.

Claims

1. 1. A method for encoding a video bitstream executed by at least one processor, comprising: determining, for a coding unit (CU), whether (i) an implicit transform scheme is enabled, and (ii) whether at least one of a low frequency non-separable transform (LFNST) and a matrix-based intra prediction mode (MIP) is disabled; determining a primary transformation type based on a size of the CU in response to determining that an implicit transformation scheme is enabled and at least one of the LFNST and the MIP is invalid; performing a primary transformation on a transform block divided from the CU according to the determined primary transformation type; generating an encoded bitstream including transform block signaling information indicating whether the implicit transform scheme is enabled and at least one of information indicating whether the LFNST is disabled and information indicating whether the MIP is disabled; The method according to claim 1,

2. The step of determining a primary transformation type comprises: Determines whether conversion skip mode is enabled, and In response to determining that the transform skip mode is enabled, (i) determining a transform type DST-7 for a horizontal transform on the transform block in response to a width of the CU being greater than or equal to T1 and less than or equal to T2; (ii) determining a transform type DCT-2 for the horizontal transform on the transform block in response to the width of the CU being smaller than T1 or larger than T2; (iii) determining a transform type DST-7 for a vertical transform for the transform block in response to a height of the CU being greater than or equal to T1 and less than or equal to T2; (iv) determining a transform type DCT-2 for the vertical transform for the transform block in response to the height of the CU being smaller than T1 or larger than T2; The method of claim 1 comprising:

3. 3. The method of claim 2, wherein T1 is equal to one of 2 pixels, 4 pixels, or 8 pixels, and T2 is equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels.

4. In response to determining that the implicit transformation scheme is enabled and the at least one of the LFNST or the MIP is valid, the method includes: (i) determining a first transform type DCT-2 for the transform block; and (ii) determining a second transform type for the transform block that is not DCT-7, the second transform type including at least one of a Hadamard transform, a DST-1, a DCT-5, a complex orthonormal transform (COT), or a Karhunen-Loeve transform; The method of claim 1 , comprising at least one of:

5. In response to determining that the implicit transformation scheme is enabled and the MIP is disabled indicating that the MIP is not applied to the transformation block, the method includes: (i) determining a transform type DST-7 for a horizontal transform on the transform block in response to a width of the CU being greater than or equal to T1 and less than or equal to T2; (ii) determining a transform type DCT-2 for the horizontal transform on the transform block in response to the width of the CU being smaller than T1 or larger than T2; (iii) determining a transform type DST-7 for a vertical transform for the transform block in response to a height of the CU being greater than or equal to T1 and less than or equal to T2; and (iv) determining a transform type DCT-2 for the vertical transform on the transform block in response to the height of the CU being smaller than T1 or larger than T2; The method of claim 1 , comprising at least one of:

6. The T1 and the T2 are T1 is equal to 2 pixels and T2 is equal to one of 4 pixels or 8 pixels; T1 is equal to 4 pixels, and T2 is equal to one of 4 pixels or 8 pixels; T1 is equal to 8 pixels, and T2 is equal to one of 8 pixels, 16 pixels, or 32 pixels; and T1 is equal to 16 pixels, and T2 is equal to one of 16 pixels or 32 pixels; The method of claim 5 , wherein the at least one of

7. In response to determining that the implicit transformation scheme is enabled and both the LFNST and the MIP are invalid indicating that neither the LFNST nor the MIP is applied to the transformation block, the method includes: (i) determining a transform type DST-7 for a horizontal transform on the transform block in response to a width of the CU being greater than or equal to T1 and less than or equal to T2; (ii) determining a transform type DCT-2 for the horizontal transform on the transform block in response to the width of the CU being smaller than T1 or larger than T2; (iii) determining a transform type DST-7 for a vertical transform for the transform block in response to a height of the CU being greater than or equal to T1 and less than or equal to T2; and (iv) determining a transform type DCT-2 for the vertical transform on the transform block in response to the height of the CU being smaller than T1 or larger than T2; The method of claim 1 , comprising at least one of:

8. 1. A method for encoding a video bitstream executed by at least one processor, comprising: determining a primary transform type from a plurality of transform types based on a coding unit (CU) size; In response to the size of the CU being within a first range, the primary transform type is determined as a first primary transform type; In response to the size of the CU being within a second range, the primary transform type is determined to be one of the first primary transform type or a second primary transform type; determining, in response to the size of the CU being within a third range, that the primary transform type is determined as the second primary transform type; performing a primary transformation on a transform block divided from the CU according to the determined primary transformation type; generating an encoded bitstream including an index indicating the first primary transform type or the second primary transform type if the size of the CU is within the second range; The method according to claim 1,

9. The determining step includes: (i) determining, in response to the size of the CU being within the first range indicating a width of the CU being greater than or equal to T1 and less than or equal to T2, that the primary transform type is DST-7 for a horizontal transform on the transform block; (ii) determining, in response to the size of the CU being within the second range indicating that the width of the CU is greater than T2 and less than or equal to T3, that the primary transform type is one of DST-7 and DCT-2 for the horizontal transform of the transform block; (iii) determining, in response to the size of the CU being within the third range, indicating that the width of the CU is less than T1 or greater than T3, that the primary transform type is DCT-2 for the horizontal transform on the transform block; The method of claim 8 , comprising at least one of:

10. The determining step includes: (i) determining, in response to the size of the CU being within the first range indicating a height of the CU being greater than or equal to T1 and less than or equal to T2, that the primary transform type is DST-7 for a vertical transform for the transform block; (ii) determining, in response to the size of the CU being within the second range indicating that the height of the CU is greater than T2 and less than or equal to T3, that the primary transform type is one of DST-7 and DCT-2 for the vertical transform of the transform block; (iii) determining, in response to the size of the CU being within the third range, indicating that the height of the CU is less than T1 or greater than T3, that the primary transform type is DCT-2 for the vertical transform for the transform block; The method of claim 9 , comprising at least one of:

11. 11. The method of claim 10, wherein T1 is equal to one of 2 pixels, 4 pixels, or 8 pixels, T2 is equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels, and T3 is equal to one of 8 pixels, 16 pixels, 32 pixels, or 64 pixels.

12. 1. An apparatus for video encoding, comprising: At least one memory storing a program; At least one processor; having The program causes the at least one processor to carry out a method according to any one of claims 1 to 11. Device.

Images