Video coding method, apparatus, and program

Abstract

To provide a method and apparatus for video coding based on an improved implicit transform approach.SOLUTION: In a method, transform block signaling information is acquired from a coded video bitstream. In addition, a determination is made to determine whether the transform block signaling information indicates an implicit transform scheme, and at least one of a low-frequency non-separable transform (LFNST) and a matrix-based intra predication mode (MIP) is invalid. In response to the determination that the transform block signaling information indicates the implicit transform scheme, and at least one of the LFNST and MIP is signaled as invalid, a primary transform type is determined based on a size of a coding block unit (CU), and a primary transform is performed for a transform block that is partitioned from the CU in accordance with the determined primary transform type.SELECTED DRAWING: Figure 15

Description

[Technical Field]

[0001] This application claims the benefit of priority to U.S. Provisional Application No. 62 / 858,887, entitled "IMPLICIT TRANSFORM SELECTION," filed June 7, 2019, which claims the benefit of priority to U.S. Patent Application No. 16 / 883,545, entitled "METHOD AND APPARATUS FOR IMPROVED IMPLICIT TRANSFORM SELECTION," filed May 26, 2020. The entire disclosures of these prior applications are incorporated herein by reference in their entirety.

[0002] This disclosure presents a set of advanced video coding techniques. More specifically, an improved implicit transformation approach is proposed. [Background technology]

[0003] The background discussion provided herein is intended to provide a general overview of the context for the disclosure. To the extent described in this background section, the work of the named inventors, and aspects of the disclosure that may not otherwise qualify as prior art at the time of filing, are not admitted, explicitly or implicitly, as prior art to the present disclosure.

[0004] Video encoding and decoding can be performed using inter-picture prediction with motion compensation. Uncompressed digital video includes a series of pictures, each with spatial dimensions of, for example, 1920 x 1080 luminance (luminance) samples and associated color (chrominance) samples. The series of pictures can have a fixed or variable picture rate (informally known as frame rate), for example, 60 pictures per second, i.e., a 60 Hz picture rate. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (1920 x 1080 luminance sample resolution at a 60 Hz frame rate) requires a bandwidth approaching 1.5 Gbit / s. One hour of such video requires more than 600 gigabytes (GB) of storage space.

[0005] One goal of video encoding and decoding can be reducing redundancy in the input video signal through compression. Compression can help reduce the aforementioned bandwidth or storage space requirements, sometimes by more than two orders of magnitude. Both lossless and lossy compression, as well as combinations of these, can be used. Lossless compression refers to techniques that allow an exact replica of the original signal to be reconstructed from a compressed version of the original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signal is small enough to make the reconstructed signal useful for its intended application. For video, lossy compression is widely used. The amount of acceptable distortion depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television distribution applications. The achievable compression ratio reflects this, and higher acceptable / tolerable distortion can result in a higher compression ratio.

[0006] Video encoders and decoders can utilize techniques from several broad categories, including, for example, motion compensation, transform, quantization, and entropy coding.

[0007] Video codec technology can include a technique known as intra-coding. In intra-coding, sample values are represented without reference to samples or other data from a previously reconstructed reference picture. In some video codecs, a picture is spatially subdivided into multiple block samples. When all block samples are coded in intra mode, the picture can be an intra-picture. Intra-pictures and their derivatives, such as independent decoder refresh pictures, can be used to reset the decoder state and can therefore be used as the first picture in a coded video bitstream and video session, or as a still image. Samples of an intra-block can be subjected to a transform, and the transform coefficients can be quantized before entropy coding. Intra-prediction can be a technique that minimizes sample values in the pre-transform domain. In some cases, the smaller the DC value after the transform and the smaller the AC coefficients, the fewer bits are required for a given quantization step size to represent the block after entropy coding.

[0008] Traditional intra-coding, such as that known from MPEG-2 generation encoding techniques, does not use intra-prediction. However, some newer video compression techniques include techniques that rely on surrounding sample data and / or metadata obtained during the encoding / decoding of data of spatially adjacent and preceding blocks in decoding order. Such techniques are hereinafter referred to as "intra-prediction" techniques. Note that, at least in some cases, intra-prediction uses only reference data from the current picture being reconstructed, and not from reference pictures.

[0009] There may be many different forms of intra-prediction. If two or more of such techniques can be used in a given video coding technique, the technique in use may be coded as an intra-prediction mode. In certain cases, a mode may have sub-modes and / or parameters, which may be coded separately or included in a mode codeword. Which codeword is used for a given mode / sub-mode / parameter combination may affect the coding efficiency gain through intra-prediction, as may the entropy coding technique used to convert the codeword into a bitstream.

[0010] Certain intra-prediction modes were introduced in H.264, improved in H.265, and further improved in newer coding techniques, such as the Joint Search Model (JEM), Versatile Video Coding (VVC), and Benchmark Set (BMS). A predictor block can be formed using neighboring sample values belonging to already available samples. The sample values of the neighboring samples are replicated in the predictor block according to the direction. A reference to the direction in use can be coded into the bitstream, or it can be predicted itself. Summary of the Invention

[0011] Aspects of the present disclosure provide a method and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes a receiving circuit and a processing circuit.

[0012] According to one aspect of the present disclosure, a video decoding method for a decoder is provided. The method includes obtaining transform block signaling information from a coded video bitstream. Furthermore, a determination is made to determine whether the transform block signaling information indicates an implicit transform scheme and whether at least one of a low-frequency non-separable transform (LFNST) and a matrix-based intra prediction mode (MIP) is disabled. In response to determining that the transform block signaling information indicates the implicit transform scheme and at least one of the LFNST and the MIP is disabled, a primary transform type is determined based on a size of a coding block unit (CU), and a primary transform is performed on transform blocks divided from the CU according to the determined primary transform type.

[0013] In some embodiments, to determine the primary transform type, a determination may be made to determine whether transform skip mode is enabled. In response to determining that transform skip mode is enabled, a transform type DST-7 may be determined for the horizontal transform for the transform block depending on whether the width of the CU is greater than or equal to T1 and less than or equal to T2. A transform type DCT-2 may be determined for the horizontal transform for the transform block depending on whether the width of the CU is less than T1 or greater than T2. A transform type DST-7 may be determined for the vertical transform for the transform block depending on whether the height of the CU is greater than or equal to T1 and less than or equal to T2. A transform type DCT-2 may be determined for the vertical transform for the transform block depending on whether the height of the CU is less than T1 or greater than T2.

[0014] In some embodiments, T1 may be equal to one of 2 pixels, 4 pixels, or 8 pixels, and T2 is equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels.

[0015] In response to determining that the transform block signaling information indicates an implicit transform scheme and that at least one of LFNST or MIP is signaled as true, in a first example, a first transform type DCT-2 may be determined for the transform block. In a second example, a second transform type other than DCT-7 may be determined for the transform block, the second transform type including at least one of DST-1, DCT-5, compound orthonormal transform (COT), or Karhunen-Loeve transform.

[0016] In response to determining that the transform block signaling information indicates an implicit transform scheme and MIP is signaled as false, indicating that MIP is not applied to the transform block, a transform type DST-7 may be determined for the horizontal transform for the transform block depending on whether the width of the CU is greater than or equal to T1 and less than or equal to T2. A transform type DCT-2 may be determined for the horizontal transform for the transform block depending on whether the width of the CU is less than T1 or greater than T2. A transform type DST-7 may be determined for the vertical transform for the transform block depending on whether the height of the CU is greater than or equal to T1 and less than or equal to T2. A transform type DCT-2 may be determined for the vertical transform for the transform block depending on whether the height of the CU is less than T1 or greater than T2.

[0017] In a first example, T1 may be equal to 2 pixels and T2 may be equal to one of 4 pixels or 8 pixels. In a second example, T1 may be equal to 4 pixels and T2 may be equal to one of 4 pixels or 8 pixels. In a third example, T1 may be equal to 8 pixels and T2 may be equal to one of 8 pixels, 16 pixels, or 32 pixels. In a fourth example, T1 may be equal to 16 pixels and T2 may be equal to one of 16 pixels or 32 pixels.

[0018] In some embodiments, in response to determining that the transform block signaling information indicates an implicit transform scheme and both LFNST and MIP are signaled as false, indicating that neither LFNST nor MIP is applied to the transform block, a transform type DST-7 may be determined for the horizontal transform for the transform block in response to the width of the CU being greater than or equal to T1 and less than or equal to T2. A transform type DCT-2 may be determined for the horizontal transform for the transform block in response to the width of the CU being less than T1 or greater than T2. A transform type DST-7 may be determined for the vertical transform for the transform block in response to the height of the CU being greater than or equal to T1 and less than or equal to T2. A transform type DCT-2 may be determined for the vertical transform for the transform block in response to the height of the CU being less than T1 or greater than T2.

[0019] According to another aspect of the present disclosure, there is provided a video decoding method for a decoder, in which transform block signaling information is obtained from a coded video bitstream, a primary transform type is determined based on the transform block signaling information and a size of a coding block unit (CU), and a primary transform is performed on transform blocks divided from the CU according to the determined primary transform type.

[0020] In some embodiments, to determine the primary transform type, a transform type DST-7 may be determined for the horizontal transform on the transform block depending on whether the width of the CU is greater than or equal to T1 and less than or equal to T2. A transform type may be determined for the horizontal transform of the transform block based on a signaled index from the transform block signaling information depending on whether the width of the CU is greater than T2 and less than or equal to T3. The signaled index indicates that the transform type is one of DCT-2 or DST-7. A transform type DCT-2 may be determined for the horizontal transform on the transform block depending on whether the width of the CU is less than T1 or greater than T3.

[0021] In some embodiments, to determine the primary transform type, a transform type DST-7 may be determined for the vertical transform of the transform block depending on whether the height of the CU is greater than or equal to T1 and less than or equal to T2. A transform type may be determined for the vertical transform of the transform block based on a signaled index from the transform block signaling information depending on whether the height of the CU is greater than T2 and less than or equal to T3. The signaled index indicates that the transform type is one of DCT-2 or DST-7. A transform type DCT-2 may be determined for the vertical transform of the transform block depending on whether the height of the CU is less than T1 or greater than T3.

[0022] In some embodiments, T1 may be equal to one of 2 pixels, 4 pixels, or 8 pixels, T2 may be equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels, and T3 may be equal to one of 8 pixels, 16 pixels, 32 pixels, or 64 pixels.

[0023] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform a method for video decoding. [Brief explanation of the drawings]

[0024] Further features, nature and various advantages of the disclosed subject matter will become more apparent from the following detailed description and the accompanying drawings. [Figure 1] 1 illustrates generally a simplified block diagram of a communication system (100) according to one embodiment. [Figure 2] 2 illustrates generally a simplified block diagram of a communication system (200) according to one embodiment. [Figure 3] 2A and 2B illustrate a simplified block diagram of a decoder according to one embodiment; [Figure 4] 1 illustrates schematically a simplified block diagram of an encoder according to one embodiment; [Figure 5] 10 shows a block diagram of an encoder according to another embodiment. [Figure 6] 10 shows a block diagram of a decoder according to another embodiment. [Figure 7A] 7A-7D show four exemplary sub-block transform modes. [Figure 7B] 7A-7D show four exemplary sub-block transform modes. [Figure 7C] 7A-7D show four exemplary sub-block transform modes. [Figure 7D] 7A-7D show four exemplary sub-block transform modes. [Figure 8] 1 illustrates a first exemplary partitioning of a luma intra prediction block based on an intra sub-partition (ISP) coding mode. [Figure 9] 10 illustrates a second exemplary partitioning of a luma intra prediction block based on an intra sub-partition (ISP) coding mode. [Figure 10] A reduced quadratic transform (RST) is shown using a 16x64 quadratic transform core. [Figure 11] A reduced quadratic transform (RST) is shown using a 16x48 quadratic transform core. [Figure 12A] 1 shows a forward contractive transformation. [Figure 12B] 1 shows an inverse contractive transformation. [Figure 13A] 1 shows a first embodiment of RST8×8. [Figure 13B] 1 shows a second embodiment of RST8×8. [Figure 14] 1 illustrates a matrix-based intra-prediction (MIP) mode. [Figure 15] 1 shows a flowchart outlining an example process according to some embodiments of the present disclosure. [Figure 16]FIG. 1 is a schematic diagram of a computer system according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0025] FIG. 1 illustrates a simplified block diagram of a communication system (100) according to one embodiment of the present disclosure. The communication system (100) includes multiple terminal devices capable of communicating with each other, e.g., via a network (150). For example, the communication system (100) includes a first pair of terminal devices (110) and (120) interconnected via the network (150). In the example of FIG. 1, the first pair of terminal devices (110) and (120) engage in unidirectional data transmission. For example, the terminal device (110) may encode video data (e.g., a stream of video pictures captured by the terminal device (110)) for transmission to another terminal device (120) via the network (150). The encoded video data may be transmitted in the form of one or more coded video bitstreams. The terminal device (120) may receive the coded video data from the network (150), decode the coded video data to reconstruct the video pictures, and display the video pictures according to the reconstructed video data. One-way data transmission may be common in media serving applications and the like.

[0026] In another example, the communication system 100 includes a second pair of terminal devices 130 and 140 for bidirectional transmission of encoded video data, such as may occur during a video conference. In one example, each of the terminal devices 130 and 140 may encode video data (e.g., a stream of video pictures captured by that terminal device) for transmission to the other of the terminal devices 130 and 140 over the network 150. Each of the terminal devices 130 and 140 may also receive encoded video data transmitted by the other of the terminal devices 130 and 140, decode the encoded video data to recover the video pictures, and display the video pictures on an accessible display device in accordance with the recovered video data.

[0027] Although terminal devices 110, 120, 130, and 140 are depicted in FIG. 1 as a server, a personal computer, and a smartphone, the principles of the present disclosure may not be so limited. Embodiments of the present disclosure find application in laptop computers, tablet computers, media players, and / or dedicated videoconferencing equipment. Network 150 represents any number of networks, including, for example, wired and / or wireless communication networks, that convey encoded video data between terminal devices 110, 120, 130, and 140. Communication network 150 may exchange data over circuit-switched and / or packet-switched channels. Exemplary networks include telecommunications networks, local area networks, wide area networks, and / or the Internet. For purposes of this description, the architecture and topology of network 150 may not be important to the operation of the present disclosure, unless otherwise described below.

[0028] 2 illustrates the arrangement of a video encoder and a video decoder in a streaming environment as an example of an application of the disclosed subject matter, which may be equally applied to other uses where video can be used, including, for example, video conferencing, digital TV, and storage of compressed video on digital media including CDs, DVDs, memory sticks, and the like.

[0029] The streaming system may include a capture subsystem (213), which may include a video source (201), such as a digital camera, that produces a stream of uncompressed video pictures (202). In one example, the stream of video pictures (202) includes samples captured by the digital camera. The stream of video pictures (202), depicted as a thick line to emphasize its high data volume compared to the encoded video data (204) (or encoded video bitstream), may be processed by an electronics device (220) that includes a video encoder (203) coupled to the video source (201). The video encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data (204) (or encoded video bitstream (204)), depicted as a thin line to emphasize its low data volume compared to the stream of video pictures (202), may be stored on a streaming server (205) for later use. One or more streaming client subsystems, such as the client subsystems 206 and 208 of FIG. 2, can access the streaming server 205 to retrieve copies 207 and 209 of the encoded video data 204. The client subsystem 206 can include a video decoder 210, for example, within an electronics device 230. The video decoder 210 can decode the incoming copy of the encoded video data 207 and produce an outgoing stream of video pictures 211, which can be rendered on a display 212 (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data 204, 207, and 209 (e.g., a video bitstream) can be encoded according to a particular video encoding / compression standard. Examples of such standards include ITU-T Recommendation H.265.In one example, one video coding standard under development is informally known as Versatile Video Coding (VVC), and the subject matter disclosed herein may be used in the context of VVC.

[0030] It should be noted that electronics devices 220 and 230 may include other components (not shown). For example, electronics device 220 may include a video decoder (not shown), and electronics device 230 may also include a video encoder (not shown).

[0031] 3 shows a block diagram of a video decoder (310) according to one embodiment of the present disclosure. The video decoder (310) may be included in an electronics device (330). The electronics device (330) may include a receiver (331) (e.g., a receiving circuit). The video decoder (310) may be used in place of the video decoder (210) in the example of FIG. 2.

[0032] In this or another embodiment, the receiver (331) can receive one or more coded video sequences to be decoded by the video decoder (310), one coded video sequence at a time, with the decoding of each coded video sequence being independent of the other coded video sequences. The coded video sequences can be received from a channel (301), which can be a hardware / software link to a storage device that stores the coded video data. The receiver (331) can also receive the coded video data along with other data, such as coded audio data and / or auxiliary data streams, which can be forwarded to their respective using entities (not shown). The receiver (331) can separate the coded video sequences from other data. To combat network jitter, a buffer memory (315) can be coupled between the receiver (331) and the entropy decoder / parser 320 (hereinafter, "parser (320)"). In certain applications, the buffer memory (315) is part of the video decoder (310). In others, it may be external to the video decoder (310) (not shown). In still others, there may be a buffer memory (not shown) external to the video decoder (310), e.g., to combat network jitter, and there may be another buffer memory (315) internal to the video decoder (310), e.g., to handle playback timing. When the receiver (331) is receiving data from a store-and-forward device with sufficient bandwidth and controllability or from an equally synchronous network, the buffer memory (315) may not be required or may be small. For use over best-effort packet networks, such as the Internet, the buffer memory (315) may be required and may be relatively large and advantageously sized adaptively, and may be implemented, at least in part, in an operating system or similar element (not shown) external to the video decoder (310).

[0033] The video decoder (310) may include a parser (320) for reconstructing symbols (321) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (310) and possibly information for controlling a rendering device, such as a render device (312) (e.g., a display screen) that is not an integral part of the electronics device (330) but may be coupled to the electronics device (330), as shown in FIG. 3. The control information for the rendering device(s) may be in the form of a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (320) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may be according to a video coding technique or standard and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (320) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser (320) may also extract information from the coded video sequence information, such as transform coefficients, quantization parameter values, motion vectors, etc.

[0034] The parser (320) may perform an entropy decoding / parsing process on the video sequence received from the buffer memory (315) to produce symbols (321).

[0035] The reconstruction of the symbols (321) may involve several different units, depending on the type of coded video picture or portion thereof and other factors (e.g., inter-picture and intra-picture, inter-block and intra-block, etc.). Which units are involved and how they are involved can be controlled by subgroup control information parsed from the coded video sequence by the parser (320). The flow of such subgroup control information between the parser (320) and the following units is not shown for clarity.

[0036] Beyond the functional blocks already described, the video decoder (310) may be conceptually subdivided into a number of functional units, as described below. In practical implementations operating within commercial constraints, many of these units may interact closely with each other and may be at least partially integrated with each other. However, for purposes of illustrating the subject matter of this disclosure, the following conceptual division into functional units is appropriate:

[0037] The first unit is a scalar / inverse transform unit (351), which receives quantized transform coefficients as symbol(s) (321) from the parser (320), along with control information including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. The scalar / inverse transform unit (351) can output blocks of sample values that can be input to an aggregator (355).

[0038] In some cases, the output samples of the scaler / inverse transform unit (351) may relate to intra-coded blocks, i.e., blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information may be provided by an intra-picture prediction unit (352). In some cases, the intra-picture prediction unit (352) generates blocks of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from a current picture buffer (358). The current picture buffer (358), for example, buffers partially reconstructed and / or fully reconstructed current pictures. The aggregator (355) optionally adds, on a sample-by-sample basis, the prediction information generated by the intra-prediction unit (352) to the output sample information provided by the scaler / inverse transform unit (351).

[0039] In other cases, the output samples of the scalar / inverse transform unit (351) may relate to a block that may be inter-coded and motion-compensated. In such cases, the motion-compensated prediction unit (353) may access the reference picture memory (357) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (321) related to the block, these samples may be appended by the aggregator (355) to the output of the scalar / inverse transform unit (351) (in this case, referred to as residual samples or residual signals) to generate output sample information. The addresses in the reference picture memory (357) from which the motion-compensated prediction unit (353) fetches prediction samples may be controlled by a motion vector and are available to the motion compensation unit (353) in the form of symbols (321), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values fetched from the reference picture memory (357) when sub-sample accurate motion vectors are used, motion vector prediction mechanisms, and the like.

[0040] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in a loop filter unit (356). Video compression techniques may include in-loop filter techniques controlled by parameters included in the coded video sequence (also referred to as coded video bitstream) and made available to the loop filter unit (356) as symbols (321) from the parser (320), but may also be responsive to meta-information obtained during decoding of a previous portion of the coded picture or coded video sequence (in decoding order), as well as to previously reconstructed and loop-filtered sample values.

[0041] The output of the loop filter unit (356) can be a sample stream that can be output to a render device (312), which can also be stored in a reference picture memory (357) for use in future inter-picture prediction.

[0042] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and identified as a reference picture (e.g., by the parser (320)), the current picture buffer (358) can become part of the reference picture memory (357), and a new current picture buffer can be reallocated before beginning reconstruction of the next coded picture.

[0043] The video decoder (310) may perform decoding according to a predetermined video compression technique in a standard, such as ITU-T Recommendation H.265. The coded video sequence may conform to the syntax specified by the video compression technique or standard used, in the sense of adhering to both the syntax of the video compression technique or standard and the profile documented in the video compression technique or standard. Specifically, a profile may select specific tools from all tools available in the video compression technique or standard, such that only those tools are available for use under that profile. Compliance also requires that the complexity of the coded video sequence be within a range specified by the level of the video compression technique or standard. In some cases, the level constrains the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may optionally be further constrained through a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled in the coded video sequence.

[0044] In one embodiment, the receiver (331) may receive additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. The additional data may be used by the video decoder (310) to properly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0045] 4 shows a block diagram of a video encoder (403) according to one embodiment of the present disclosure. The video encoder (403) is included in an electronics device (420). For example, the electronics device (420) includes a transmitter (440) (e.g., a transmitting circuit). The video encoder (403) can be used in place of the video encoder (203) in the example of FIG. 2.

[0046] The video encoder (403) may receive video samples from a video source (401) (which in the example of FIG. 4 is not part of the electronics device (420)) that may capture the video image(s) to be encoded by the encoder (403). In another example, the video source (401) is part of the electronics device (420).

[0047] The video source (401) may provide a source video sequence to be encoded by the video encoder (403) in the form of a digital video sample stream that may be of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., Y CrCb 4:2:0, Y CrCb 4:4:4). In a media service provision system, the video source (401) may be a storage device containing pre-prepared video. In a video conferencing system, the video source (401) may be a camera capturing local image information as a video sequence. The video data may be provided as multiple individual pictures that convey motion when viewed in sequence. The pictures themselves may be organized as a spatial array of pixels, each of which may have one or more samples, depending on the sampling structure, color space, etc. used. Those skilled in the art will readily understand the relationship between pixels and samples. The following description focuses on samples.

[0048] According to one embodiment, the video encoder (403) may encode and compress pictures of a source video sequence into an encoded video sequence (443) in real time or under other time constraints required by the application. Enforcing an appropriate encoding rate is one function of the controller (450). In some embodiments, the controller (450) controls and is operatively coupled to other functional units, as described below, which are not shown for clarity. Parameters set by the controller (450) may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, etc.), picture size, group-of-picture (GOP) layout, maximum motion vector search range, etc. The controller (450) may be configured with other suitable functions associated with the video encoder (403) that are optimized for a particular system design.

[0049] In some embodiments, the video encoder (403) is configured to operate in an encoding loop. As an overly simplified explanation, in one example, the encoding loop can include a source coder (430) (e.g., responsible for creating symbols, e.g., a symbol stream, based on an input picture to be encoded and one or more reference pictures) and a (local) decoder (433) embedded in the video encoder (403). The decoder (433) reconstructs the symbols to generate sample data, in a manner similar to that created by a (remote) decoder (when the compression between the symbols and the encoded video bitstream is lossless in the video compression techniques considered in the disclosed subject matter). The reconstructed sample stream (sample data) is input to a reference picture memory (434). Because decoding of the symbol stream yields bit-accurate results independent of the decoder location (local or remote), the contents of the reference picture memory (434) are also bit-accurate between the local and remote encoders. In other words, the prediction part of the encoder "sees" the exact same sample values as the reference picture samples that the decoder "sees" when using the prediction during decoding. This basic principle of reference picture synchrony (and the resulting drift when synchrony cannot be maintained, e.g., due to channel errors) is also used in some related technologies.

[0050] The operation of the "local" decoder (433) may be the same as that of a "remote" decoder, such as the video decoder (310), which has already been described in detail above in connection with Figure 3. However, briefly referring also to Figure 3, because symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder (445) and parser (320) may be lossless, the entropy decoding portion of the video decoder (310), including the buffer memory (315) and parser (320), may not be fully implemented in the local decoder (433).

[0051] At this point, it can be noted that any decoder technology, with the exception of parsing / entropy decoding, present in the decoder must necessarily be present in the corresponding encoder, in substantially the same functional form. The description of the encoder technology can be omitted, as it is the reverse of the decoder technology that has been thoroughly described. Only in certain areas is a more detailed description required, which is provided below.

[0052] In operation, in some examples, the source coder (430) may perform motion-compensated predictive coding, which predictively codes an input picture relative to one or more previously coded pictures from a video sequence designated as “reference pictures.” Thus, the coding engine (432) codes differences between pixel blocks of the input picture and pixel blocks of one or more reference pictures that can be selected as prediction basis(s) for the input picture.

[0053] The local video decoder (433) may decode coded video data for pictures that may be designated as reference pictures based on symbols created by the source coder (430). The operation of the coding engine (432) may advantageously be a lossy process. When the coded video data is decoded by a video decoder (not shown in FIG. 4), the reconstructed video sequence may typically be a replica of the source video sequence, with some error. The local video decoder (433) may replicate the decoding process that may be performed by a video decoder on the reference pictures and cause the reconstructed reference pictures to be stored in a reference picture cache (434). In this way, the video encoder (403) may locally store copies of reconstructed reference pictures that have content in common with reconstructed reference pictures that will be obtained by a far-end video decoder.

[0054] The predictor (435) may perform a predictive search for the coding engine (432). That is, for a new picture to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate reference pixel blocks) or specific metadata, such as reference picture motion vectors or block shapes, that can serve as suitable prediction references for the new picture. The predictor (435) may operate pixel block by pixel block to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (434).

[0055] The controller (450) may manage the encoding process of the source coder (430), including, for example, setting the parameters and subgroup parameters used to encode the video data.

[0056] The outputs of all the aforementioned functional units may be subjected to entropy coding in an entropy coder (445), which converts the symbols produced by the various functional units into a coded video sequence by losslessly compressing the symbols according to techniques such as Huffman coding, variable length coding, or arithmetic coding.

[0057] A transmitter (440) may buffer the coded video sequence(s) produced by the entropy coder (445) and prepare them for transmission over a communication channel (460), which may be a hardware or software link to a storage device that stores the coded video data. The transmitter (440) may merge the coded video data from the video coder (403) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0058] The controller (450) may manage the operation of the video encoder (403). During encoding, the controller (450) may assign each coded picture a particular coded picture type, which may affect the coding technique that may be applied to the respective picture. For example, pictures may often be assigned one of the following picture types:

[0059] An intra picture (I-picture) may be one that can be coded and decoded without using any other picture in a sequence as a source of prediction. Some video codecs allow several different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are aware of these variants of I-pictures and their respective uses and characteristics.

[0060] A predicted picture (P picture) may be encoded and decoded using intra- or inter-prediction, using at most one motion vector and reference index to predict the sample values of each block.

[0061] Bidirectionally predicted pictures (B-pictures) may be coded and decoded using intra- or inter-prediction, using at most two motion vectors and reference indices to predict the sample values of each block. Similarly, multi-predicted pictures may use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0062] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks as determined by the coding assignment applied to their respective pictures. For example, blocks of an I-picture may be coded non-predictively, or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be coded non-predictively or via spatial or temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be coded non-predictively or via spatial or temporal prediction with reference to one or two previously coded reference pictures.

[0063] The video encoder (403) may perform an encoding process according to a predetermined video encoding technique or standard, such as ITU-T Recommendation H.265. In its operation, the video encoder (403) may perform various compression processes, including predictive encoding processes that exploit temporal and spatial redundancies in the input video sequence. The encoded video data may therefore conform to a syntax specified by the video encoding technique or standard being used.

[0064] In one embodiment, the transmitter (440) may transmit additional data along with the coded video. The source coder (430) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0065] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (often abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or otherwise) between pictures. In one example, a particular picture being coded / decoded, called the current picture, is divided into multiple blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. A motion vector points to a reference block in the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are used.

[0066] In some embodiments, bi-prediction techniques can be used in inter-picture prediction. Bi-prediction techniques use two reference pictures, such as a first reference picture and a second reference picture, both of which are prior to the current picture in decoding order (but may be prior and future, respectively, in display order) in a video. A block in the current picture can be coded with a first motion vector pointing to a first reference block in the first reference picture and a second motion vector pointing to a second reference block in the second reference picture. The block can be predicted by a combination of the first and second reference blocks.

[0067] Furthermore, merge mode techniques can be used to improve coding efficiency in inter-picture prediction.

[0068] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed on a block-by-block basis. For example, according to the HEVC standard, a picture in a sequence of video pictures is divided into multiple coding tree units (CTUs) for compression, and the CTUs within a picture have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs), one luma CTB and two chroma CTBs. Each CTU may be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64×64 pixel CTU may be divided into one CU of 64×64 pixels, four CUs of 32×32 pixels, or 16 CUs of 16×16 pixels. In one example, each CU is analyzed to determine the prediction type of that CU, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. Generally, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations during coding (encoding / decoding) are performed in units of prediction blocks. Taking a luma prediction block as an example of a prediction block, the prediction block includes a matrix of pixel values (e.g., luma values), such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, and the like.

[0069] 5 shows a diagram of a video encoder (503) according to another embodiment of the present disclosure. The video encoder (503) is configured to receive sample values of a processing block (e.g., a predictive block) in a current video picture in a sequence of video pictures and encode the processing block into a coded picture that is part of a coded video sequence. In one example, the video encoder (503) is used in place of the video encoder (203) in the example of FIG. 2.

[0070] In an HEVC example, the video encoder (503) receives a matrix of sample values for a processing block, such as 8x8 samples of a predictive block. The video encoder (503) determines, for example, using rate-distortion optimization, whether the processing block is best coded using intra mode, inter mode, or bi-predictive mode. If the processing block is coded in intra mode, the video encoder (503) can code the processing block into a coded picture using intra prediction techniques; if the processing block is coded in inter mode or bi-predictive mode, the video encoder (503) can code the processing block into a coded picture using inter prediction techniques or bi-predictive techniques, respectively. In certain video coding techniques, the merge mode may be an inter-picture prediction sub-mode in which a motion vector is derived from one or more motion vector predictors without the benefit of coded motion vector components outside the predictors. In certain other video coding techniques, there may be motion vector components applicable to the current block. In one example, the video encoder (503) includes other components, such as a mode decision module (not shown) for determining the mode of a processing block.

[0071] In the example of Figure 5, the video encoder (503) includes an inter-encoder (530), an intra-encoder (522), a residual calculator (523), a switch (526), a residual encoder (524), a general controller (521), and an entropy encoder (525), coupled together as shown in Figure 5.

[0072] The inter-encoder (530) is configured to receive samples of a current block (e.g., a processing block), compare the block to one or more reference blocks in a reference picture (e.g., blocks in a previous picture and a later picture), generate inter-prediction information (e.g., a description of redundant information according to an inter-coding technique, motion vectors, merge mode information), and calculate an inter-prediction result (e.g., a predicted block) based on the inter-prediction information using any suitable technique. In some examples, the reference picture is a decoded reference picture based on coded video information.

[0073] The intra encoder (522) receives samples of a current block (e.g., a processing block), in some cases compares the block with previously coded blocks in the same picture, generates transformed quantized coefficients, and in some cases also generates intra prediction information (e.g., intra prediction direction information according to one or more intra coding techniques). In one example, the intra encoder (522) also calculates intra prediction results (e.g., predicted blocks) based on the intra prediction information and reference blocks in the same picture.

[0074] The general controller (521) is configured to determine general control data and control other components of the video encoder (503) based on the general control data. In one example, the general controller (521) determines the mode of the block and provides a control signal to the switch (526) based on the mode. For example, when the mode is intra mode, the general controller (521) controls the switch (526) to select intra mode results for use by the residual calculator (523) and controls the entropy encoder (525) to select intra prediction information and include the intra prediction information in the bitstream. When the mode is inter mode, the general controller (521) controls the switch (526) to select inter prediction results for use by the residual calculator (523) and controls the entropy encoder (525) to select inter prediction information and include the inter prediction information in the bitstream.

[0075] The residual calculator (523) is configured to calculate the difference (residual data) between the received block and a prediction result selected from the intra-encoder (522) or the inter-encoder (530). The residual encoder (524) operates based on the residual data and is configured to encode the residual data to generate transform coefficients. In one example, the residual encoder (524) is configured to transform the residual data from the spatial domain to the frequency domain to generate transform coefficients. The transform coefficients are then subjected to a quantization process to obtain quantized transform coefficients. In various embodiments, the video encoder (503) also includes a residual decoder (528). The residual decoder (528) is configured to perform an inverse transform and generate decoded residual data. The decoded residual data can be suitably used by the intra-encoder (522) and the inter-encoder (530). For example, the inter-encoder (530) may generate decoded blocks based on the decoded residual data and inter-prediction information, and the intra-encoder (522) may generate decoded blocks based on the decoded residual data and intra-prediction information. The decoded blocks are suitably processed to generate decoded pictures, which may be buffered in a memory circuit (not shown) and, in some cases, used as reference pictures.

[0076] The entropy encoder (525) is configured to format a bitstream to include the coded blocks. The entropy encoder (525) is configured to include various information in accordance with a suitable standard, such as the HEVC standard. In one example, the entropy encoder (525) is configured to include general control data, selected prediction information (e.g., intra-prediction information or inter-prediction information), residual information, and other suitable information in the bitstream. Note that, according to the subject matter of this disclosure, residual information is not present when coding blocks in a merged sub-mode of either the inter-mode or bi-prediction mode.

[0077] 6 shows a diagram of a video decoder (610) according to another embodiment of the present disclosure. The video decoder (610) is configured to receive coded pictures that are part of a coded video sequence and decode the coded pictures to generate reconstructed pictures. In one example, the video decoder (610) is used in place of the video decoder (210) in the example of FIG. 2.

[0078] In the example of Figure 6, the video decoder (610) includes an entropy decoder (671), an inter-decoder (680), a residual decoder (673), a reconstruction module (674), and an intra-decoder (672), coupled together as shown in Figure 6.

[0079] The entropy decoder (671) may be configured to reconstruct, from the coded picture, certain symbols that represent the syntax elements that make up the coded picture. Such symbols may include, for example, prediction information (e.g., intra-prediction information or inter-prediction information) that may identify the mode in which the block is coded (e.g., intra-mode, inter-mode, bi-prediction mode, merged sub-mode, or the latter two in other sub-modes), certain samples or metadata to be used for prediction by the intra-decoder (672) or inter-decoder (680), respectively, residual information in the form of, for example, quantized transform coefficients, and the like. In one example, when the prediction mode is an inter-mode or bi-prediction mode, the inter-prediction information is provided to the inter-decoder (680), and when the prediction type is an intra-prediction type, the intra-prediction information is provided to the intra-decoder (672). The residual information may be subjected to inverse quantization and provided to the residual decoder (673).

[0080] The inter decoder (680) is configured to receive the inter prediction information and to generate an inter prediction result based on the inter prediction information.

[0081] The intra decoder (672) is configured to receive intra prediction information and generate a prediction result based on the intra prediction information.

[0082] The residual decoder (673) is configured to perform inverse quantization to extract dequantized transform coefficients, and then process the dequantized transform coefficients to transform the residual from the frequency domain to the spatial domain. The residual decoder (673) may also require certain control information (including quantizer parameters (QP)), which may be provided by the entropy decoder (671) (data path not shown as this may be only low-volume control information).

[0083] The reconstruction module (674) is configured to combine the residual output by the residual decoder (673) and the prediction result (output by the inter- or intra-prediction module, as the case may be) in the spatial domain to form a reconstructed block. The reconstructed block may be part of a reconstructed picture, which in turn may be part of a reconstructed video. Note that other suitable processes, such as deblocking and the like, may be performed to improve visual quality.

[0084] It should be noted that the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using any suitable technology. In one embodiment, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (203), (403), and (503) and the video decoders (210), (310), and (610) may be implemented using one or more processors executing software instructions.

[0085] Aspects of the present disclosure provide a set of advanced video coding techniques. More specifically, an improved implicit transformation approach is proposed.

[0086] In the video coding community, ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC1 / SC29 / WG11) published the H.265 / HEVC (High Efficiency Video Coding) standard in 2013 (Version 1), 2014 (Version 2), 2015 (Version 3), and 2016 (Version 4). Since then, ITU-T and ISO / IEC have been studying the potential need for standardization of future video coding technologies with compression capabilities significantly exceeding those of the HEVC standard (including its extensions). In October 2017, ITU-T and ISO / IEC jointly issued a call for proposals for video compression technologies with capabilities exceeding those of HEVC (CfP). By February 15, 2018, a total of 22 CfP responses for Standard Dynamic Range (SDR), 12 for High Dynamic Range (HDR), and 12 for the 360 video category had been submitted. In April 2018, all received CfP responses were evaluated at the 122 MPEG / 10th Joint Video Exploration Team (JVET) meeting. JVET officially launched the standardization of the next generation video coding beyond HEVC (i.e., Versatile Video Coding (VVC)) and the latest version of the VVC Test Model (VTM) (i.e., VTM3).

[0087] In HEVC, the primary transform can be a 4-point, 8-point, 16-point, and 32-point DCT-2, and the transform core matrix can be represented using 8-bit integers, i.e., an 8-bit transform core. The transform core matrix of a smaller DCT-2 is a part of a larger DCT-2, as shown below: 4x4 conversion

number

number

number

number

[0088] The DCT-2 core exhibits symmetric / antisymmetric characteristics, so a so-called "partial butterfly" implementation is supported to reduce the number of operation counts (multiplications, additions / subtractions, shifts) and to obtain equivalent matrix multiplication results using partial butterflies.

[0089] VVC provides two sub-block transforms. The first sub-block transform is SVT or SBT. JVET-J0024, JVET-K0139, and JVET-L0358 propose a spatially varying transform (SVT) scheme. In SVT, for inter-prediction residuals, only a residual block can exist within a coding block. Because the residual block is smaller than the coding block, the transform size in SVT is smaller than the coding block size. For regions not covered by the residual block or transform, zero residual can be assumed.

[0090] More specifically, in JVET-L0358, SVT is also called Sub-block Transform (SBT). The sub-block types (SVT-H, SVT-V), sizes, and positions (left half, left quarter, right half, right quarter, top half, top quarter, bottom half, bottom quarter) supported by SBT are shown in Figures 7A-7D. Figures 7A-7D show the sub-block types (SVT-H, SVT-V) and positions (left half, right half, top half, bottom half) supported by SBT, respectively. The shaded areas labeled with the letter "A" are residual blocks with transform, and other areas are assumed to be zero residual without transform.

[0091] The second sub-block transform is Intra Sub-Partitions (ISP). The ISP coding mode divides a luma intra-prediction block into two or four sub-partitions vertically or horizontally, depending on the block size dimension, as shown in Table 1. Figures 8 and 9 show two example possibilities. Figure 8 shows an exemplary division of a 4x8 block or an 8x4 block. Figure 9 shows an exemplary division of a block that is neither a 4x8 block, an 8x4 box, nor a 4x4 block. All sub-partitions satisfy the condition that they have at least 16 samples. ISP is not applied to chroma components. [Table 1]

[0092] In some embodiments, for each of these subpartitions, a residual signal may be generated by entropy decoding the coefficients transmitted by the encoder, and then inverse quantizing and inverse transforming the coefficients. The subpartitions are then intra-predicted, and finally, the corresponding reconstructed samples are obtained by adding the residual signal to the prediction signal. Thus, the reconstructed values of each subpartition can be used to generate a prediction for the next subpartition, which can then repeat the process. All subpartitions share the same intra-mode.

[0093] In some embodiments, the ISP algorithm is only tested on intra modes that are part of the MPM list. For this reason, if a block uses an ISP, the MPM flag can be inferred to be 1. Additionally, if an ISP is used for a particular block, the MPM list can be modified to exclude DC mode and to prioritize horizontal intra mode for ISP horizontal split and vertical intra mode for ISP vertical split.

[0094] In ISP, the transformation and reconstruction are performed on each subpartition separately, so each subpartition can be considered a sub-TU.

[0095] In the current VVC, besides the 4-point, 8-point, 16-point, and 32-point DCT-2 transforms that are the same as HEVC, additional 2-point and 64-point DCT-2 transforms are also added for the primary transform. The 64-point DCT-2 core specified in VVC can be shown below as a 64x64 matrix: { {aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa ,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa,aa} {bf,bg,bh,bi,bj,bk,bl,bm,bn,bo,bp,bq,br,bs,bt,bu,bv,bw,bx,by,bz,ca,cb,cc,cd,ce,cf,cg,ch,ci,cj,ck,-ck,-cj,-ci,-ch ,-cg,-cf,-ce,-cd,-cc,-cb,-ca,-bz,-by,-bx,-bw,-bv,-bu,-bt,-bs,-br,-bq,-bp,-bo,-bn,-bm,-bl,-bk,-bj,-bi,-bh,-bg,-bf} {ap,aq,ar,as,at,au,av,aw,ax,ay,az,ba,bb,bc,bd,be,-be,-bd,-bc,-bb,-ba,-az,-ay,-ax,-aw,-av,-au,-at,-as,-ar,-aq,-ap ,-ap,-aq,-ar,-as,-at,-au,-av,-aw,-ax,-ay,-az,-ba,-bb,-bc,-bd,-be,be,bd,bc,bb,ba,az,ay,ax,aw,av,au,at,as,ar,aq,ap} {bg,bj,bm,bp,bs,bv,by,cb,ce,ch,ck,-ci,-cf,-cc,-bz,-bw,-bt,-bq,-bn,-bk,-bh,-bf,-bi,-bl,-bo,-br,-bu,-bx,-ca,-cd,-cd g,-cj,cj,cg,cd,ca,bx,bu,br,bo,bl,bi,bf,bh,bk,bn,bq,bt,bw,bz,cc,cf,ci,-ck,-ch,-ce,-cb,-by,-bv,-bs,-bp,-bm,-bj,-bg}} {ah,ai,aj,ak,al,am,an,ao,-ao,-an,-am,-al,-ak,-aj,-ai,-ah,-ah,-ai,-aj,-ak,-al,-am,-an,-ao,ao,an,am,al,ak,aj,ai,ah ,ah,ai,aj,ak,al,am,an,ao,-ao,-an,-am,-al,-ak,-aj,-ai,-ah,-ah,-ai,-aj,-ak,-al,-am,-an,-ao,ao,an,am,al,ak,aj,ai,ah} {bh,bm,br,bw,cb,cg,-ck,-cf,-ca,-bv,-bq,-bl,-bg,-bi,-bn,-bs,-bx,-cc,-ch,cj,ce,bz,bu,bp,bk,bf,bj,bo,bt,by,cd,ci,-c i,-cd,-by,-bt,-bo,-bj,-bf,-bk,-bp,-bu,-bz,-ce,-cj,ch,cc,bx,bs,bn,bi,bg,bl,bq,bv,ca,cf,ck,-cg,-cb,-bw,-br,-bm,-bh}} {aq,at,aw,-av,-as,-ap,-ar,-au,-ax,-ba,-bd,bd,ba,ax,au,ar,ap,as,av,ay,bb,be,-bc,-az,-aw,-at,-aq ,-aq,b,ay,av,as,ap,ar,au,ax,ba,bd,-bd,-ba,-ax,-au,-ar,-ap,-as,-av,-ay,-bb,-be,bc,az,aw,at,aq} {bi,bp,bw,cd,ck,-ce,-bx,-bq,-bj,-bh,-bo,-bv,-cc,-cj,cf,by,br,bk,bg,bn,bu,cb,ci,-cg,-bz,-bs,-bl,-bf,-bm,-b ch,ch,ca,bt,bm,bf,bl,bs,bz,cg,-ci,-cb,-bu,-bn,-bg,-bk,-br,-by,-cf,cj,cc,bv,bo,bh,bj,bq,bx,ce,-ck,-cd,-bw,-bp {ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad,ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad ,ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad,ad,ae,af,ag,-ag,-af,-ae,-ad,-ad,-ae,-af,-ag,ag,af,ae,ad} {bj,bs,cb,ck,-cc,-bt,-bk,-bi,-br,-ca,-cj,cd,bu,bl,bh,bq,bz,ci,-ce,-bv,-bm,-bg,-bp,-by,-ch,cf,bw,bn,bf,bg,bo,-bx cg,-bx,-bo,-bf,-bn,-bw,-cf,ch,by,bp,bg,bm,bv,ce,-ci,-bz,-bq,-bh,-bl,-bu,-cd,cj,ca,br,bi,bk,bt,cc,-ck,-bj,-bs {ar,aw,bb,-bd,-ay,-at,-ap,-au,-az,-be,ba,av,aq,as,ax,bc,-bc,-ax,-as,-aq,-av,-ba,be,az,au,ap,at,ay,bd,-bb,-aw,-ar ,-ar,-aw,-bb,bd,ay,at,ap,au,az,be,-ba,-av,-aq,-as,-ax,-bc,bc,ax,as,aq,av,ba,-be,-az,-au,-ap,-at,-ay,-bd,bb,aw,ar} {bk,bv,cg,-ce,-bt,-bi,-bm,-bx,-ci,cc,br,bg,bo,bz,ck,-ca,-bp,-bf,-bq,-cb,cj,by,bn,bh,bs,cd,-ch,-bw,-bl,-bj,-bu,-c f,cf,bu,bj,bl,bw,ch,-cd,-bs,-bh,-bn,-by,-cj,cb,bq,bf,bp,ca,-ck,-bz,-bo,-bg,-br,-cc,ci,bx,bm,bi,bt,ce,-cg,-bv,-bk}} {ai,al,ao,-am,-aj,-ah,-ak,-an,an,ak,ah,aj,am,-ao,-al,-ai,-ai,-al,-ao,am,aj,ah,ak,an,-ak,-ah,-aj,-am,ao,al,ai ,ai,al,ao,-am,-aj,-ah,-ak,-an,an,ak,ah,aj,am,-ao,-al,-ai,-ai,-al,-ao,am,aj,ah,ak,an,-ak,-ah,-aj,-am,ao,al,ai} {bl,by,-ck,-bx,-bk,-bm,-bz,cj,bw,bj,bn,ca,-ci,-bv,-bi,-bo,-cb,ch,bu,bh,bp,cc,-cg,-bt,-bg,-bq,-cd,cf,bs,bf,bk,ce,b -ce,-br,-bf,-bs,-cf,cd,bq,bg,bt,cg,-cc,-bp,-bh,-bu,-ch,cb,bo,bi,bv,ci,-ca,-bn,-bj,-bw,-cj,bz,bm,bk,-bx,ck,-bl} {as,az,-bd,-aw,-ap,-av,-bc,ba,at,ar,ay,-be,-ax,-aq,-au,-bb,bb,au,aq,ax,be,-ay,-ar,-at,-ba,bc,av,ap,aw,bd,-az,-as ,-as,-az,bd,aw,ap,av,bc,-ba,-at,-ar,-ay,be,ax,aq,au,bb,-bb,-au,-aq,-ax,-be,ay,ar,at,ba,-bc,-av,-ap,-aw,-bd,az,as} {bm,cb,-cf,-bq,-bi,-bx,cj,bu,bf,bt,ci,-by,-bj,-bp,-ce,cc,bn,bl,ca,-cg,-br,-bh,-bw,ck,bv,bg,bs,ch,-bz,-bk,-bk, ,cd,bo,bk,bz,-ch,-bs,-bg,-bv,-ck,bw,bh,br,cg,-ca,-bl,-bn,-cc,ce,bp,bj,by,-ci,-bt,-bf,-bu,-cj,bx,bi,bq,-cf,-c {ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab ,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab,ab,ac,-ac,-ab,-ab,-ac,ac,ab} {bn,ce,-ca,-bj,-br,-ci,bw,bf,bv,-cj,-bs,-bi,-bz,cf,bo,bm,cd,-cb,-bk,-bq,-ch,bx,bg,bu,-ck,-bt,-bh,-by,bcg,ccbp ,-cc,-bl,-bp,-cg,by,bh,bt,ck,-bu,-bg,-bx,ch,bq,bk,cb,-cd,-bm,-bo,-cf,bz,bi,bs,cj,-bv,-bf,-bw,ci,br,bj,bn,}- {at,bc,-ay,-ap,-ax,bd,au,as,bb,-az,-aq,-aw,be,av,ar,ba,-ba,-ar,-av,-be,aw,aq,az,-bb,-as,-au,-bd,ax,ap,ay,-bc,-at ,-at,-bc,ay,ap,ax,-bd,-au,-as,-bb,az,aq,aw,-be,-av,-ar,-ba,ba,ar,av,be,-aw,-aq,-az,bb,as,au,bd,-ax,-ap,-ay,bc,at} {bo,ch,-bv,-bh,-ca,cc,bj,bt,-cj,-bq,-bm,-cf,bx,bf,by,-ce,-bl,-br,-ck,bs,bk,cd,-bz,-bg,-bw,cg,bn,bp,ci,-bu,-bi,-c b,cb,bi,bu,-ci,-bp,-bn,-cg,bw,bg,bz,-cd,-bk,-bs,ck,br,bl,ce,-by,-bf,-bx,cf,bm,bq,cj,-bt,-bj,-cc,ca,bh,bv,-bow}} {aj,ao,-ak,-ai,-an,al,ah,am,-am,-ah,-al,an,ai,ak,-ao,-aj,-aj,-ao,ak,ai,an,-ah,-am,am,ah,al,-an,-ai,-ak,ao,aj ,aj,ao,-ak,-ai,-an,al,ah,am,-am,-ah,-al,an,ai,ak,-ao,-aj,-aj,-ao,ak,ai,an,-ah,-am,am,ah,al,-an,-ai,-ak,ao,aj} {bp,ck,-bq,-bo,-cj,br,bn,ci,-bs,-bm,-ch,bt,bl,cg,-bu,-bk,-cf,bv,bj,ce,-bw,-bi,-cd,bx,bh,cc,-by,-bg,-cb,bz,bf,ca -ca,-bf,-bz,cb,bg,by,-cc,-bh,-bx,cd,bi,bw,-ce,-bj,-bv,cf,bk,bu,-cg,-bl,-bt,ch,bm,bs,-ci,-bn,-br,cj,bo,bq,-bp}} {au,-be,-at,-av,bd,as,aw,-bc,-ar,-ax,bb,aq,ay,-ba,-ap,-az,az,ap,ba,-ay,-aq,-bb,ax,ar,bc,-aw,-as,-bd,av,at,be,-au ,-au,be,at,av,-bd,-as,-aw,bc,ar,ax,-bb,-aq,-ay,ba,ap,az,-az,-ap,-ba,ay,aq,bb,-ax,-ar,-bc,aw,as,bd,-av,-at,-be,au} {bq,-ci,-bl,-bv,cd,bg,ca,-by,-bi,-cf,bt,bn,ck,-bo,-bs,cg,bj,bx,-cb,-bf,-cc,bw,bk,ch,-br,-bp,cj,bm,bu,-ce,- ,bz,bh,ce,-bu,-bm,-cj,bp,br,-ch,-bk,-bw,cc,bf,cb,-bx,-bj,-cg,bs,bo,-ck,-bn,-bt,cf,bi,by,-ca,-bg,-cd,bv,-bql} {ae,-ag,-ad,-af,af,ad,ag,-ae,-ae,ag,ad,af,-af,-ad,-ag,ae,ae,-ag,-ad,-af,af,ad,ag,-ae,-ae,ad,af,-af,-ad,-ag,ae ,ae,-ag,-ad,-af,af,ad,ag,-ae,-ae,ag,ad,af,-af,-ad,-ag,ae,ae,-ag,-ad,-af,af,ad,ag,-ae,-ae,ag,ad,af,-af,-ad,-ag,ae} {br,-cf,-bg,-cc,bu,bo,-ci,-bj,-bz,bx,bl,ck,-bm,-bw,ca,bi,ch,-bp,-bt,cd,bf,ce,-bs,-bq,cg,bh,cb,-bv,-bkn,bycj-b by,-bk,-cj,bn,bv,-cb,-bh,-cg,bq,bs,-ce,-bf,-cd,bt,bp,-ch,-bi,-ca,bj,bm,-ck,-bl,-bx,bz,bj,ci,-bo,-bu,cc,bg,bg,cf {av,-bb,-ap,-bc,au,aw,-ba,-aq,-bd,at,ax,-az,-ar,-be,as,ay,-ay,-as,be,ar,az,-ax,-at,bd,aq,ba,-aw,-au,bc,ap,bb,-av ,-av,bb,ap,bc,-au,-aw,ba,aq,bd,-at,-ax,az,ar,be,-as,-ay,ay,as,-be,-ar,-az,ax,at,-bd,-aq,-ba,aw,au,-bc,-ap,-bb,av} {bs,-cc,-bi,-cj,bl,bz,-bv,-bp,cf,bf,cg,-bo,-bw,by,bm,-ci,-bh,-cd,br,bt,-cb,-bj,-ck,bk,ca,-bu,-bq,ce,bg,ch,-bn x,bx,bn,-ch,-bg,-ce,bq,bu,-ca,-bk,ck,bj,cb,-bt,-br,cd,bh,ci,-bm,-by,bw,bo,-cg,-bf,-cf,bp,bv,-bz,-bl,cj,bi,cc,-bs}} {ak,-am,-ai,ao,ah,an,-aj,-al,al,aj,-an,-ah,-ao,ai,am,-ak,-ak,am,ai,-ao,-ah,-an,aj,al,-al,-aj,an,ah,ao,-ai,-am,ak ,ak,-am,-ai,ao,ah,an,-aj,-al,al,aj,-an,-ah,-ao,ai,am,-ak,-ak,am,ai,-ao,-ah,-an,aj,al,-al,-aj,an,ah,ao,-ai,-am,ak} {bt,-bz,-bn,cf,bh,ck,-bi,-ce,bo,by,-bu,-bs,ca,bm,-cg,-bg,-cj,bj,cd,-bp,-bx,bv,br,-cb,-bl,ch,bf,ci,-bk,-cc,bq,bw, -bw,-bq,cc,bk,-ci,-bf,-ch,bl,cb,-br,-bv,bx,bp,-cd,-bj,cj,bg,cg,-bm,-ca,bs,bu,-by,-bo,ce,bi,-ck,-bh,-cf,bn,bz,-bt}} {aw,-ay,-au,ba,as,-bc,-aq,be,ap,bd,-ar,-bb,at,az,-av,-ax,ax,av,-az,-at,bb,ar,-bd,-ap,-be,aq,bc,-as,-ba,au,ay,-aw ,-aw,ay,au,-ba,-as,bc,aq,-be,-ap,-bd,ar,bb,-at,-az,av,ax,-ax,-av,az,at,-bb,-ar,bd,ap,be,-aq,-bc,as,ba,-au,-ay,aw} {bu,-bw,-bs,by,bq,-ca,-bo,cc,bm,-ce,-bk,cg,bi,-ci,-bg,ck,bf,cj,-bh,-ch,bj,cf,-bl,-cd,bn,cb,-bp,-bz,br,bx,-b ,bv,bt,-bx,-br,bz,bp,-cb,-bn,cd,bl,-cf,-bj,ch,bh,-cj,-bf,-ck,bg,ci,-bi,-cg,bk,ce,-bm,-cc,bo,ca,-bq,-bw,-bu},bs {Ah,-aa,-aa,aa,aa,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa being being ,aa,-aa,-aa,aa,aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa being being {bv,-bt,-bx,br,bz,-bp,-cb,bn,cd,-bl,-cf,bj,ch,-bh,-cj,bf,-ck,-bg,ci,bi,-cg,-bk,ce,bm,-cc,-bo,ca,bq,bq,-wby,bu,-bs ,-bu,-bw,bs,by,-bq,-ca,bo,cc,-bm,-ce,bk,cg,-bi,-ci,bg,ck,-bf,cj,bh,-ch,-bj,cf,bl,-cd,-bn,cb,bp,-bz,-br,,bx,}b {ax,-av,-az,at,bb,-ar,-bd,ap,-be,-aq,bc,as,-ba,-au,ay,aw,-aw,-ay,au,ba,-as,-bc,aq,be,-ap,bd,ar,-bb,-at,az,av,-ax ,-ax,av,az,-at,-bb,ar,bd,-ap,be,aq,-bc,-as,ba,au,-ay,-aw,aw,ay,-au,-ba,as,bc,-aq,-be,ap,-bd,-ar,bb,at,-az,-av,ax} {bw,-bq,-cc,bk,ci,-bf,ch,bl,-cb,-br,bv,bx,-bp,-cd,bj,cj,-bg,cg,bm,-ca,-bs,bu,by,-bo,-ce,bi,ck,-bh,cf,bn,-bz,-bt,-bt bt,bz,-bn,-cf,bh,-ck,-bi,ce,bo,-by,-bu,bs,ca,-bm,-cg,bg,-cj,-bj,cd,bp,-bx,-bv,br,cb,-bl,-ch,bf,-ci,-bk,cc,bq,-bw}} {al,-aj,-an,ah,-ao,-ai,am,ak,-ak,-am,ai,ao,-ah,an,aj,-al,-al,aj,an,-ah,ao,ai,-am,-ak,ak,am,-ai,-ao,ah,-an,-aj,al ,al,-aj,-an,ah,-ao,-ai,am,ak,-ak,-am,ai,ao,-ah,an,aj,-al,-al,aj,an,-ah,ao,ai,-am,-ak,ak,am,-ai,-ao,ah,-an,-aj,al} {bx,-bn,-ch,bg,-ce,-bq,bu,ca,-bk,-ck,bj,-cb,-bt,br,cd,-bh,ci,bm,-by,-bw,bo,cg,-bf,cf,bp,-bv,-bz,bl,cj,-bi,cc,bs -bs,-cc,bi,-cj,-bl,bz,bv,-bp,-cf,bf,-cg,-bo,bw,by,-bm,-ci,bh,-cd,-br,bt,cb,-bj,ck,bk,-ca,-bu,bq,ce,-bg,ch,bn,-bx}} {ay,-as,-be,ar,-az,-ax,at,bd,-aq,ba,aw,-au,-bc,ap,-bb,-av,av,bb,-ap,bc,au,-aw,-ba,aq,-bd,-at,ax,az,-ar,be,as,-ay ,-ay,as,be,-ar,az,ax,-at,-bd,aq,-ba,-aw,au,bc,-ap,bb,av,-av,-bb,ap,-bc,-au,aw,ba,-aq,bd,at,-ax,-az,ar,-be,-as,ay} {by,-bk,cj,bn,-bv,-cb,bh,-cg,-bq,bs,ce,-bf,cd,bt,-bp,-ch,bi,-ca,-bw,bm,ck,-bl,bx,bz,-bj,ci,bo,-bu,-ccf,bg,- ,br,cf,-bg,cc,bu,-bo,-ci,bj,-bz,-bx,bl,-ck,-bm,bw,ca,-bi,ch,bp,-bt,-cd,bf,-ce,-bs,bq,cg,-bh,cb,bv,-bn,-by}cj, {af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af,af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af ,af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af,af,-ad,ag,ae,-ae,-ag,ad,-af,-af,ad,-ag,-ae,ae,ag,-ad,af} {bz,-bh,ce,bu,-bm,cj,bp,-br,-ch,bk,-bw,-cc,bf,-cb,-bx,bj,-cg,-bs,bo,ck,-bn,bt,cf,-bi,by,ca,-bg,cd,bv,b,-b,-ci bq,-ci,bl,-bv,-cd,bg,-ca,-by,bi,-cf,-bt,bn,-ck,-bo,bs,cg,-bj,bx,cb,-bf,cc,bw,-bk,ch,br,-bp,-cj,bm,-bu,-bz,-bh {az,-ap,ba,ay,-aq,bb,ax,-ar,bc,aw,-as,bd,av,-at,be,au,-au,-be,at,-av,-bd,as,-aw,-bc,ar,-ax,-bb,aq,-ay,-ba,ap,-az ,-az,ap,-ba,-ay,aq,-bb,-ax,ar,-bc,-aw,as,-bd,-av,at,-be,-au,au,be,-at,av,bd,-as,aw,bc,-ar,ax,bb,-aq,ay,ba,-ap,az} {ca,-bf,bz,cb,-bg,by,cc,-bh,bx,cd,-bi,bw,ce,-bj,bv,cf,-bk,bu,cg,-bl,bt,ch,-bm,bs,ci,-bn,br,cj,-bo,bq,ck,-bp,-bp ck,-bq,bo,-cj,-br,bn,-ci,-bs,bm,-ch,-bt,bl,-cg,-bu,bk,-cf,-bv,bj,-ce,-bw,bi,-cd,-bx,bh,-cc,-by,bg,-cb,-bf,-ca} {am,-ah,al,an,-ai,ak,ao,-aj,aj,-ao,-ak,ai,-an,-al,ah,-am,-am,ah,-al,-an,ai,-ak,-ao,aj,-aj,ao,ak,-ai,an,al,-ah,am ,am,-ah,al,an,-ai,ak,ao,-aj,aj,-ao,-ak,ai,-an,-al,ah,-am,-am,ah,-al,-an,ai,-ak,-ao,aj,-aj,ao,ak,-ai,an,al,-ah,am} {cb,-bi,bu,ci,-bp,bn,-cg,-bw,bg,-bz,-cd,bk,-bs,-ck,br,-bl,ce,by,-bf,bx,cf,-bm,bq,-cj,-bt,bj,-cc,-ca,bh,-bv,-bv o,-bo,ch,bv,-bh,ca,cc,-bj,bt,cj,-bq,bm,-cf,-bx,bf,-by,-ce,bl,-br,ck,bs,-bk,cd,bz,-bg,bw,cg,-bn,bp,-ci,-bu,bi,-cb} {ba,-ar,av,-be,-aw,aq,-az,-bb,as,-au,bd,ax,-ap,ay,bc,-at,at,-bc,-ay,ap,-ax,-bd,au,-as,bb,az,-aq,aw,be,-av,ar,-ba ,-ba,ar,-av,be,aw,-aq,az,bb,-as,au,-bd,-ax,ap,-ay,-bc,at,-at,bc,ay,-ap,ax,bd,-au,as,-bb,-az,aq,-aw,-be,av,-ar,ba} {cc,-bl,bp,-cg,-by,bh,-bt,ck,bu,-bg,bx,ch,-bq,bk,-cb,-cd,bm,-bo,cf,bz,-bi,bs,-cj,-bv,bf,-bw,-ci,br,ce,-bj,ca, ,bn,-ce,-ca,bj,-br,ci,bw,-bf,bv,cj,-bs,bi,-bz,-cf,bo,-bm,cd,cb,-bk,bq,-ch,-bx,bg,-bu,-ck,bt,-bh,by,cl,-ccp {ac,-ab,ab,-ac,-ac,-ab,-ab,ac,ac,-ab,-ac,-ac,-ab,-ab,ac,ac,-ab,-ac,-ac,-ab,-ab,ac,ac,-ab,-ac,-ac,ab,-ab,ac ,ac,-ab,ab,-ac,-ac,-ab,-ab,ac,ac,-ab,-ac,-ac,-ab,-ab,ac,ac,-ab,-ac,-ac,-ab,-ab,ac,ac,-ab,-ac,-ac,ab,-ab,ac} {cd,-bo,bk,-bz,-ch,bs,-bg,bv,-ck,-bw,bh,-br,cg,ca,-bl,bn,-cc,-ce,bp,-bj,by,ci,-bt,bf,-bu,cj,bx,-bi,bcb,-cf,- ,-bm,cb,cf,-bq,bi,-bx,-cj,bu,-bf,bt,-ci,-by,bj,-bp,ce,cc,-bn,bl,-ca,-cg,br,-bh,bw,ck,-bv,bg,-bs,ch,bz,-cd,bk} {bb,-au,aq,-ax,be,ay,-ar,at,-ba,-bc,av,-ap,aw,-bd,-az,as,-as,az,bd,-aw,ap,-av,bc,ba,-at,ar,-ay,-be,ax,-aq,au,-bb ,-bb,au,-aq,ax,-be,-ay,ar,-at,ba,bc,-av,ap,-aw,bd,az,-as,as,-az,-bd,aw,-ap,av,-bc,-ba,at,-ar,ay,be,-ax,aq,-au,bb} {ce,-br,bf,-bs,cf,cd,-bq,bg,-bt,cg,cc,-bp,bh,-bu,ch,cb,-bo,bi,-bv,ci,ca,-bn,bj,-bw,cj,bz,-bm,bk,-bx,ck,by,-bl,bl ,-by,-ck,bx,-bk,bm,-bz,-cj,bw,-bj,bn,-ca,-ci,bv,-bi,bo,-cb,-ch,bu,-bh,bp,-cc,-cg,bt,-bg,bq,-cd,-cf,bs,-bf,-br}} {an,-ak,ah,-aj,am,ao,-al,ai,-ai,al,-ao,-am,aj,-ah,ak,-an,-an,ak,-ah,aj,-am,-ao,al,-ai,ai,-al,ao,am,-aj,ah,-ak,an ,an,-ak,ah,-aj,am,ao,-al,ai,-ai,al,-ao,-am,aj,-ah,ak,-an,-an,ak,-ah,aj,-am,-ao,al,-ai,ai,-al,ao,am,-aj,ah,-ak,an} {cf,-bu,bj,-bl,bw,-ch,-cd,bs,-bh,bn,-by,cj,cb,-bq,bf,-bp,ca,ck,-bz,bo,-bg,br,-cc,-ci,bx,-bm,bi,-bt,ce,cg,-bv,bk -bk,bv,-cg,-ce,bt,-bi,bm,-bx,ci,cc,-br,bg,-bo,bz,-ck,-ca,bp,-bf,bq,-cb,-cj,by,-bn,bh,-bs,cd,ch,-bw,bl,-bj,bu,-cf} {bc,-ax,as,-aq,av,-ba,-be,az,-au,ap,-at,ay,-bd,-bb,aw,-ar,ar,-aw,bb,bd,-ay,at,-ap,au,-az,be,ba,-av,aq,-as,ax,-bc ,-bc,ax,-as,aq,-av,ba,be,-az,au,-ap,at,-ay,bd,bb,-aw,ar,-ar,aw,-bb,-bd,ay,-at,ap,-au,az,-be,-ba,av,-aq,as,-ax,bc} {cg,-bx,bo,-bf,bn,-bw,cf,ch,-by,bp,-bg,bm,-bv,ce,ci,-bz,bq,-bh,bl,-bu,cd,cj,-ca,br,-bi,bk,-bt,cc,ck,-bjb,bs, j,-bs,cb,-ck,-cc,bt,-bk,bi,-br,ca,-cj,-cd,bu,-bl,bh,-bq,bz,-ci,-ce,bv,-bm,bg,-bp,by,-ch,-cf,bw,-bn,bf,-cg,}b {ag,-af,ae,-ad,ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag,ag,-af,ae,-ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag ,ag,-af,ae,-ad,ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag,ag,-af,ae,-ad,-ae,af,-ag,-ag,af,-ae,ad,-ad,ae,-af,ag} {ch,-ca,bt,-bm,bf,-bl,bs,-bz,cg,ci,-cb,bu,-bn,bg,-bk,br,-by,cf,cj,-cc,bv,-bo,bh,-bj,bq,-bx,ce,ck,-cd,biw,-bx,ce,ck,-cd,bw,- bi,bp,-bw,cd,-ck,-ce,bx,-bq,bj,-bh,bo,-bv,cc,-cj,-cf,by,-br,bk,-bg,bn,-bu,cb,-ci,-cg,bz,-bs,bl,-bf,cam,-ch}bt, {bd,-ba,ax,-au,ar,-ap,as,-av,ay,-bb,be,bc,-az,aw,-at,aq,-aq,at,-aw,az,-bc,-be,bb,-ay,av,-as,ap,-ar,au,-ax,ba,-bd ,-bd,ba,-ax,au,-ar,ap,-as,av,-ay,bb,-be,-bc,az,-aw,at,-aq,aq,-at,aw,-az,bc,be,-bb,ay,-av,as,-ap,ar,-au,ax,-ba,bd} {ci,-cd,by,-bt,bo,-bj,bf,-bk,bp,-bu,bz,-ce,cj,ch,-cc,bx,-bs,bn,-bi,bg,-bl,bq,-bv,ca,-cf,ck,cg,-cb,bw,-br,bm,-bh bh,-bm,br,-bw,cb,-cg,-ck,cf,-ca,bv,-bq,bl,-bg,bi,-bn,bs,-bx,cc,-ch,-cj,ce,-bz,bu,-bp,bk,-bf,bj,-bo,bt,-bp,-cd,-ci} {ao,-an,am,-al,ak,-aj,ai,-ah,ah,-ai,aj,-ak,al,-am,an,-ao,-ao,an,-am,al,-ak,aj,-ai,ah,-ah,ai,-aj,ak,-al,am,-an,ao ,ao,-an,am,-al,ak,-aj,ai,-ah,ah,-ai,aj,-ak,al,-am,an,-ao,-ao,an,-am,al,-ak,aj,-ai,ah,-ah,ai,-aj,ak,-al,am,-an,ao} {cj,-cg,cd,-ca,bx,-bu,br,-bo,bl,-bi,bf,-bh,bk,-bn,bq,-bt,bw,-bz,cc,-cf,ci,ck,-ch,ce,-cb,by,-bv,bs,-bp,bm,-bj,bg -bg,bj,-bm,bp,-bs,bv,-by,cb,-ce,ch,-ck,-ci,cf,-cc,bz,-bw,bt,-bq,bn,-bk,bh,-bf,bi,-bl,bo,-br,bu,-bx,ca,-cd,cg,-cj}} {be,-bd,bc,-bb,ba,-az,ay,-ax,aw,-av,au,-at,as,-ar,aq,-ap,ap,-aq,ar,-as,at,-au,av,-aw,ax,-ay,az,-ba,bb,-bc,bd,-be ,-be,bd,-bc,bb,-ba,az,-ay,ax,-aw,av,-au,at,-as,ar,-aq,ap,-ap,aq,-ar,as,-at,au,-av,aw,-ax,ay,-az,ba,-bb,bc,-bd,be} {ck,-cj,ci,-ch,cg,-cf,ce,-cd,cc,-cb,ca,-bz,by,-bx,bw,-bv,bu,-bt,bs,-br,bq,-bp,bo,-bn,bm,-bl,bk,-bj,bi,-bh,b ,bf,-bg,bh,-bi,bj,-bk,bl,-bm,bn,-bo,bp,-bq,br,-bs,bt,-bu,bv,-bw,bx,-by,bz,-ca,cb,-cc,cd,-ce,cf,-cj,ch,-ck,ci, } ただし、 {aa,ab,ac,ad,ae,af,ag,ah,ai,aj,ak,al,am,an,ao,ap,aq,ar,as,at,au,av,aw,ax,ay,z,ba,bb,bc,bd,be,b f,bg,bh,b,bj,bk,bl,bm,bn,bo,bp,bq,br,b,bt,bu,bv,bw,bx,bx,ca,cb,cc,cd,ce,cf,cg,ch,ci,cj,ck}= {64,83,36,89,75,50,18,90,87,80,70,57,43,25,9,90,90,88,85,82,78,73,67,61,54,46,38,31,22,13,4,9 1,90,90,90,88,87,86,84,83,81,79,77,73,71,69,65,62,59,56,52,48,44,41,37,33,28,24,20,15,11,7,2} である。

[0096] In addition to the DCT-2 and 4x4 DST-7 adopted in HEVC, an Adaptive Multiple Transform (AMT, also known as Enhanced Multiple Transform (EMT), or Multiple Transform Selection (MTS)) scheme is used in VVC for residual coding of both inter-coded and intra-coded blocks. MTS uses multiple selected transforms from the DCT / DST family other than the current transforms in HEVC. The newly introduced transform matrices are DST-7 and DCT-8. Table 2 shows the selected DST / DCT basis functions. [Table 2]

[0097] All linear transform matrices in VVC can be used in 8-bit representation. AMT is applied to CUs whose width and height are both less than or equal to 32, and whether to apply AMT is controlled by a flag called mts_flag. When mts_flag is equal to 0, only DCT-2 can be applied to encode the residual. When mts_flag is equal to 1, an index mts_idx is further signaled using two bins to specify the horizontal and vertical transforms to be used according to Table 3, where a value of 1 means to use DCT-7 and a value of 2 means to use DCT-8. [Table 3]

[0098] In VVC Draft 4, implicit MTS can also be applied when the above signaling-based MTS (i.e., explicit MTS) is not used. In implicit MTS, transform selection is made according to block width and height instead of signaling. More specifically, in the implicit MTS proposed in JVET-M0303, DST-7 is selected for the short side of the block, and DCT-2 is selected for the long side of the block. The transform core of DST-7, which is a matrix composed of basis vectors, can also be expressed as follows: 4-point DST-7: {a,b,c,d} {c,c,0,-c} {d,-a,-c,b} {b,-d,c,-a} Where {a,b,c,d}={29,55,74,84}. 8-point DST-7: {a,b,c,d,e,f,g,h,} {c,f,h,e,b,-a,-d,-g,} {e,g,b,-c,-h,-d,a,f,} {g,c,-d,-f,a,h,b,-e,} {h,-a,-g,b,f,-c,-e,d,} {f,-e,-a,g,-d,-b,h,-c,} {d,-h,e,-a,-c,g,-f,b,} {b,-d,f,-h,g,-e,c,-a,} where {a,b,c,d,e,f,g,h}={17,32,46,60,71,78,85,86}. 16-point DST-7: {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,} {c,f,i,l,o,o,l,i,f,c,0,-c,-f,-i,-l,-o,} {e,j,o,m,h,c,-b,-g,-l,-p,-k,-f,-a,d,i,n,} {g,n,l,e,-b,-i,-p,-j,-c,d,k,o,h,a,-f,-m,} {i,o,f,-c,-l,-l,-c,f,o,i,0,-i,-o,-f,c,l,} {k,k,0,-k,-k,0,k,k,0,-k,-k,0,k,k,0,-k,} {m,g,-f,-n,-a,l,h,-e,-o,-b,k,i,-d,-p,-c,j,} {o,c,-l,-f,i,i,-f,-l,c,o,0,-o,-c,l,f,-i,} {p,-a,-o,b,n,-c,-m,d,l,-e,-k,f,j,-g,-i,h,} {n,-e,-i,j,d,-o,a,m,-f,-h,k,c,-p,b,l,-g,} {l,-i,-c,o,-f,-f,o,-c,-i,l,0,-l,i,c,-o,f,} {j,-m,c,g,-p,f,d,-n,i,a,-k,l,-b,-h,o,-e,} {h,-p,i,-a,-g,o,-j,b,f,-n,k,-c,-e,m,-l,d,} {f,-l,o,-i,c,c,-i,o,-l,f,0,-f,l,-o,i,-c,} {d,-h,l,-p,m,-i,e,-a,-c,g,-k,o,-n,j,-f,b,} {b,-d,f,-h,j,-l,n,-p,o,-m,k,-i,g,-e,c,-a,} However, {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p} = {9,17,25,33,41,49,56,62,66,72,77,81,83,87,89,90}. 32 - point DST - 7: {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,A,B,C,D,E,F,} {c,f,i,l,o,r,u,x,A,D,F,C,z,w,t,q,n,k,h,e,b,-a,-d,-g,-j,-m,-p,-s,-v,-y,-B,-E,} {e,j,o,t,y,D,D,y,t,o,j,e,0,-e,-j,-o,-t,-y,-D,-D,-y,-t,-o,-j,-e,0,e,j,o,t,y,D,} {g,n,u,B,D,w,p,i,b,-e,-l,-s,-z,-F,-y,-r,-k,-d,c,j,q,x,E,A,t,m,f,-a,-h,-o,-v,-C,} {i,r,A,C,t,k,b,-g,-p,-y,-E,-v,-m,-d,e,n,w,F,x,o,f,-c,-l,-u,-D,-z,-q,-h,a,j,s,B,} {k,v,F,u,j,-a,-l,-w,-E,-t,-i,b,m,x,D,s,h,-c,-n,-y,-C,-r,-g,d,o,z,B,q,f,-e,-p,-A,} {m,z,z,m,0,-m,-z,-z,-m,0,m,z,z,m,0,-m,-z,-z,-m,0,m,z,z,m,0,-m,-z,-z,-m,0,m,z,} {o,D,t,e,-j,-y,-y,-j,e,t,D,o,0,-o,-D,-t,-e,j,y,y,j,-e,-t,-D,-o,0,o,D,t,e,-j,-y,} {q,E,n,-c,-t,-B,-k,f,w,y,h,-i,-z,-v,-e,l,C,s,b,-o,-F,-p,a,r,D,m,-d,-u,-A,-j,g,x,} {s,A,h,-k,-D,-p,c,v,x,e,-n,-F,-m,f,y,u,b,-q,-C,-j,i,B,r,-a,-t,-z,-g,l,E,o,-d,-w,} {u,w,b,-s,-y,-d,q,A,f,-o,-C,-h,m,E,j,-k,-F,-l,i,D,n,-g,-B,-p,e,z,r,-c,-x,-t,a,v,} {w,s,-d,-A,-o,h,E,k,-l,-D,-g,p,z,c,-t,-v,a,x,r,-e,-B,-n,i,F,j,-m,-C,-f,q,y,b,-u,} {y,o,-j,-D,-e,t,t,-e,-D,-j,o,y,0,-y,-o,j,D,e,-t,-t,e,D,j,-o,-y,0,y,o,-j,-D,-e,t,} {A,k,-p,-v,e,F,f,-u,-q,j,B,a,-z,-l,o,w,-d,-E,-g,t,r,-i,-C,-b,y,m,-n,-x,c,D,h,-s,} {C,g,-v,-n,o,u,-h,-B,a,D,f,-w,-m,p,t,-i,-A,b,E,e,-x,-l,q,s,-j,-z,c,F,d,-y,-k,r,} {E,c,-B,-f,y,i,-v,-l,s,o,-p,-r,m,u,-j,-x,g,A,-d,-D,a,F,b,-C,-e,z,h,-w,-k,t,n,-q,} {F,-a,-E,b,D,-c,-C,d,B,-e,-A,f,z,-g,-y,h,x,-i,-w,j,v,-k,-u,l,t,-m,-s,n,r,-o,-q,p,} {D,-e,-y,j,t,-o,-o,t,j,-y,-e,D,0,-D,e,y,-j,-t,o,o,-t,-j,y,e,-D,0,D,-e,-y,j,t,-o,} {B,-i,-s,r,j,-A,-a,C,-h,-t,q,k,-z,-b,D,-g,-u,p,l,-y,-c,E,-f,-v,o,m,-x,-d,F,-e,-w,n,} {z,-m,-m,z,0,-z,m,m,-z,0,z,-m,-m,z,0,-z,m,m,-z,0,z,-m,-m,z,0,-z,m,m,-z,0,z,-m,} {x,-q,-g,E,-j,-n,A,-c,-u,t,d,-B,m,k,-D,f,r,-w,-a,y,-p,-h,F,-i,-o,z,-b,-v,s,e,-C,l,} {v,-u,-a,w,-t,-b,x,-s,-c,y,-r,-d,z,-q,-e,A,-p,-f,B,-o,-g,C,-n,-h,D,-m,-i,E,-l,-j,F,-k,} {t,-y,e,o,-D,j,j,-D,o,e,-y,t,0,-t,y,-e,-o,D,-j,-j,D,-o,-e,y,-t,0,t,-y,e,o,-D,j,} {r,-C,k,g,-y,v,-d,-n,F,-o,-c,u,-z,h,j,-B,s,-a,-q,D,-l,-f,x,-w,e,m,-E,p,b,-t,A,-i,} {p,-F,q,-a,-o,E,-r,b,n,-D,s,-c,-m,C,-t,d,l,-B,u,-e,-k,A,-v,f,j,-z,w,-g,-i,y,-x,h,} {n,-B,w,-i,-e,s,-F,r,-d,-j,x,-A,m,a,-o,C,-v,h,f,-t,E,-q,c,k,-y,z,-l,-b,p,-D,u,-g,} {l,-x,C,-q,e,g,-s,E,-v,j,b,-n,z,-A,o,-c,-i,u,-F,t,-h,-d,p,-B,y,-m,a,k,-w,D,-r,f,} {j,-t,D,-y,o,-e,-e,o,-y,D,-t,j,0,-j,t,-D,y,-o,e,e,-o,y,-D,t,-j,0,j,-t,D,-y,o,-e,} {h,-p,x,-F,y,-q,i,-a,-g,o,-w,E,-z,r,-j,b,f,-n,v,-D,A,-s,k,-c,-e,m,-u,C,-B,t,-l,d,} {f,-l,r,-x,D,-C,w,-q,k,-e,-a,g,-m,s,-y,E,-B,v,-p,j,-d,-b,h,-n,t,-z,F,-A,u,-o,i,-c,} {d,-h,l,-p,t,-x,B,-F,C,-y,u,-q,m,-i,e,-a,-c,g,-k,o,-s,w,-A,E,-D,z,-v,r,-n,j,-f,b,} {b,-d,f,-h,j,-l,n,-p,r,-t,v,-x,z,-B,D,-F,E,-C,A,-y,w,-u,s,-q,o,-m,k,-i,g,-e,c,-a,} However, {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, A, B, C, D, E, F} = {4, 9, 13, 17, 21, 26, 30, 34, 38, 42, 45, 50, 53, 56, 60, 63, 66, 68, 72, 74, 77, 78, 80, 82, 84, 85, 86, 88, 88, 89, 90, 90}. 4-point DCT-8: {a, b, c, d,} {b, 0, -b, -b,} {c, -b, -d, a,} {d, -b, a, -c,} However, {a, b, c, d} = {84, 74, 55, 29}. 8-point DCT-8: {a, b, c, d, e, f, g, h,} {b, e, h, -g, -d, -a, -c, -f,} {c, h, -e, -a, -f, g, b, d,} {d, -g, -a, -h, c, e, -f, -b,} {e, -d, -f, c, g, -b, -h, a,} {f, -a, g, e, -b, h, d, -c,} {g, -c, b, -f, -h, d, -a, e,} {h, -f, d, -b, a, -c, e, -g,} However, {a, b, c, d, e, f, g, h} = {86, 85, 78, 71, 60, 46, 32, 17}. 16-point DCT-8: {a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p,} {b, e, h, k, n, 0, -n, -k, -h, -e, -b, -b, -e, -h, -k, -n,} {c, h, m, -p, -k, -f, -a, -e, -j, -o, n, i, d, b, g, l,} {d, k, -p, -i, -b, -f, -m, n, g, a, h, o, -l, -e, -c, -j,} {e, n, -k, -b, -h, 0, h, b, k, -n, -e, -e, -n, k, b, h,} {f,0,-f,-f,0,f,f,0,-f,-f,0,f,f,0,-f,-f,} {g,-n,-a,-m,h,f,-o,-b,-l,i,e,-p,-c,-k,j,d,} {h,-k,-e,n,b,0,-b,-n,e,k,-h,-h,k,e,-n,-b,} {i,-h,-j,g,k,-f,-l,e,m,-d,-n,c,o,-b,-p,a,} {j,-e,-o,a,-n,-f,i,k,-d,-p,b,-m,-g,h,l,-c,} {k,-b,n,h,-e,0,e,-h,-n,b,-k,-k,b,-n,-h,e,} {l,-b,i,o,-e,f,-p,-h,c,-m,-k,a,-j,-n,d,-g,} {m,-e,d,-l,-n,f,-c,k,o,-g,b,-j,-p,h,-a,i,} {n,-h,b,-e,k,0,-k,e,-b,h,-n,-n,h,-b,e,-k,} {o,-k,g,-c,b,-f,j,-n,-p,l,-h,d,-a,e,-i,m,} {p,-n,l,-j,h,-f,d,-b,a,-c,e,-g,i,-k,m,-o,} However, {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p} = {90,89,87,83,81,77,72,66,62,56,49,41,33,25,17,9}. 32-point DCT-8: {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,A,B,C,D,E,F,} {b,e,h,k,n,q,t,w,z,C,F,-E,-B,-y,-v,-s,-p,-m,-j,-g,-d,-a,-c,-f,-i,-l,-o,-r,-u,-x,-A,-D,} {c,h,m,r,w,B,0,-B,-w,-r,-m,-h,-c,-c,-h,-m,-r,-w,-B,0,B,w,r,m,h,c,c,h,m,r,w,B,} {d,k,r,y,F,-A,-t,-m,-f,-b,-i,-p,-w,-D,C,v,o,h,a,g,n,u,B,-E,-x,-q,-j,-c,-e,-l,-s,-z,} {e,n,w,F,-y,-p,-g,-c,-l,-u,-D,A,r,i,a,j,s,B,-C,-t,-k,-b,-h,-q,-z,E,v,m,d,f,o,x,} {f,q,B,-A,-p,-e,-g,-r,-C,z,o,d,h,s,D,-y,-n,-c,-i,-t,-E,x,m,b,j,u,F,-w,-l,-a,-k,-v,} {g,t,0,-t,-g,-g,-t,0,t,g,g,t,0,-t,-g,-g,-t,0,t,g,g,t,0,-t,-g,-g,-t,0,t,g,g,t,} {h,w,-B,-m,-c,-r,0,r,c,m,B,-w,-h,-h,-w,B,m,c,r,0,-r,-c,-m,-B,w,h,h,w,-B,-m,-c,-r,} {i,z,-w,-f,-l,-C,t,c,o,F,-q,-a,-r,E,n,d,u,-B,-k,-g,-x,y,h,j,A,-v,-e,-m,-D,s,b,p,} {j,C,-r,-b,-u,z,g,m,F,-o,-e,-x,w,d,p,-E,-l,-h,-A,t,a,s,-B,-i,-k,-D,q,c,v,-y,-f,-n,} {k,F,-m,-i,-D,o,g,B,-q,-e,-z,s,c,x,-u,-a,-v,w,b,t,-y,-d,-r,A,f,p,-C,-h,-n,E,j,l,} {l,-E,-h,-p,A,d,t,-w,-a,-x,s,e,B,-o,-i,-F,k,m,-D,-g,-q,z,c,u,-v,-b,-y,r,f,C,-n,-j,} {m,-B,-c,-w,r,h,0,-h,-r,w,c,B,-m,-m,B,c,w,-r,-h,0,h,r,-w,-c,-B,m,m,-B,-c,-w,r,h,} {n,-y,-c,-D,i,s,-t,-h,E,d,x,-o,-m,z,b,C,-j,-r,u,g,-F,-e,-w,p,l,-A,-a,-B,k,q,-v,-f,} {o,-v,-h,C,a,D,-g,-w,n,p,-u,-i,B,b,E,-f,-x,m,q,-t,-j,A,c,F,-e,-y,l,r,-s,-k,z,d,} {p,-s,-m,v,j,-y,-g,B,d,-E,-a,-F,c,C,-f,-z,i,w,-l,-t,o,q,-r,-n,u,k,-x,-h,A,e,-D,-b,} {q,-p,-r,o,s,-n,-t,m,u,-l,-v,k,w,-j,-x,i,y,-h,-z,g,A,-f,-B,e,C,-d,-D,c,E,-b,-F,a,} {r,-m,-w,h,B,-c,0,c,-B,-h,w,m,-r,-r,m,w,-h,-B,c,0,-c,B,h,-w,-m,r,r,-m,-w,h,B,-c,} {s,-j,-B,a,-C,-i,t,r,-k,-A,b,-D,-h,u,q,-l,-z,c,-E,-g,v,p,-m,-y,d,-F,-f,w,o,-n,-x,e,} {t,-g,0,g,-t,-t,g,0,-g,t,t,-g,0,g,-t,-t,g,0,-g,t,t,-g,0,g,-t,-t,g,0,-g,t,t,-g,} {u,-d,B,n,-k,-E,g,-r,-x,a,-y,-q,h,-F,-j,o,A,-c,v,t,-e,C,m,-l,-D,f,-s,-w,b,-z,-p,i,} {v,-a,w,u,-b,x,t,-c,y,s,-d,z,r,-e,A,q,-f,B,p,-g,C,o,-h,D,n,-i,E,m,-j,F,l,-k,} {w,-c,r,B,-h,m,0,-m,h,-B,-r,c,-w,-w,c,-r,-B,h,-m,0,m,-h,B,r,-c,w,w,-c,r,B,-h,m,} {x,-f,m,-E,-q,b,-t,-B,j,-i,A,u,-c,p,F,-n,e,-w,-y,g,-l,D,r,-a,s,C,-k,h,-z,-v,d,-o,} {y,-i,h,-x,-z,j,-g,w,A,-k,f,-v,-B,l,-e,u,C,-m,d,-t,-D,n,-c,s,E,-o,b,-r,-F,p,-a,q,} {z,-l,c,-q,E,u,-g,h,-v,-D,p,-b,m,-A,-y,k,-d,r,-F,-t,f,-i,w,C,-o,a,-n,B,x,-j,e,-s,} {A,-o,c,-j,v,F,-t,h,-e,q,-C,-y,m,-a,l,-x,-D,r,-f,g,-s,E,w,-k,b,-n,z,B,-p,d,-i,u,} {B,-r,h,-c,m,-w,0,w,-m,c,-h,r,-B,-B,r,-h,c,-m,w,0,-w,m,-c,h,-r,B,B,-r,h,-c,m,-w,} {C,-u,m,-e,d,-l,t,-B,-D,v,-n,f,-c,k,-s,A,E,-w,o,-g,b,-j,r,-z,-F,x,-p,h,-a,i,-q,y,} {D,-x,r,-l,f,-a,g,-m,s,-y,E,C,-w,q,-k,e,-b,h,-n,t,-z,F,B,-v,p,-j,d,-c,i,-o,u,-A,} {E,-A,w,-s,o,-k,g,-c,b,-f,j,-n,r,-v,z,-D,-F,B,-x,t,-p,l,-h,d,-a,e,-i,m,-q,u,-y,C,} {F,-D,B,-z,x,-v,t,-r,p,-n,l,-j,h,-f,d,-b,a,-c,e,-g,i,-k,m,-o,q,-s,u,-w,y,-A,C,-E,} However, {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y,z,A,B,C,D,E,F}={90,90,89,88, 88,86,85,84,82,80,78,77,74,72,68,66,63,60,56,53,50,45,42,38,34,30,26,21,17,13,9,4}.

[0099] In VVC, in some examples, the transform size can always be the same as the coding block size if both the height and width of the coding block are less than or equal to 64. If either the height or width of the coding block is greater than 64, when performing transform or intra prediction, the coding block is further divided into multiple sub-blocks, each with a width and height less than or equal to 64, and one transform is performed on each sub-block.

[0100] In VVC draft v5, MTS can be enabled or disabled in an SPS using the syntax in Table 4 below. [Table 4]

[0101] In VVC draft v5, in some cases, DST-7 and / or DCT-8 can be used without being explicitly signaled, i.e., DST-7 and / or DCT-8 can be used implicitly based on information available to both the encoder and decoder. These cases include: (a) Intra Sub-Partitioning (ISP): In ISP mode, the horizontal transform is selected as DST-7 as long as the block width is greater than or equal to 4 and less than or equal to 16, and the vertical transform is selected as DST-7 as long as the block height is greater than or equal to 4 and less than or equal to 16. (b) Sub-Block Transform (SBT): In SBT mode, for sub-TUs located in the left half (or quarter) and right half (or quarter) of the current CU, the horizontal transform is DCT-8 and DST-7, respectively. Otherwise, if the sub-TU has the same width as the current CU, DCT-2 is used. For sub-TUs located in the top half (or quarter) and bottom half (or quarter) of the current CU, the vertical transform is DCT-8 and DST-7, respectively. Otherwise, if the sub-TU has the same height as the current CU, DCT-2 is used. (c) MTS is disabled in SPS: If sps_mts_enabled_flag is signaled as true but both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false, then for intra prediction residuals, the horizontal transform is selected as DST-7 as long as the block width is greater than or equal to 4 and less than or equal to 16, and the vertical transform is selected as DST-7 as long as the block height is greater than or equal to 4 and less than or equal to 16.

[0102] In VVC, a mode-dependent non-separable secondary transform (NSST) can be applied between the forward core transform and quantization (at the encoder) and between the inverse quantization and inverse core transform (at the decoder). To maintain low complexity, the NSST is only applied to the low-frequency coefficients after the primary transform. If the width (W) and height (H) of a transform coefficient block are both equal to or greater than 8, an 8x8 non-separable secondary transform may be applied to the top-left 8x8 region of the transform coefficient block. Otherwise, if either W or H of the transform coefficient block is equal to 4, a 4x4 non-separable secondary transform may be applied to the top-left min(8,W) x min(8,H) region of the transform coefficient block, and a 4x4 non-separable transform may be performed. The above transform selection rules can be applied to both the luma and chroma components.

[0103] The matrix multiplication implementation of a non-separable transform is described as follows in equation (1), taking a 4x4 input block as an example: To apply a non-separable transform, given a 4x4 input block X:

number

number

[0104] Non-separable transformations are

number

[0105] In one design of NSST, for both 4x4 and 8x8 block sizes, there can be a total of 35x3 non-separable secondary transforms, where 35 is the number of transform sets defined by the intra prediction modes denoted as sets, and 3 is the number of NSST candidates for each intra prediction mode. The mapping from intra prediction modes to transform sets is specified in Table 5. The transform set applied to luma / chroma transform coefficients is specified by the corresponding luma / chroma intra prediction mode according to Table 5. For intra prediction modes greater than 34 (diagonal prediction direction), transform coefficient blocks are transposed before / after secondary transforms at the encoder / decoder.

[0106] For each transform set, the selected non-separable secondary transform candidate is further specified by an explicitly signaled CU-level NSST index. This index may be signaled in the bitstream once for each CU after the transform coefficients and truncated unary binarization are used. The truncation value is 2 for planar or DC mode and 3 for angular intra-prediction mode. This NSST index may only be signaled if there are two or more non-zero coefficients in the CU. The default value may be zero if not signaled. A zero value for this syntax element may indicate that no secondary transform is applied to the current CU, while values 1 to 3 indicate which secondary transform from the set should be applied.

[0107] In some embodiments, for each transform set, the selected non-separable secondary transform candidate can be further specified by an explicitly signaled CU-level NSST index. This index is signaled in the bitstream once for each CU after the transform coefficients and truncated unary binarization are used. The truncation value is 2 for planar or DC modes and 3 for angular intra-prediction modes. This NSST index is signaled only if there are two or more non-zero coefficients in the CU. The default value is zero if not signaled. A zero value for this syntax element indicates that no secondary transform is applied to the current CU, and values 1 to 3 indicate which secondary transform from the set should be applied. [Table 5]

[0108] In JVET-N0193, a modification of NSST using a conversion zero-out scheme, i.e., a Reduced Size Transform (RST), also known as Low-Frequency Non-Separable Secondary Transform (LFNST) in VVC Draft 5, has been proposed. JVET-N0193 checks whether the intra prediction mode is planar or DC in order to entropy-encode the conversion index of NSST. In JVET-N0193, four conversion sets are applied, and each conversion set includes three RST conversion cores. Those three RST conversion cores can be either of size 16×48 (or 16×64) (applied to a conversion coefficient block where both height and width are 8 or more) or 16×16 (applied to a conversion coefficient block where either height or width is equal to 4). For notation convenience, the 16×48 (or 16×64) conversion is denoted as RST8×8, and the 16×16 conversion is denoted as RST4×4. For RST8×8, two alternatives using a 16×64 conversion core and a 16×48 conversion core are shown in FIGS. 10 and 11, respectively. FIG. 10 shows a reduced secondary transform (RST) using a 16×64 secondary conversion core. FIG. 11 shows a reduced secondary transform (RST) using a 16×48 secondary conversion core. In VVC Draft 5, the conversion using a 16×48 conversion core is adopted.

[0109] An index indicating the selection of the LFNST kernel, i.e., lfnst_idx, is signaled at the end of the CU-level syntax as shown in Table 6. Table 6 provides the syntax at the CU level.

Table 6

[0110] In some examples, a Reduced Transform (RT) maps an N-dimensional vector to an R-dimensional vector in a different space, where R / N (R < N) is the reduction factor. The RST matrix is given by Equation (3):

number

[0111] RST8×8, which has a reduction factor of 4 (quarter size), can be applied. Therefore, a 16×64 direct matrix is used instead of the traditional 8×8 non-separable transform matrix size of 64×64. In other words, a 64×16 inverse RST matrix is used at the decoder side to generate the core (linear) transform coefficients in the top-left region of the 8×8. Forward RST8×8 uses a 16×64 (or 8×64 for 8×8 blocks) matrix so that forward RST8×8 generates non-zero coefficients only in the top-left 4×4 region within a given 8×8 region. In other words, when RST is applied, 8×8 regions except the top-left 4×4 region can only have zero coefficients. RST4×4 can apply 16×16 (or 8×16 for 4×4 blocks) direct matrix multiplication.

[0112] Also, in RST8x8, to further reduce the transform matrix size, instead of using the entire top-left 8x8 coefficient as the input to calculate the secondary transform, the top-left three 4x4 coefficients are used as the input to calculate the secondary transform. Figures 13A-13B show different alternatives to RST8x8. Figure 13A shows an example 16x64 transform matrix, where the entire top-left 8x8 coefficient is applied as the input to calculate the secondary transform. Figure 13B shows an example 16x46 transform matrix, where the top-left three 4x4 coefficients are used as the input to calculate the secondary transform.

[0113] In some embodiments, the inverse RST can be conditionally applied when two conditions are met: (a) the block size is greater than or equal to a given threshold (W>=4 && H>=4); and (b) the transform skip mode flag is equal to zero.

[0114] If both the width (W) and height (H) of a transform coefficient block are greater than 4, RST8×8 is applied to the top-left 8×8 region of the transform coefficient block. Otherwise, RST4×4 is applied to the top-left min(8,W)×min(8,H) region of the transform coefficient block.

[0115] If the RST index is equal to 0, the RST is not applied. Otherwise, if the RST index is equal to 1, the RST is applied and the kernel is selected by the RST index.

[0116] Furthermore, RST is applied to intra CUs in both intra and inter slices, and to both luma and chroma. When dual tree is enabled, RST indices for luma and chroma are signaled separately. In inter slices (dual tree is disabled), a single RST index is signaled and used for both luma and chroma components. When ISP mode is selected, RST is disabled and no RST index is signaled.

[0117] In some embodiments, the RST matrix may be selected from four transform sets, each consisting of two transforms. Which transform set is applied is determined from the intra prediction mode as follows: (a) if one of the three CCLM modes is indicated, then transform set 0 is selected; (b) otherwise, transform set selection is made according to Table 7. [Table 7] The index for accessing Table 7 (ie, IntraPredMode) has a range of [-14, 83], which are the transform mode indices used for wide angle intra prediction.

[0118] VVC also includes a matrix-based intra prediction (MIP) mode. To predict samples of a rectangular block of width W and height H, MIP takes as input a row of H reconstructed adjacent boundary samples located to the left of the block and a row of W reconstructed adjacent boundary samples above the block. If these reconstructed samples are not available, they are generated as is done in conventional intra prediction.

[0119] The generation of the predicted signal is based on three steps: (a) From the boundary samples, 4 samples are extracted if W=H=4, and 8 samples in all other cases by averaging. (b) Using the averaged samples as input, a matrix-vector multiplication followed by the addition of an offset is performed. The result is a reduced prediction signal for the subsampled sample set within the original block. (c) The predicted signals at the remaining positions are generated from the predicted signals for the subsampled set by linear interpolation, which is a single-step linear interpolation in each direction.

[0120] The matrices and offset vectors required to generate the prediction signal are taken from three sets of matrices S0, S1, and S2. Set S0 consists of 18 matrices A0 i ,i∈{0,…,17}, each of which has 16 rows, 4 columns, and 18 offset vectors b0 i ,i∈{0,…,17}. Offset vector b0 i Each has size 16. The matrices and offset vectors in set S0 are used for blocks of size 4x4. Set S1 contains 10 matrices A1 i ,i∈{0,…,9}, and each of these matrices has 16 rows, 8 columns, and 10 offset vectors b1 i ,i∈{0,…,9}. Offset vector b1 iEach has size 16. The matrices and offset vectors in set S1 are used for blocks of size 4x8, 8x4, and 8x8. Finally, set S2 contains six matrices A2 i ,i∈{0,…,5}, and each of these matrices has 64 rows, 8 columns, and 6 offset vectors b2 of size 64. i ,i∈{0,...,5}. The matrices and offset vectors in set S2, or a subset of these intense and offset vectors, are used for all other block shapes.

[0121] Figure 14 shows an example of MIP for an 8x8 block. As shown in Figure 14, given an 8x8 block, MIP takes four averages along each axis of the boundary. The resulting eight input samples enter a matrix-vector multiplication. A matrix is taken from set S1. This produces 16 samples in odd positions of the prediction block. Therefore, a total of (8 16) / (8 8) = 2 multiplications per sample are performed. After adding the offset, these samples are interpolated vertically by using the reduced top boundary. Horizontal interpolation follows by using the original left boundary. In this case, the interpolation process does not require any multiplications.

[0122] Regarding MIP mode signaling, for each intra-mode coding unit (CU), a flag is sent in the bitstream indicating whether MIP mode is applied to the corresponding prediction unit (PU). If MIP mode is applied, the MIP mode index predmode is signaled using an MPM list containing three MPMs.

[0123] Here, the MPM is derived using the intra modes of the top and left PUs as follows: Three fixed mapping tables map_angular_to_mip idx , idx∈{0,1,2}, and each table is expressed as follows:

number

number

[0124] To generate an MPM list for a current block coded by MIP mode, first select the MIP mode, i.e., mode MIP above , and the left MIP mode, i.e. mode MIP left is derived. mode MIP above The value of is derived as follows: (a) PU, which is the PU above above is available and belongs to the same CTU as the current PU, and above But MIP mode predmode MIP above and idx(PU) = idx(PU above ) and encoded by MIP,

number

number

number

[0125] Finally, the derived mode MIP above and mode MIP left and three predefined fixed default MPM lists idx , idx∈{0,1,2} (each of these MPM lists contains three different MIP modes), an MPM list is constructed. The MPM list is constructed by adding these given default lists, list idx(PU) and mode MIP above and mode MIP left It is constructed based on and by replacing default values with -1 and removing duplicate MIP modes.

[0126] The flags signaling the MIP mode may be shown in Table 8, which is a CU level syntax table. [Table 8]

[0127] In some embodiments, MIP modes can be harmonized with MPM-based coding of conventional intra prediction modes as follows: The luma and chroma MPM list derivation process for conventional intra prediction modes is MIP to one of the traditional intra prediction modes:

number

[0128] Although various methods are provided above, the above methods have some drawbacks. For example, currently, when LFNST is applied, sps_mts_enabled_flag is signaled as true, but sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both signaled as false. For small block sizes, the primary transform will be selected using the implicit transform scheme introduced above, which means that DST-7 can always be enabled. However, the LFNST kernel may not operate very efficiently with DST-7.

[0129] Additionally, currently, when MIP is applied, sps_mts_enabled_flag is signaled as true, but sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both signaled as false. For small block sizes, the primary transform will be selected using the implicit transform scheme introduced above, which means that DST-7 can always be enabled. However, DST-7 may not work very efficiently in MIP mode.

[0130] According to embodiments of the present disclosure, methods for improved implicit transform selection are provided. Each of the methods (or embodiments), the encoder, and the decoder may be implemented by a processing circuit (e.g., one or more processors or one or more integrated circuits). In one example, the one or more processors execute a program stored on a non-transitory computer-readable medium. In the following disclosure, the term block may be interpreted as a prediction block, a coding block, or a coding unit (CU).

[0131] According to embodiments of the present disclosure, the term NSST may also refer to the Reduced Quadratic Transform (RST), which is an alternative design of the Non-Separable Quadratic Transform, for example, as described in JVET-M0292 or JVET-N0193, and it may also refer to the Low-Frequency Non-Separable Quadratic Transform (LFNST) adopted in VVC Draft v5.

[0132] According to an embodiment of the present disclosure, the DST-7 may also be replaced by a DST-4.

[0133] According to embodiments of the present disclosure, an "implicit transform" refers to a transform scheme that selects a set of non-DCT2 transforms (e.g., DST-1, DCT-5, DST-7, DCT-8, DST-4, DCT-4, etc.) without transform index signaling. In this regard, the set of non-DCT2 transforms can be selected using already coded information available to both the encoder and the decoder, including, but not limited to, intra-prediction mode (planar mode, DC mode, angular mode), block size, block width, block height, block aspect ratio, block region size, intra-coding mode (whether MRL, ISP, MIP is used), position of selected spatial merge candidate (top merge candidate, left merge candidate), inter-prediction mode (inter-PDPC mode, CIIP mode, etc.).

[0134] In the following disclosure, "explicit transform" refers to a transform scheme that selects one transform from a set of candidate transform types (e.g., DCT-2, DST-1, DCT-5, DST-7, DCT-8, DST-4, DCT-4, etc.) using an index that is signaled to indicate which transform type is selected.

[0135] In a first embodiment, the disclosed method includes obtaining transform block signaling information from a coded video bitstream to determine whether an implicit transform scheme is applied for primary transform type selection. When the implicit transform scheme is selected, it means that the flag sps_mts_enabled_flag is signaled as true, and both the flags sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false. Also, for a W×H block, if the LFNST index (lfnst_idx) is signaled as 0 (i.e., disabled), and if TSM is not enabled, the primary transform is selected using the following algorithm: (a) If W>=T1 and W<=T2, then the horizontal transform is DCT-7. Otherwise, the horizontal transform is DCT-2. Example values for T1 include 2 pixels, 4 pixels, or 8 pixels. Example values for T2 include 4, 8, 16, or 32. (b) If H>=T1 and H<=T2, then the vertical transform is DCT-7. Otherwise, the vertical transform is DCT-2. Example values for T1 include 2, 4, or 8. Example values for T2 include 4, 8, 16, or 32.

[0136] In contrast, in some embodiments, if the LFNST index is not signaled as 0 (i.e., LFNST is applied), the primary transform may be selected as follows: (a) In one example, DCT-2 is always selected. (b) In another example, a predetermined transform type other than DCT-7 is selected, such as a Hadamard transform, a DST-1, a DCT-5, a complex orthonormal transform (COT), or a Karhunen-Loeve transform (KLT).

[0137] The changes to the method proposed in the first embodiment in addition to VVC draft v5 are shown below, with the changes highlighted by underlines.

[0138] In one example, the inputs to the method disclosed above in the first embodiment are: (a) a luma position (xTbY, yTbY) that defines the top-left sample of the current luma transform block relative to the top-left luma sample of the current picture; (b) a variable nTbW that specifies the width of the current transform block; (c) variable nTbH that specifies the height of the current transformation block; (d) a variable cIdx that specifies the color component of the current block; (e) The (nTbW) × (nTbH) array d[x][y] of scaled transform coefficients for x=0...nTbW-1, y=0...nTbH-1. The corresponding output of the above method in the first embodiment may be a (nTbW) x (nTbH) array rd[x][y] of residual samples at x=0...nTbW-1, y=0...nTbH-1.

[0139] The variable implicitMtsEnabled in the first embodiment may be derived as follows: (a) If sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, then implicitMtsEnabled is set equal to 1: (b) IntraSubpartitionsSplitType is not equal to ISP_NO_SPLIT. (c) cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is less than or equal to 32. (d) sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA; and lfnst_idx[x0][y0] is equal to 0 . (e) Otherwise, implicitMtsEnabled is set equal to 0.

[0140] In the first embodiment, the variable trTypeHor that defines the horizontal transformation kernel and the variable trTypeVer that defines the vertical transformation kernel can be derived as follows: (a) If cIdx is greater than 0, then trTypeHor and trTypeVer are set equal to 0. (b) Otherwise, if implicitMtsEnabled is equal to 1, the following applies: (i) If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, or if sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0 and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, then trTypeHor and trTypeVer are derived as follows: trTypeHor=(nTbW>=4 && nTbW<=16)? 1:0 trTypeVer=(nTbH>=4 && nTbH<=16)? 1:0 (ii) Otherwise (cu_sbt_flag is equal to 1), trTypeHor and trTypeVer are specified in Table 10 below depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag. (iii) Otherwise, trTypeHor and trTypeVer are specified in Table 9 below according to tu_mts_idx[xTbY][yTbY].

[0141] Tables 9-10 provide the trTypeHor and trTypeVer designations associated with the method in the first embodiment. [Table 9] [Table 10]

[0142] In a second embodiment, when an implicit transformation scheme is applied for primary transformation type selection (e.g., sps_mts_enabled_flag is signaled as true, but both sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are signaled as false), for W×H blocks that have the MIP flag (intr_mip_flag) signaled as 0 (i.e., MIP is not applied), if TSM is not enabled, the primary transformation may be selected using the following algorithm: (a) If W>=T1 and W<=T2, then the horizontal transform is DCT-7. Otherwise, the horizontal transform is DCT-2. Example values of T1 include 2, 4, or 8. Example values of T2 include 4, 8, 16, or 32. (b) If H>=T1 and H<=T2, then the vertical transform is DCT-7. Otherwise, the vertical transform is DCT-2. Example values for T1 include 2, 4, or 8. Example values for T2 include 4, 8, 16, or 32.

[0143] When the MIP flag is not signaled as 0 (i.e., MIP is applied), the primary transform uses a DCT-2. In another example, the primary transform uses a non-DST-7, such as a Hadamard transform, a DST-1, a DCT-5, a COT, a KLT, etc.

[0144] Alternatively, when the MIP flag (intra_mip_flag) is not equal to 0 (i.e., MIP is applied), the implicit conversion scheme may still be applied. However, the thresholds T1 and T2 may differ from those used for the implicit conversion as introduced above. In one example, T1 is equal to 2 and T2 is equal to 4 or 8. In another example, T1 is equal to 4 and T2 is equal to 4 or 8. In yet another example, T1 is equal to 8 and T2 is equal to 8, 16, or 32. T1 may also be equal to 16 and T2 is equal to 16 or 32.

[0145] An example of changes to the method proposed in the second embodiment in addition to VVC draft v5 is shown below, with the changes highlighted by underlines.

[0146] The inputs to the method in the second embodiment are: (a) a luma position (xTbY, yTbY) that defines the top-left sample of the current luma transform block relative to the top-left luma sample of the current picture; (b) a variable nTbW that specifies the width of the current transform block; (c) variable nTbH that specifies the height of the current transformation block; (d) a variable cIdx that specifies the color component of the current block; (e) The (nTbW) × (nTbH) array d[x][y] of scaled transform coefficients for x=0...nTbW-1, y=0...nTbH-1. The output of this process may be a (nTbW) x (nTbH) array rd[x][y] of residual samples at x=0...nTbW-1, y=0...nTbH-1.

[0147] The variable implicitMtsEnabled is derived as follows: (a) If sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, then implicitMtsEnabled is set equal to 1: (b) IntraSubpartitionsSplitType is not equal to ISP_NO_SPLIT. (c) cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is less than or equal to 32. (d) sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, and intra_mip_flag[x0][y0] is equal to 0. (e) Otherwise, implicitMtsEnabled is set equal to 0.

[0148] In the second embodiment, a variable trTypeHor that defines the horizontal transformation kernel and a variable trTypeVer that defines the vertical transformation kernel can be derived as follows: (a) If cIdx is greater than 0, then trTypeHor and trTypeVer are set equal to 0. (b) Otherwise, if implicitMtsEnabled is equal to 1, the following applies: (i) If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, or if sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0 and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, then trTypeHor and trTypeVer are derived as follows: trTypeHor=(nTbW>=4 && nTbW<=16)? 1:0 trTypeVer=(nTbH>=4 && nTbH<=16)? 1:0 (ii) Otherwise (cu_sbt_flag is equal to 1), trTypeHor and trTypeVer are as specified in Table 10 depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag. (iii) Otherwise, trTypeHor and trTypeVer are as specified in Table 9 according to tu_mts_idx[xTbY][yTbY].

[0149] In the third embodiment, the first and second embodiments can be combined, and an example of specification changes for the proposed method in addition to VVC draft v5 is shown below, with the changes highlighted by underlines:

[0150] The inputs to the method in the third embodiment are: (a) a luma position (xTbY, yTbY) that defines the top-left sample of the current luma transform block relative to the top-left luma sample of the current picture; (b) a variable nTbW that specifies the width of the current transform block; (c) variable nTbH that specifies the height of the current transformation block; (d) a variable cIdx that specifies the color component of the current block; (e) The (nTbW) × (nTbH) array d[x][y] of scaled transform coefficients for x=0...nTbW-1, y=0...nTbH-1. The output of this process may be a (nTbW) x (nTbH) array rd[x][y] of residual samples at x=0...nTbW-1, y=0...nTbH-1.

[0151] The variable implicitMtsEnabled in the third embodiment can be derived as follows: (a) If sps_mts_enabled_flag is equal to 1 and one of the following conditions is true, then implicitMtsEnabled is set equal to 1: (b) IntraSubpartitionsSplitType is not equal to ISP_NO_SPLIT. (c) cu_sbt_flag is equal to 1 and Max(nTbW, nTbH) is less than or equal to 32. (d) sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0, and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, and lfnst_idx[x0][y0] is equal to 0, and intra_mip_flag[x0][y0] is equal to 0. (e) Otherwise, implicitMtsEnabled is set equal to 0.

[0152] In the third embodiment, a variable trTypeHor that defines the horizontal transformation kernel and a variable trTypeVer that defines the vertical transformation kernel are derived as follows: (a) If cIdx is greater than 0, then trTypeHor and trTypeVer are set equal to 0. (b) Otherwise, if implicitMtsEnabled is equal to 1, the following applies: (i) If IntraSubPartitionsSplitType is not equal to ISP_NO_SPLIT, or if sps_explicit_mts_intra_enabled_flag and sps_explicit_mts_inter_enabled_flag are both equal to 0 and CuPredMode[xTbY][yTbY] is equal to MODE_INTRA, then trTypeHor and trTypeVer are derived as follows: trTypeHor=(nTbW>=4 && nTbW<=16)? 1:0 trTypeVer=(nTbH>=4 && nTbH<=16)? 1:0 (ii) Otherwise (cu_sbt_flag is equal to 1), trTypeHor and trTypeVer are as specified in Table 10 depending on cu_sbt_horizontal_flag and cu_sbt_pos_flag. (iii) Otherwise, trTypeHor and trTypeVer are as specified in Table 9 according to tu_mts_idx[xTbY][yTbY].

[0153] In the fourth embodiment of the present disclosure, a combination of implicit transform and explicit transform signaling can be applied. For horizontal (or vertical) transform, either implicit transform or explicit transform can be used.

[0154] In one example, for a W×H block, if W is greater than or equal to T1 and W is less than or equal to T2, the horizontal transform type is DST-7. Otherwise, if W is greater than T2 and less than or equal to T3, the horizontal transform type can be either DCT-2 or DST-7, and the selection is signaled. Otherwise, if W is greater than T3 or less than T1, a default transform type, such as DCT-2, is applied. Example values for T1 include 2, 4, or 8. Example values for T2 include 4, 8, 16, or 32. Example values for T3 include 8, 16, 32, or 64. A combination setting of T1, T2, and T3 can be: T1 equals 4 (or 2), T2 equals 16, and T3 equals 32 (or 64).

[0155] In another example, for a W×H block, if H is greater than or equal to T1 and H is less than or equal to T2, the vertical transform type is DST-7. Otherwise, if H is greater than T2 and less than or equal to T3, the vertical transform type can be either DCT-2 or DST-7, and the selection is signaled. Otherwise, if H is greater than T3 or less than T1, a default transform type, such as DCT-2, is applied. Example values of T1 include 2, 4, or 8. Example values of T2 include 4, 8, 16, or 32. Example values of T3 include 8, 16, 32, or 64. A combined setting of T1, T2, and T3 can be: T1 equals 4 (or 2), T2 equals 16, and T3 equals 32 (or 64).

[0156] FIG. 15 shows a flowchart outlining a process (1500) according to one embodiment of the present disclosure. The process (1500) can be used to reconstruct blocks coded in intra-mode and thus generate a prediction block for the block being reconstructed. In various embodiments, the process (1500) can be performed by a processing circuit, such as the processing circuitry of the terminal devices (110), (120), (130), and (140), a processing circuit performing the functions of the video encoder (203), a processing circuit performing the functions of the video decoder (210), a processing circuit performing the functions of the video decoder (310), a processing circuit performing the functions of the video encoder (403), and the like. In some embodiments, the process (1500) can be implemented in software instructions, such that the processing circuit performs the process (1500) when it executes the software instructions. The process begins at (S1501) and proceeds to (S1510).

[0157] At (S1510), transform block signaling information is obtained from the coded video bitstream. The transform block signaling information may include at least one of a flag sps_mts_enabled_flag, a flag sps_explicit_mts_intra_enabled_flag, or a flag transform skip mode (TSM), or the like. The transform block signaling information may also include an LFNST index (lfnst_idx) or a MIP flag (intra_mip_flag). If the flag sps_mts_enabled_flag is signaled as true, but both the flag sps_explicit_mts_intra_enabled_flag and the flag sps_explicit_mts_inter_enabled_flag are signaled as false, an implicit transform scheme is applied for primary transform type selection.

[0158] At (S1520), a determination may be made by the decoder whether the transform block signaling information indicates an implicit transform scheme and whether at least one of the low-frequency non-separable transform (LFNST) and matrix-based intra-prediction mode (MIP) is impaired. If an implicit transform scheme is applied and at least one of the low-frequency non-separable transform (LFNST) and matrix-based intra-prediction mode (MIP) is impaired, process 1500 proceeds to (S1530).

[0159] At (S1530), in response to determining that the transform block signaling information indicates an implicit transform scheme and at least one of LFNST and MIP is signaled as impaired, a primary transform type is determined based on the size of the coding block unit (CU). Process 1500 then proceeds to (S1540), where a primary transform is performed on the transform blocks split from the CU in accordance with the determined primary transform type.

[0160] If the determination at (S1520) is that an implicit transform scheme is applied but at least one of a low frequency non-separable transform (LFNST) and a matrix-based intra prediction mode (MIP) is true, process 1500 proceeds to (S1550), where, in response to this determination, a transform type other than DCT-2 or DCT-7 may be selected. Transform types other than DCT-7 may include the Hadamard transform, DST-1, DCT-5, COT, and KLT.

[0161] The techniques described above can be implemented as computer software with computer-readable instructions physically stored on one or more computer-readable media. For example, Figure 16 illustrates a computer system (1600) suitable for implementing certain embodiments of the disclosed subject matter.

[0162] Computer software may be coded using any suitable machine code or computer language that can be assembled, compiled, linked, or similarly subjected to mechanisms to produce code having instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), and the like, either directly or via interpretation, microcode execution, and the like.

[0163] The instructions may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0164] 16 with respect to computer system (1600) are exemplary in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing embodiments of the present disclosure, nor should the arrangement of components be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in this exemplary embodiment of computer system (1600).

[0165] The computer system (1600) may include certain human interface input devices. Such human interface input devices may respond to input by one or more human users, for example, via tactile input (e.g., keystrokes, swipes, moving a data glove, etc.), audio input (e.g., voice, clapping, etc.), visual input (e.g., gestures, etc.), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds, etc.), images (e.g., scanned images, photographic images obtained from a still camera, etc.), or video (e.g., two-dimensional video, three-dimensional video including stereoscopic video, etc.).

[0166] The input human interface devices may include one or more of a keyboard (1601), a mouse (1602), a trackpad (1603), a touchscreen (1610), a data glove (not shown), a joystick (1605), a microphone (1606), a scanner (1607), and a camera (1608) (only one of each is shown).

[0167] The computer system (1600) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses, for example, through tactile output, sound, light, and smell / taste. Such human interface output devices may include haptic output devices (e.g., haptic feedback via a touchscreen (1610), data gloves (not shown), or joystick (1605), although some haptic feedback devices may not function as input devices), audio output devices (e.g., speakers (1609), headphones (not shown), etc.), visual output devices (e.g., screens (1610) including CRT screens, LCD screens, plasma screens, and OLED screens (each with or without touchscreen input capability, each with or without haptic feedback capability, some of which may be capable of outputting two-dimensional visual output or four or more dimensional output through means such as stereoscopic output), virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), etc.), and printers (not shown).

[0168] The computer system (1600) may also include human-accessible storage devices and their associated media, such as optical media including, for example, a CD / DVD ROM / RW (1620) with CD / DVD or similar media (1621), a thumb drive (1622), a removable hard drive or solid state drive (1623), legacy magnetic media such as tape and floppy disks (registered trademark, not shown), specialized ROM / ASIC / PLD-based devices (not shown) such as security dongles, and the like.

[0169] Those skilled in the art will also appreciate that the term "computer-readable medium" as used in connection with the subject matter disclosed herein does not include transmission media, carrier waves, or other transitory signals.

[0170] The computer system 1600 may also include interfaces to one or more communications networks. Networks may be, for example, wireless, wired, or optical. Networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, latency-tolerant, and the like. Examples of networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, and the like, TV wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, and vehicular and industrial networks including CAN Bus. Certain networks typically require an external network interface adapter that attaches to a particular general-purpose data port or peripheral bus 1649 (e.g., a USB port on the computer system 1600), while others are typically integrated into the core of the computer system 1600 by attachment to a system bus (e.g., an Ethernet interface to a PC computer system or a cellular network interface to a smartphone computer system). Using any of these networks, computer system 1600 can communicate with other entities. Such communication may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a specific CANbus device), or two-way, for example, to other computer systems using local or wide-area digital networks. Specific protocols and protocol stacks may be used on each network and network interface, as described above.

[0171] The aforementioned human interface devices, human-accessible storage devices, and network interfaces may be attached to the core (1640) of the computer system (1600).

[0172] The core (1640) may include one or more central processing units (CPUs) (1641), graphics processing units (GPUs) (1642), specialized programmable processing units in the form of field programmable gate arrays (FPGAs) (1643), task-specific hardware accelerators (1644), etc. These devices may be connected via a system bus (1648), along with read-only memory (ROM) (1645), random access memory (1646), internal mass storage (1647), such as an internal non-user-accessible hard drive, SSD, and the like. In some computer systems, the system bus (1648) may be made accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, and the like. Peripheral devices may be attached either directly to the core's system bus (1648) or via a peripheral bus (1649). Peripheral bus architectures include PCI, USB, and the like.

[0173] The CPU (1641), GPU (1642), FPGA (1643), and accelerator (1644) may execute specific instructions that, in combination, may constitute the aforementioned computer code. The computer code may be stored in ROM (1645) or RAM (1646). Transient data may also be stored in RAM (1646), while permanent data may be stored, for example, in internal mass storage (1647). Rapid storage and retrieval from any of the memory devices may be enabled through the use of cache memory, which may be associated with one or more of the CPU (1641), GPU (1642), mass storage (1647), ROM (1645), RAM (1646), and the like.

[0174] The computer-readable media may have computer code thereon for performing various computer-implemented processes. The media and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0175] By way of example, and not limitation, a computer system having the architecture (1600), and in particular the core (1640), can provide functionality as a result of the execution by one or more processors (including CPUs, GPUs, FPGAs, accelerators, and the like) of software embodied in one or more tangible computer-readable media. Such computer-readable media can be specific storage of the core (1640) that is non-transitory in nature, such as the core's internal mass storage (1647) or ROM (1645), and media associated with user-accessible mass storage as introduced above. Software implementing various embodiments of the present disclosure can be stored in such devices and executed by the core (1640). The computer-readable media can include one or more memory devices or chips, depending on specific needs. Software may cause the core (1640) and particularly the processors therein (including CPUs, GPUs, FPGAs, and the like) to perform particular processes or portions of particular processes described herein, including by defining data structures stored in RAM (1646) and modifying such data structures according to processes defined by the software. Additionally, or alternatively, the computer system may provide functionality as a result of logic hardwired or otherwise embodied in circuitry (e.g., accelerators (1644)) that can operate in place of or in conjunction with software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include circuitry (e.g., integrated circuits (ICs)) storing software for execution, circuitry embodying logic for execution, or both, where appropriate. The present disclosure includes any suitable combination of hardware and software. Appendix A: Acronyms JEM: joint exploration model VVC: versatile video coding BMS:benchmark set MV: Motion Vector HEVC: High Efficiency Video Coding SEI: Supplementary Enhancement Information VUI: Video Usability Information GOPs: Groups of Pictures TUs: Transform Units PUs: Prediction Units CTUs: Coding Tree Units CTBs: Coding Tree Blocks PBs: Prediction Blocks HRD: Hypothetical Reference Decoder SNR: Signal-to-Noise Ratio CPUs: Central Processing Units GPUs: Graphics Processing Units CRT:Cathode Ray Tube LCD: Liquid-Crystal Display OLED: Organic Light-Emitting Diode CD: Compact Disc DVD: Digital Video Disc ROM: Read-Only Memory RAM: Random Access Memory ASIC: Application-Specific Integrated Circuit PLD: Programmable Logic Device LAN: Local Area Network GSM: Global System for Mobile communications LTE: Long-Term Evolution CANBus: Controller Area Network Bus USB: Universal Serial Bus PCI: Peripheral Component Interconnect FPGA: Field Programmable Gate Array SSD: solid-state drive IC: Integrated Circuit CU: Coding Unit

[0176] While this disclosure describes several exemplary embodiments, there are alterations, permutations, and various equivalent alternatives that fall within the scope of the disclosure. It will thus be appreciated that those skilled in the art will be able to devise numerous systems and methods that, although not explicitly shown or described herein, embody the principles of the disclosure and are therefore within its spirit and scope.

Claims

1. A method of video decoding performed by a decoder, Steps include obtaining signaling information from an encoded video bitstream, The step involves determining, based on the signaling information, whether an implicit conversion scheme is enabled or disabled for a coding unit (CU) coded in intra-mode. The step of determining that the implicit transformation scheme is enabled when the signaling information indicates that multiple transformation selection (MTS) is enabled, that the MTS index is not signaled, and that both low-frequency non-separated transformation (LFNST) and matrix-based intra-predictive mode (MIP) are disabled. In response to the determination that the implicit transformation scheme is enabled, The primary conversion type is determined based on the size of the CU, and The steps include: performing a primary transformation on the transformation block separated from the CU according to the primary transformation type determined above; A method of having.

2. The step of determining the primary conversion type is: Determine whether the conversion skip mode is enabled, and In response to the determination that the conversion skip mode is enabled, (i) Depending on whether the width of the CU is T1 or greater and T2 or less, the conversion type DST-7 is determined for the horizontal conversion of the conversion block, (ii) Depending on whether the width of the CU is less than T1 or greater than T2, the conversion type DCT-2 is determined with respect to the horizontal conversion of the conversion block. (iii) Depending on whether the height of the CU is T1 or greater and T2 or less, the conversion type DST-7 is determined with respect to the vertical conversion of the conversion block. (iv) Depending on whether the height of the CU is less than T1 or greater than T2, the conversion type DCT-2 is determined with respect to the vertical conversion of the conversion block. The method according to claim 1, wherein the method is as follows:

3. The method according to claim 2, wherein T1 is equal to one of 2 pixels, 4 pixels, or 8 pixels, and T2 is equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels.

4. A method of video encoding performed by an encoder, This step determines whether an implicit transformation scheme is enabled or disabled for a coding unit (CU) coded in intra-mode. The step of determining that the implicit transformation scheme is enabled when Multiple Transform Selection (MTS) is enabled, the MTS index is not signaled, and both Low Frequency Non-Separated Transform (LFNST) and Matrix-Based Intra Prediction Mode (MIP) are disabled, In response to the determination that the implicit conversion scheme is enabled, the steps include determining the primary conversion type based on the size of the CU, The steps include: performing a primary transformation on the transformation block separated from the CU according to the primary transformation type determined above; The steps include signaling syntax elements indicating whether the MTS is valid and syntax elements indicating whether the MTS index is signaled, When the LFNST is enabled, a syntax element indicating that the LFNST is enabled is signaled. When the aforementioned MIP is enabled, the step of signaling a syntax element indicating that the aforementioned MIP is enabled, A method of having.

5. The step of determining the primary conversion type is: Determine whether the conversion skip mode is enabled, and In response to the determination that the aforementioned conversion skip mode is enabled, (i) Depending on whether the width of the CU is T1 or greater and T2 or less, the conversion type DST-7 is determined for the horizontal conversion of the conversion block, (ii) Depending on whether the width of the CU is less than T1 or greater than T2, the conversion type DCT-2 is determined with respect to the horizontal conversion of the conversion block. (iii) Depending on whether the height of the CU is T1 or greater and T2 or less, the conversion type DST-7 is determined with respect to the vertical conversion of the conversion block. (iv) Depending on whether the height of the CU is less than T1 or greater than T2, the conversion type DCT-2 is determined with respect to the vertical conversion of the conversion block. The method according to claim 4, wherein the above is achieved.

6. The method according to claim 5, wherein T1 is equal to one of 2 pixels, 4 pixels, or 8 pixels, and T2 is equal to one of 4 pixels, 8 pixels, 16 pixels, or 32 pixels.

7. A method of video encoding performed by an encoder, A step of determining whether an implicit transformation scheme is enabled or disabled for an intra-coded coding unit (CU), wherein it is determined that the implicit transformation scheme is enabled if multiple transformation selection (MTS) is enabled, the MTS index is not signaled, and both low-frequency non-separable transformation (LFNST) and matrix-based intra-prediction mode (MIP) are disabled. In response to the determination that the implicit conversion scheme is enabled, the steps include determining the primary conversion type based on the size of the CU, The steps include: performing a primary transformation on the transformation block separated from the CU according to the primary transformation type determined above; The steps include encoding a syntax element indicating whether the MTS is valid and a syntax element indicating whether the MTS index is signaled into a video bitstream, If the LFNST is enabled, the step of encoding a syntax element indicating that the LFNST is enabled into the video bitstream, If the MIP is enabled, the step of encoding a syntax element indicating that the MIP is enabled into the video bitstream, The steps include transmitting the aforementioned video bitstream, A method of having.

8. A method of video encoding performed by an encoder, A step of determining whether an implicit transformation scheme is enabled or disabled for an intra-coded coding unit (CU), wherein it is determined that the implicit transformation scheme is enabled if multiple transformation selection (MTS) is enabled, the MTS index is not signaled, and both low-frequency non-separable transformation (LFNST) and matrix-based intra-prediction mode (MIP) are disabled. In response to the determination that the implicit conversion scheme is enabled, the steps include determining the primary conversion type based on the size of the CU, The steps include: performing a primary transformation on the transformation block separated from the CU according to the primary transformation type determined above; The steps include encoding a syntax element indicating whether the MTS is valid and a syntax element indicating whether the MTS index is signaled into a video bitstream, If the LFNST is enabled, the step of encoding a syntax element indicating that the LFNST is enabled into the video bitstream, If the MIP is enabled, the step of encoding a syntax element indicating that the MIP is enabled into the video bitstream, The steps include storing the aforementioned video bitstream in a computer-readable medium, A method of having.

9. At least one memory location where the program is stored, At least one processor, It has, The program causes the at least one processor to execute the method according to any one of claims 1 to 3. decoder.

10. At least one memory location where the program is stored, At least one processor, It has, The program causes the at least one processor to execute the method described in any one of claims 4 to 8. Encoder.

11. A computer program that causes a computer to perform the method described in any one of claims 1 to 3.

12. A computer program that causes a computer to perform the method described in any one of claims 4 to 8.

Images