Method and system for signaling and performing a quadratic transformation.

Abstract

Methods and systems for encoding and decoding video are described. In one aspect, a video decoding method includes: receiving video data from a video bitstream, which includes a first block and a first syntax element. The method also includes determining a secondary transformation kernel type value for the first block based on the first syntax element. According to the determination that the secondary transformation kernel type has a first value, a secondary transformation set identifier is determined based on a second syntax element from the video bitstream, and an inverse secondary transformation is performed on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier. According to the determination that the secondary transformation kernel type has a second value, no inverse secondary transformation is performed on the first block.

Description

Technical Field

[0001] Related Applications This application claims the priority of U.S. Provisional Patent Application No. 63 / 530,461, entitled "Signaling Method for Transformation," filed on August 2, 2023, and is a continuation application of U.S. Patent Application No. 18 / 520,259, entitled "Method and System for Signaling and Performing Secondary Transformation," filed on November 27, 2023, and claims the priority thereof.

[0002] Technical Field The disclosed embodiments generally relate to, but are not limited to, the coding and compression of images and videos, including systems and methods for signaling and performing a secondary transform.

Background Art

[0003] Digital video is supported by a variety of electronic devices, including digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video teleconferencing devices, and video streaming devices. These electronic devices transmit and receive digital video data over communication networks, or otherwise communicate and / or store digital video data in storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video coding may be used to compress video data according to one or more video coding standards before it is transmitted or stored. Video coding can be performed by hardware and / or software on electronic / client devices or servers providing cloud services. Video coding generally utilizes predictive methods (e.g., inter-predictive, intra-predictive, etc.) that take advantage of the redundancy inherent in video data. The goal of video coding is to compress video data to a lower bitrate while avoiding or minimizing degradation of video quality. Multiple video codec standards have been developed.

[0004] For example, High-Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H project. The ITU-T and ISO / IEC published the HEVC / H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC / H.266) is a video compression standard intended as a successor to HEVC. The ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AOMedia Video 1 (AV1) is an open video coding format designed as an alternative to HEVC. Verified version 1.0.0, along with errata 1 for the specification, was released on January 8, 2019. [Overview of the project] [Problems that the invention aims to solve]

[0005] Video codecs generally operate on pixel blocks. For example, each pixel block is processed with a predictive transform coding scheme, where the prediction comes from either an in-frame reference pixel, inter-frame motion compensation, or some combination thereof. The residual from the prediction may undergo a 2D transform to remove spatial correlations, and then the transform coefficients may be quantized. Both the predicted syntax elements and the quantized transform coefficient indices may then be entropically coded using arithmetic coding. Some video codecs have begun to use directional or inseparable primary and secondary image transforms to further improve compression efficiency (for example, due to residual correlations between transformed coefficients after a primary transform). [Means for solving the problem]

[0006] In some methods, the parameters of the quadratic transform are implicitly derived (for example, based on the primary transform parameters). However, it may be beneficial to signal at least a subset of the quadratic transform parameters, as this allows the most accurate and / or precise quadratic transform to be applied, thereby improving coding efficiency. Signaling the quadratic transform parameters can increase coding overhead. However, as described herein, coding overhead can be reduced / minimized (for example, by signaling the kernel type first and refraining from signaling the transform set ID if the kernel type indicates that the quadratic transform is not used).

[0007] According to several embodiments, a method for video encoding is provided. The method includes the steps of: (i) receiving video data comprising a plurality of blocks, including a first block; (ii) determining a secondary transformation kernel type value for the first block; (iii) transmitting a first syntax element in a video bitstream, the first syntax element indicating a secondary transformation kernel type value; (iv) in accordance with the determination that the secondary transformation kernel type has a first value: (a) determining a secondary transformation set identifier; (b) transmitting a second syntax element in the video bitstream, the second syntax element indicating a secondary transformation set identifier; and (c) performing a secondary transformation on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier; and (v) in accordance with the determination that the secondary transformation kernel type has a second value: (a) refraining from transmitting a second syntax element; and (b) refraining from performing a secondary transformation on the first block.

[0008] According to several embodiments, a method for video decoding is provided. The method includes: (i) receiving video data from a video bitstream, comprising a plurality of blocks, each comprising a first block, and a first syntax element; (ii) determining a secondary transformation kernel type value for the first block based on the first syntax element; (iii) according to the determination that the secondary transformation kernel type has a first value, (a) determining a secondary transformation set identifier based on a second syntax element from the video bitstream, and (b) performing an inverse secondary transformation on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier, wherein the inverse secondary transformation corresponds to the secondary transformation performed during encoding of the video data; and (iv) refraining from performing an inverse secondary transformation on the first block according to the determination that the secondary transformation kernel type has a second value.

[0009] According to some embodiments, computing systems such as streaming systems, server systems, personal computer systems, or other electronic devices are provided. The computing system includes a control circuit and a memory for storing one or more instruction sets. One or more instruction sets include instructions for performing any of the methods described herein. In some embodiments, the computing system includes encoder and decoder components (e.g., a transcoder).

[0010] According to some embodiments, a non-temporary computer-readable storage medium is provided. The non-temporary computer-readable storage medium stores one or more instruction sets for execution by a computing system. One or more instruction sets include instructions for performing any of the methods described herein. Thus, devices and systems having methods for encoding and decoding video are disclosed. Such methods, devices, and systems may complement or replace conventional methods, devices, and systems for video encoding / decoding.

[0011] The features and advantages described herein are not necessarily exhaustive. In particular, several additional features and advantages will become apparent to those skilled in the art in light of the drawings, specification and claims provided herein. Furthermore, it should be noted that the language used herein has been selected primarily for readability and educational purposes and is not necessarily selected to define or limit the subject matter described herein. [Brief explanation of the drawing]

[0012] To enable a more detailed understanding of this disclosure, a more specific description can be obtained by referring to the features of various embodiments, some of which are shown in the accompanying drawings. However, the accompanying drawings merely illustrate relevant features of this disclosure and should not be considered limiting, and their description may recognize other valid features that a person skilled in the art would understand by reading this disclosure.

[0013] [Figure 1] This block diagram shows an exemplary communication system in several embodiments.

[0014] [Figure 2A] A directory diagram showing exemplary elements of an encoder component according to several embodiments.

[0015] [Figure 2B] A block diagram showing exemplary elements of a decoder component according to some embodiments.

[0016] [Figure 3] A block diagram showing an exemplary server system according to some embodiments.

[0017] [Figure 4] A block diagram showing an exemplary encoding and decoding process according to some embodiments.

[0018] [Figure 5A] A flowchart showing an exemplary method of encoding video according to some embodiments.

[0019] [Figure 5B] A flowchart showing an exemplary method of decoding video according to some embodiments.

[0020] [Figure 6] A flowchart showing an exemplary coding process according to some embodiments.

[0021] In accordance with common practice, the various features shown in the drawings are not necessarily drawn to scale, and like reference numerals may be used throughout the specification and drawings to indicate like features.

Best Mode for Carrying Out the Invention

[0022] This disclosure describes systems and methods for signaling primary and secondary transforms. In some methods, the parameters of the secondary transform are implicitly derived (based on the encoding context, such as the parameters of the primary transform). In some embodiments, as described in more detail below, secondary transform information is signaled (for example, in a video bitstream), which can improve encoding efficiency (for example, by allowing the most accurate and precise secondary transform to be used). Furthermore, some embodiments described herein reduce encoding overhead (for example, minimize it) by signaling the secondary transform kernel type before signaling the transform set identifier. For example, the system refrains from signaling the transform set identifier in response to the secondary transform kernel type indicating that a secondary transform is not used.

[0023] Exemplary Systems and Devices Figure 1 is a block diagram showing a communication system 100 according to several embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (for example, electronic devices 120-1 to 120-m) that are communicatively coupled to one or more networks. In some embodiments, the communication system 100 is a streaming system for use with video-enabled applications such as video conferencing applications, digital TV applications, and media storage and / or distribution applications.

[0024] Source device 102 includes a video source 104 (e.g., a camera component or media storage) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video bitstreams from the video stream. The video stream from video source 104 may have a higher data volume compared to the encoded video bitstream 108 generated by the encoder component 106. Since the encoded video bitstream 108 has a lower data volume (less data) compared to the video stream from the video source, the encoded video bitstream 108 requires less bandwidth to transmit and less storage space to store compared to the video stream from video source 104. In some embodiments, source device 102 does not include an encoder component 106 (e.g., it is configured to transmit uncompressed video to a network(s) 110).

[0025] One or more networks 110 represent any number of networks that carry information between the source device 102, the server system 112, and / or electronic devices 120, including, for example, wireline and / or wireless communication networks. One or more networks 110 may exchange data over circuit-switched channels and / or packet-switched channels. Typical networks include telecommunications networks, local area networks, wide area networks, and / or the Internet.

[0026] One or more networks 110 include a server system 112 (for example, a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (configured to store and / or deliver video content, for example, an encoded video stream from a source device 102). The server system 112 includes a coder component 114 (configured to encode and / or decode video data, for example). In some embodiments, the coder component 114 includes an encoder component and / or a decoder component. In various embodiments, the coder component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the coder component 114 is configured to decode an encoded video bitstream 108 and re-encode the video data using different encoding standards and / or methods to produce encoded video data 116. In some embodiments, the server system 112 is configured to produce multiple video formats and / or encodings from the encoded video bitstream 108. In some embodiments, the server system 112 functions as a Media-Aware Network Element (MANE). For example, the server system 112 may be configured to prune an encoded video bitstream 108 to tailor potentially different bitstreams for one or more of the electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0027] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to produce an outgoing video stream that can be rendered on a display or other type of rendering device. In some embodiments, one or more of the electronic devices 120 do not include a display component (for example, they include media storage that is communicatively coupled to an external display device). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access a server system 112 to retrieve encoded video data 116.

[0028] The source device and / or the plurality of electronic devices 120 are sometimes referred to as “terminal devices” or “user devices.” In some embodiments, the source device 102 and / or one or more of the electronic devices 120 are instances of a server system, a personal computer, a portable device (e.g., a smartphone, tablet, or laptop), a wearable device, a video conferencing device, and / or other types of electronic devices.

[0029] In an exemplary operation of the communication system 100, source device 102 transmits an encoded video bitstream 108 to server system 112. For example, source device 102 may encode a stream of pictures captured by the source device. Server system 112 receives the encoded video bitstream 108 and may decode and / or encode the encoded video bitstream 108 using coder component 114. For example, server system 112 may apply a more optimal encoding to the video data for network transmission and / or storage. Server system 112 may transmit the encoded video data 116 (e.g., one or more encoded video bitstreams) to one or more electronic devices 120. Each electronic device 120 may decode the encoded video data 116 and optionally display a video picture.

[0030] Figure 2A is a block diagram showing exemplary elements of an encoder component 106 according to several embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., a transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source that is a component of a different device from the encoder component 106). The video source 104 may provide the source video sequence in the form of a digital video sample stream that can be any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601 Y CrCb, or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0 or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. The video data may be provided as a plurality of individual pictures that give motion when viewed sequentially. The pictures themselves may be organized as a spatial array of pixels, each pixel may contain one or more samples, depending on the sampling structure used, color space, etc. Those skilled in the art will readily understand the relationship between pixels and samples. The following description will focus on samples.

[0031] The encoder component 106 is configured to encode and / or compress the pictures of the source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. One function of the controller 204 is to implement an appropriate encoding rate. In some embodiments, the controller 204 controls and is functionally coupled to other functional units, as described below. Parameters set by the controller 204 may include rate control-related parameters (e.g., picture skip, quantizer, and / or lambda values ​​for rate-distortion optimization techniques), picture size, picture group (GOP) layout, and maximum motion vector search range. Those skilled in the art will readily identify the other functions of the controller 204, as they may relate to the encoder component 106, which they are optimized for certain system designs.

[0032] In some embodiments, the encoder component 106 is configured to operate in an encoding loop. In a simplified example, the encoding loop includes a source coder 202 (responsible for creating symbols, such as a symbol stream, based, for example, the input picture to be encoded and a reference picture(s) or more) and a (local) decoder 210. The decoder 210 reconstructs the symbols to create sample data in a similar manner to the (remote) decoder (when the compression between the symbols and the encoded video bitstream is lossless). The reconstructed sample stream (sample data) is input to the reference picture memory 208. Since decoding the symbol stream yields bitwise-accurate results independent of the decoder's location (local or remote), the contents of the reference picture memory 208 are also bitwise-accurate between the local encoder and the remote encoder. In this way, the prediction portion of the encoder interprets the same sample values ​​as reference picture samples that the decoder interprets when using predictions during decoding. This principle of reference picture synchronization (and the resulting drift if synchronization cannot be maintained, for example, due to channel errors) is known to those skilled in the art.

[0033] The operation of decoder 210 may be the same as that of a remote decoder, such as decoder component 122, which will be described in detail below in relation to Figure 2B. However, referring to Figure 2B for the moment, since symbols are available and the encoding / decoding of symbols to the encoded video sequence by the entropy encoder 214 and parser 254 may be reversible, the entropy decoding portion of decoder component 122, including buffer memory 252 and parser 254, may not be fully implemented in local decoder 210.

[0034] With the exception of parse / entropy decoding, the decoder techniques described in this paper can exist in substantially the same functional form in the corresponding encoders. For this reason, the subject matter disclosed will focus on decoder operation. The description of encoder techniques can be omitted, as it is the reverse of the decoder techniques.

[0035] As part of its operation, the source coder 202 can perform motion-compensated predictive coding, which predictively codes the input frame by referencing one or more previously coded frames from a video sequence designated as reference frames. In this way, the coding engine 212 codes the difference between the pixel blocks of the input frame and the pixel blocks of one or more reference frames that may be selected as predictive references for the input frame. The controller 204 can manage the coding operation of the source coder 202, including, for example, setting parameters and subgroup parameters used to encode the video data.

[0036] Decoder 210 decodes the encoded video data of a frame that may be designated as a reference frame, based on the symbols created by source coder 202. The operation of encoding engine 212 can be, advantageously, a lossy process. When the encoded video data is decoded in a video decoder (not shown in Figure 2A), the reconstructed video sequence may be a replica of the source video sequence with some errors. Decoder 210 can replicate the decoding process that may be performed by the remote video decoder on the reference frame and store the reconstructed reference frame in reference picture memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame that has content common to the reconstructed reference frame that would be obtained by the remote video decoder (if there are no transmission errors).

[0037] The predictor 206 can perform a predictive search for the encoding engine 212. That is, for a new frame to be encoded, the predictor 206 can search the reference picture memory 208 for sample data (as candidate criterion pixel blocks) or some metadata such as reference picture motion vectors, block shapes, etc., which can serve as appropriate predictive criteria for the new picture. The predictor 206 may operate sample block pixel block by sample block to find appropriate predictive criteria. As determined by the search results obtained by the predictor 206, the input picture may have predictive references drawn from multiple reference pictures stored in the reference picture memory 208.

[0038] The outputs of all the aforementioned functional units can be entropically coded by the entropy encoder 214. The entropy encoder 214 converts the symbols generated by the various functional units into coded video sequences by lossless compression of the symbols according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding).

[0039] In some embodiments, the output of the entropy encoder 214 is coupled to the transmitter. The transmitter may be configured to buffer the encoded video sequences created by the entropy encoder 214 and prepare them for transmission over a communication channel 218, which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter may be configured to merge the encoded video data from the source coder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown). In some embodiments, the transmitter may transmit additional data along with the encoded video. The source coder 202 may include such data as part of the encoded video sequence. The additional data may include time / space / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, Supplemental Enhancement Information (SEI) messages, Visual Usability Information (VUI) parameter set fragments, and the like.

[0040] The controller 204 can manage the operation of the encoder component 106. During encoding, the controller 204 can assign each encoded picture to a type of encoded picture that may affect the encoding technique applied to each picture. For example, a picture may be assigned as an intra-picture (I-picture), a predictive picture (P-picture), or a bidirectional predictive picture (B-picture). An intra-picture can be encoded and decoded without using other frames in the sequence as a source for prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will be familiar with these variations of I-pictures, as well as their respective uses and characteristics, and therefore they will not be repeated here. A predictive picture can be encoded and decoded using intra-prediction or inter-prediction, which uses at most one motion vector and a reference index to predict the sample value of each block. A bidirectional predictive picture can be encoded and decoded using intra-prediction or inter-prediction, which uses at most two motion vectors and reference indices to predict the sample value of each block. Similarly, multiple predictive pictures can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0041] A source picture can generally be spatially subdivided into multiple sample blocks (for example, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and encoded block by block. Blocks can be predictively encoded by referencing other (already encoded) blocks, as determined by the encoding assignment applied to each picture in the block. For example, a block of picture I may be encoded unpredictably or predictively by referencing already encoded blocks of the same picture (spatial prediction or intra-prediction). A pixel block of picture P may be encoded unpredictably, via spatial prediction, or via temporal prediction by referencing one previously encoded reference picture. A block of picture B may be encoded unpredictably, via spatial prediction, or via temporal prediction by referencing one or two previously encoded reference pictures.

[0042] Video can be captured as multiple source pictures (video pictures) in a time series. Intra-picture prediction (often abbreviated as intra-prediction) utilizes spatial correlations within a given picture, while inter-picture prediction utilizes (temporal or other) correlations between pictures. In one example, a particular picture being encoded / decoded, called the current picture, is partitioned into blocks. When a block in the current picture is analogous to a reference block in a previously encoded and still-buffering reference picture in the video, the block in the current picture can be encoded by a vector called a motion vector. The motion vector points to a reference block in the reference picture and may have a third dimension to identify the reference picture if multiple reference pictures are used.

[0043] The encoder component 106 may perform encoding operations in accordance with a given video encoding technique or standard, such as any of those described herein. In these operations, the encoder component 106 may perform various compression operations, including predictive encoding operations that utilize temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0044] Figure 2B is a block diagram showing exemplary elements of a decoder component 122 according to several embodiments. The decoder component 122 in Figure 2B is coupled to channel 218 and display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to a loop filter 256 and configured to transmit data to display 124 (for example, via a wired or wireless connection).

[0045] In some embodiments, the decoder component 122 includes a receiver coupled to channel 218 and configured to receive data from channel 218 (for example, via a wired or wireless connection). The receiver may be configured to receive one or more encoded video sequences to be decoded by the decoder component 122. In some embodiments, the decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence may be received from channel 218, and channel 210 may be a hardware / software link to a storage device that stores encoded video data. The receiver may receive encoded video data together with other data, such as encoded audio data and / or auxiliary data streams that may be transferred to their respective use entities (not shown). The receiver may isolate the encoded video sequence from such other data. In some embodiments, the receiver receives additional (redundant) data along with the encoded video. The additional data may be included as part of one or more encoded video sequences. Additional data may be used by decoder component 122 to decode the data and / or to more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0046] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes called an entropy decoder), a scaler / inverse unit 258, an intra-picture prediction unit 262, a motion compensation prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, the decoder component 122 is implemented as one or more integrated circuits and / or other electronic circuits. In some embodiments, the decoder component 122 is implemented at least partially in software.

[0047] Buffer memory 252 is coupled between channel 218 and parser 254 (for example, to address network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, in addition to buffer memory 252 within decoder component 122 (configured, for example, to handle playout timing), a separate buffer memory is provided outside decoder component 122 (for example, to address network jitter). When receiving data from a storage / transfer device with sufficient bandwidth and controllability, or from an isochronous network, buffer memory 252 may not be necessary or may be small. For use over best-effort packet networks such as the Internet, buffer memory 252 may be required, may be relatively large, may be advantageously adaptively sized, and may be implemented at least partially in an operating system or similar element (not shown) outside decoder component 122.

[0048] The parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. The symbols may include, for example, information used to manage the operation of the decoder component 122, and / or information for controlling rendering devices such as the display 124. The control information for rendering devices may be in the form of, for example, Supplemental Improvement Information (SEI) messages or video usability information (VUI) parameter set fragments (not shown). The parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may follow video coding techniques or standards and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, context-sensitive or non-context-sensitive arithmetic coding, etc. The parser 254 may extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder, based on at least one parameter corresponding to a group. Subgroups can include picture groups (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transformation units (TU), and prediction units (PU). Parser 254 can also extract information such as transformation coefficients, quantizer parameter values, and motion vectors from the coded video sequence.

[0049] The reconstruction of symbol 270 can involve multiple different units, depending on the type of encoded video picture or part thereof (interpicture and intrapicture, interblock and intrablock, etc.) and other factors. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the encoded video sequence by parser 254. The flow of such subgroup control information between parser 254 and the following multiple units is not illustrated for clarity.

[0050] The decoder component 122 can be conceptually subdivided into numerous functional units, and in some implementations, these units can interact closely with each other and be at least partially integrated. However, for clarity, the conceptual subdivision into functional units is maintained in this paper.

[0051] The scaler / inverse unit 258 receives quantized transformation coefficients, as well as control information (such as which transformation to use, block size, quantization factor, and / or quantization scaling matrix) from the parser 254 as symbols (singular or plural) 270. The scaler / inverse unit 258 can output a block containing sample values ​​that can be input to the aggregator 268.

[0052] In some cases, the output samples of the scaler / inverse unit 258 relate to intra-encoded blocks, i.e., blocks that do not use predictive information from previously reconstructed pictures, but can use predictive information from previously reconstructed portions of the current picture. Such predictive information may be provided by the intra-picture predictive unit 262. The intra-picture predictive unit 262 may generate blocks of the same size and shape as the block being reconstructed, using already reconstructed peripheral information taken from the current (partially reconstructed) picture from the current picture memory 264. The aggregator 268 may, sample by sample, add the predictive information generated by the intra-picture predictive unit 262 to the output sample information provided by the scaler / inverse unit 258.

[0053] In other cases, the output samples of the scaler / inverse unit 258 relate to intercoded and potentially motion-compensated blocks. In such cases, the motion-compensated prediction unit 260 can access the reference picture memory 266 to fetch samples to be used for prediction. After motion-compensating the fetched samples according to the symbol 270 relating to that block, these samples can be added by the aggregator 268 to the output of the scaler / inverse unit 258 (in this case, called residual samples or residual signals) to generate output sample information. The address in the reference picture memory 266 from which the motion-compensated prediction unit 260 fetches prediction samples may be controlled by a motion vector. The motion vector may be available to the motion-compensated prediction unit 260 in the form of a symbol 270 which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values ​​fetched from the reference picture memory 266 when an accurate motion vector is used in sub-sample units, a motion vector prediction mechanism, etc.

[0054] The output samples of the aggregator 268 can follow various loop filtering techniques in the loop filter unit 256. The in-loop filtering techniques that the video compression technique may include are controlled by parameters contained in the encoded video bitstream and made available to the loop filter unit 256 as symbols 270 from the parser 254, but can also respond to metadata obtained during decoding of earlier portions of the encoded picture or encoded video sequence (in decoding order), and can also respond to previously reconstructed and loop-filtered sample values. The output of the loop filter unit 256 may be a sample stream that is output to a rendering device such as the display 124, and can also be stored in the reference picture memory 266 for use in future picture-to-picture predictions.

[0055] Certain encoded pictures, once reconstructed, can be used as reference pictures for future predictions. Once an encoded picture is reconstructed and identified as a reference picture (for example, by parser 254), the current reference picture can become part of reference picture memory 266, and fresh current picture memory can be reallocated before initiating the reconstruction of subsequent encoded pictures.

[0056] The decoder component 122 can perform decoding operations according to a predetermined video compression technique that may be documented in a standard, such as one of the standards described herein. The encoded video sequence may conform to the syntax specified by the video compression technique or standard being used, in the sense that it conforms to the syntax of the video compression technique or standard, as specified in the video compression technique documentation or standard, particularly the profile documentation therein. Also, in order to conform to some video compression technique or standard, the complexity of the encoded video sequence may be within a range defined by the level of the video compression technique or standard. In some cases, the level may constrain the maximum picture size, maximum frame rate, maximum reconstruction sample rate (for example, measured in megasamples per second), maximum reference picture size, etc. The limitations set by the level may, in some cases, be further constrained through the Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management that is signaled in the encoded video sequence.

[0057] Figure 3 is a block diagram showing a server system 112 according to several embodiments. The server system 112 includes a control circuit 302, one or more network interfaces 304, memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., a CPU, GPU, and / or DPU). In some embodiments, the control circuit includes one or more field-programmable gate arrays (FPGAs), hardware accelerators, and / or integrated circuits (e.g., application-specific integrated circuits).

[0058] The network interface 304 may be configured to interface with one or more communication networks (e.g., wireless, wireline, and / or optical networks). Communication networks may be local, wide-area, metropolitan, automotive, and industrial, real-time, latency-tolerant, etc. Examples of communication networks include local area networks such as Ethernet®, cellular networks including Wi-Fi, GSM®, 3G, 4G, 5G, LTE, etc., wired or wireless wide-area digital networks including cable TV, satellite TV, and terrestrial broadcast TV, and automotive and industrial networks including CANBus. Such communications may be one-way, receive-only (e.g., broadcast TV), transmit-only (e.g., CANbus to certain CANbus devices), or bidirectional (e.g., to other computer systems using local or wide-area digital networks). Such communications may include communications to one or more cloud computing networks.

[0059] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input devices 310 may include one or more of the following: a keyboard, mouse, trackpad, touchscreen, data glove, joystick, microphone, scanner, camera, etc. The output devices 308 may include one or more of the following: an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0060] Memory 314 may include high-speed random-access memory (such as DRAM, SRAM, DDR RAM, and / or other random-access solid-state memory devices) and / or non-volatile memory (such as one or more magnetic storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). Memory 314 optionally includes one or more storage devices located away from the control circuit 302. Memory 314, or alternatively, the non-volatile solid-state memory devices within Memory 314, includes a non-temporary computer-readable storage medium. In some embodiments, Memory 314, or the non-temporary computer-readable storage medium of Memory 314, stores the following programs, modules, instructions, and data structures, or subsets or supersets thereof: ● An operating system that handles various basic system services and includes procedures for performing hardware-dependent tasks.316 ● A network communication module 318 used to connect the server system 112 to other computing devices via one or more network interfaces 304 (for example, via wired and / or wireless connections); ● Encoding module 320 for performing various functions related to encoding and / or decoding data such as video data. In some embodiments, the encoding module 320 is an instance of the coder component 114. The encoding module 320 is, but is not limited to: Decode module 322 performs various functions related to decoding encoded data, such as those previously described regarding decoder component 122; ○ Encoding module 340 for performing various functions related to data encoding, such as those previously described regarding encoder component 106. Includes one or more of the following; ● A picture memory 352 for storing pictures and picture data, for use with, for example, the encoding module 320. In some embodiments, the picture memory 352 includes one or more of the reference picture memory 208, buffer memory 252, current picture memory 264, and reference picture memory 266.

[0061] In some embodiments, the decode module 322 includes a parse module 324 (configured to perform various functions previously described with respect to, for example, the parser 254), a transform module 326 (configured to perform various functions previously described with respect to, for example, the scalar / inverse transform unit 258), a predictive module 328 (configured to perform various functions previously described with respect to, for example, the motion compensation predictive unit 260 and / or the in-picture predictive unit 262), and a filter module 330 (configured to perform various functions previously described with respect to, for example, the loop filter 256).

[0062] In some embodiments, the encoding module 340 includes a code module 342 (configured to perform various functions previously described with respect to, for example, the source coder 202 and / or the encoding engine 212) and a prediction module 344 (configured to perform various functions previously described with respect to, for example, the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 include a subset of the modules shown in Figure 3. For example, a shared prediction module is used by both the decoding module 322 and the encoding module 340.

[0063] Each of the identified modules stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., sets of instructions) do not need to be implemented as separate software programs, procedures, or modules; therefore, various subsets of these modules may be combined or otherwise rearranged in various embodiments. For example, the encoding module 320 may optionally not include separate decoding and encoding modules, but rather use the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the modules and data structures identified above. In some embodiments, memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0064] Figure 3 shows server system 112 in several embodiments, but Figure 3 is not a schematic diagram of the structures of the embodiments described herein, but is intended as a functional description of the various features that may be present in one or more server systems. In practice, as will be recognized by those skilled in the art, items shown separately may be combined, and some items may be separated. For example, some items shown separately in Figure 3 may be implemented on a single server, and a single item may be implemented by one or more servers. The actual number of servers used to implement server system 112, and how functions are allocated among them, will vary from implementation to implementation and optionally depend in part on the amount of data traffic the server system handles during peak usage periods and average usage periods.

[0065] Examples of coding processes and techniques As discussed above, video codecs broadly include several techniques, such as intra / interpretation, image transformation, quantization, coefficient coding, entropy coding, and / or in-loop filtering. In particular, decorrelating prediction error (e.g., residual) signals is beneficial. This disclosure describes techniques for signaling and applying quadratic transformations. Primary and / or quadratic transformation signaling may be implicit or explicit (e.g., explicitly signaled in the bitstream). The techniques described herein may be used separately or combined in any order. Furthermore, each of the techniques may be implemented by processing circuits (e.g., one or more processors or one or more integrated circuits).

[0066] Quadratic transformations can be applied between the linear transformation component and the quantization component, as shown in Figure 4. For example, a forward quadratic transformation is applied between the linear forward transformation component and the quantization component, and an inverse quadratic transformation is applied between the dequantization component and the linear inverse transformation component.

[0067] Figure 4 is a block diagram illustrating exemplary encoding and decoding processes according to several embodiments. As shown in Figure 4, the prediction error 402 (e.g., residual) is transformed and then quantized to generate conversion coefficients. A linear forward transform 404 (sometimes called a linear transform) is applied to the prediction error 402. According to some embodiments, a quadratic forward transform 406 (sometimes called a quadratic transform) is applied to the output of the linear forward transform 404. Quantization 408 is applied to the output of the quadratic forward transform 406 to generate conversion coefficients. In some embodiments, the conversion coefficients are signaled in the video bitstream.

[0068] The dequantization 410 is applied to the conversion coefficients (for example, in a decoder). The inverse quadratic transform 412 is applied to the output of the dequantization 410 (for example, to reverse the effect of the quadratic forward transform 406). The inverse linear transform 414 is applied to the output of the inverse quadratic transform 412 (for example, to reverse the effect of the linear forward transform 404). In this way, the decoding prediction error 416 is obtained from the conversion coefficients.

[0069] As used herein, a primary transformation set and / or a secondary transformation set (sometimes referred to as a "transformation set" or "set") refers to a group of transformation kernels. For example, an optimal transformation set may be determined for each transformation block (during the encoding process). A secondary transformation set may be identified during the encoding process if the encoder (or transcoder) determines that a secondary transformation should be applied.

[0070] As used herein, a primary and / or secondary transformation kernel type (sometimes referred to as a "kernel type") refers to an option for a transformation kernel type within each transformation set. For example, a primary and / or secondary transformation set may contain one or more primary and / or secondary transformation kernel types. As an example, a set and kernel hierarchy may contain seven sets, each set having four kernel types. In this example, one of the kernel types may indicate that there are no primary and / or secondary transformations. For example, among the four kernel type indices (0, 1, 2, and 3), the system may define kernel type index 0 to indicate that there are no primary and / or secondary transformations, and thus the transformation set is not signaled. The encoder component can select a transformation set identifier (ID) and a kernel type identifier (ID).

[0071] As used herein, a set context index (sometimes referred to as “set context”) refers to a (nominal) set context index that may be derived from an encoding context (for example, as part of standardization) and may be predefined. The “encoding context” may include information about previously coded modes (such as intra-predictive mode or inter-predictive mode), primary transform type, encoded block size, and / or transform block size.

[0072] As used herein, “context_to_probable_set” refers to a predefined mapping between a set context index and a probable transformation set ID (e.g., the most probable transformation set ID). For example, one or more most probable sets may be identified for a given encoding context, which may be defined by a video standard. As an example, 12 exemplary intra-prediction modes are mapped to set IDs having the range [0, ..., 6], as shown in code snippet 1 below (e.g., a 12-symbol to 7-symbol mapping).

[0073] The following code snippet 1 shows an exemplary context_to_probable_set[i], where the index is derived from an exemplary context grouping (e.g., consisting of exemplary intra-predictive modes). In the example of code snippet 1, the two intra-predictive modes (INTRA_PRED_1 and INTRA_PRED_2) are in the same group that maps to the same set ID 1. context_to_probable_set

[12] ={ 0, / / INTRA_PRED_0 1, / / INTRA_PRED_1 1, / / INTRA_PRED_2 2, / / INTRA_PRED_3 3, / / INTRA_PRED_4 4, / / INTRA_PRED_5 4, / / INTRA_PRED_6 5, / / INTRA_PRED_7 5, / / INTRA_PRED_8 6, / / INTRA_PRED_9 1, / / INTRA_PRED_10 1, / / INTRA_PRED_11 } Code Snippet 1 — Exemplary Mapping Table

[0074] The index for the mapping table in code snippet 1 is defined in code snippet 2 and may be intra-prediction IDs [0, ..., 11], as shown below. The range of the index may vary, for example, based on which coding context the context_to_probable_set mapping table uses. For example, code snippet 2 shows an exemplary coding context resulting from an exemplary intra-prediction mode. This coding context may form an index for a mapping function or table (e.g., context_to_probable_set[i]) for deriving a set context, which may determine the most probable set of transformations for each unique coding context. enum{ INTRA_PRED_0 / / , for example, DC_PRED INTRA_PRED_1 / / For example, V_PRED INTRA_PRED_2 / / , for example, H_PRED INTRA_PRED_3 / / For example, D45_PRED INTRA_PRED_4 / / , for example, D135_PRED INTRA_PRED_5 / / , for example, D113_PRED INTRA_PRED_6 / / , for example, D157_PRED INTRA_PRED_7 / / , for example, D203_PRED INTRA_PRED_8 / / , for example, D67_PRED INTRA_PRED_9 / / , for example, SMOOTH_PRED INTRA_PRED_10 / / , for example, SMOOTH_V_PRED INTRA_PRED_11 / / , for example, SMOOTH_H_PRED PREDICTION_MODE Code Snippet 2 — Exemplary Intra Predictive ID

[0075] In some embodiments, an intra secondary transform (IST) coding tool is used to implicitly derive a set of secondary transforms based on the context of the intra prediction direction (for example, by explicitly signaling the kernel types in the set). Table 1 below shows an exemplary mapping from intra prediction modes and primary transform types to (IST) indices. [Table 1]

[0076] In some embodiments, the quadratic transformation set is explicitly signaled. Explicitly signaling the quadratic transformation set allows any quadratic transformation set to be selected for the block, regardless of the intra-prediction mode applied. The following code snippet 3 shows exemplary transformation type syntax. [Table 2]

[0077] In the example of code snippet 3, the stx_set parameter is inserted after the stx_type parameter (for example, when stx_type is set to a specific value (e.g., 0), there is no quadratic transformation and therefore stx_set does not need to be coded, signaled, or parsed). In some embodiments, there are 14 quadratic transformation sets. In some embodiments, the quadratic transformation sets depend on the primary transformation type (e.g., two primary transformation types). Therefore, in some embodiments, seven different sets need to be signaled for the primary transformation type. In some embodiments, a probabilistic context is selected for each set. In some embodiments, the probabilistic context is derived from the intra-predictive mode.

[0078] The following code snippet 4 shows an exemplary quadratic transformation definition. / / Intra-secondary conversion #define IST_SET_SIZE 14 / / IST kernel set size #if CONFIG_IST_SET_FLAG / / Number of direction groups in the IST kernel #define IST_DIR_SIZE 7 / / (IST_SET_SIZE>>1) #endif / / CONFIG_IST_SET_FLAG Code Snippet 4 - Exemplary Secondary Transformation Definition

[0079] The following code snippet 5 shows an example of code for reading the secondary transformation type. void av1_read_sec_tx_type() ... uint8_t intra_mode=mbmi->mode; uint8_t stx_set_ctx=stx_transpose_mapping[intra_mode]; push(stx_set_ctx <IST_DIR_SIZE); uint16_t stx_set_flag=aom_read_symbol(r,ec_ctx->stx_set_cdf[stx_set_ctx],IST_DIR_SIZE,ACCT_INFO("stx_set_flag")); assert(stx_set_flag <IST_DIR_SIZE); ... } Code Snippet 5—An Exemplary Function for Reading Quadratic Transformations

[0080] In some embodiments, the stx_set_ctx parameter is derived from the intra-prediction mode of the transformation block being decoded. The following code snippet 6 shows another exemplary quadratic transformation mapping array (for example, an intra-mode mapping to an IST kernel set). static const uint8_t stx_transpose_mapping[INTRA_MODES-1]={ / / Array index: 0, / / DC_PRED 1, / / V_PRED 1, / / H_PRED 2, / / D45_PRED 3, / / D135_PRED 4, / / D113_PRED 4, / / D157_PRED 5, / / D203_PRED 6, / / SMOOTH_PRED 1, / / SMOOTH_V_PRED 1, / / SMOOTH_H_PRED } Code Snippet 6 — Exemplary Quadratic Transform Mapping Array

[0081] In some embodiments, the quadratic transformation is enabled only for a subset of intra-modes (for example, intra-modes smaller than PEATH_PRED). Thus, the array size for stx_transpose_mapping is set to INTRA_MODES-1.

[0082] A cumulative distribution function (CDF) can be used to represent the probability that a random variable can take a value below a certain threshold. For example, the CDF for a video codec might represent 32768, which is a multiple of the probability that a symbol has a value below a given level. An exemplary CDF with alphabet size 7 is shown in the following code snippet 7. static const aom_cdf_prob default_stx_set_cdf[IST_DIR_SIZE][CDF_SIZE(IST_DIR_SIZE)]={ {AOM_CDF7(32744, 32748, 32756, 32760, 32764)}, {AOM_CDF7(4, 32748, 32752, 32756, 32760, 32764)}, {AOM_CDF7(4, 8, 32752, 32756, 32760, 32764)}, {AOM_CDF7(4, 8, 12, 32756, 32760, 32764)}, {AOM_CDF7(4, 8, 12, 16, 32760, 32764)}, {AOM_CDF7(4, 8, 12, 16, 20, 32764)}, {AOM_CDF7(4, 8, 12, 16, 20, 24)}, }; Code Snippet 7 — Exemplary CDF

[0083] In code snippet 7, IST_DIR_SIZE may be defined as 7, and the size of the CDF table is 112 bytes (e.g., 7 contexts × 8 (7 symbols + 1 counter) × 2 (bytes per number)). For example, the number of stx_set_ctx contexts can be 7, as defined by #define IST_DIR_SIZE 7 in code snippet 4 above. In some embodiments, the CDF probability model is set up so that the most likely set symbol has the largest possible range size. For example, for a CDF model with a maximum range of 2^15, the largest possible range size is 2^15-6*4, where 6 is for the number of less likely set symbols and 4 is for the smallest CDF range. In some embodiments, the smallest range is defined as (#define EC_MIN_PROB 4). Thus, the probability represented by the CDF model is (2^15-(6*4)) / 2^15, which is approximately 0.9994.

[0084] The following code snippet 8 shows an exemplary configuration for a quadratic transformation. #if CONFIG_IST_SET_FLAG if (stx_flag > 0) { uint8_t is_adst_adst = get_primary_tx_type(*tx_type) == ADST_ADST ? 1 : 0; uint8_t intra_mode = mbmi->mode; uint8_t stx_set_ctx = stx_transpose_mapping[intra_mode]; assert(stx_set_ctx < IST_DIR_SIZE); uint16_t stx_set_flag = aom_read_symbol(r, ec_ctx->stx_set_cdf[stx_set_ctx], IST_DIR_SIZE, ACCT_INFO("stx_set_flag")); assert(stx_set_flag < IST_DIR_SIZE); if (is_adst_adst) stx_set_flag += IST_DIR_SIZE; *tx_type |= (stx_set_flag << (4+2)); } #endif / / CONFIG_IST_SET_FLAG Code Snippet 8 - Exemplary Secondary Transformation Configuration

[0085] Figure 5A is a flowchart illustrating a method 500 for encoding video according to several embodiments. The method 500 may be performed in a computing system (e.g., a server system 112, a source device 102, or an electronic device 120) having a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, the method 500 is performed by executing instructions stored in the memory of the computing system (e.g., memory 314).

[0086] The system receives video data containing multiple blocks, including a first block, from (for example, video source 104) (502). The system determines a quadratic transformation kernel type value for the first block (504). The system transmits a first syntax element in the video bitstream (506), the first syntax element indicating the quadratic transformation kernel type value. In some embodiments, the system performs a primary transformation (for example, a linear forward transformation 404) on the first block.

[0087] In accordance with the determination that the secondary transformation kernel type has a first value (508), the system determines a secondary transformation set identifier (510), transmits a second syntax element in the video bitstream indicating the secondary transformation set identifier (512), and performs a secondary transformation (e.g., a quadratic forward transformation 406) on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier (514). For example, the system performs a secondary transformation on the output of the primary transformation applied to the first block.

[0088] Following the decision (516) that the secondary transformation kernel type has a second value, the system refrains from sending a second syntax element (518) and refrains from performing the secondary transformation on the first block (520).

[0089] Figure 5B is a flowchart illustrating a method 550 for decoding video according to several embodiments. The method 550 may be performed in a computing system (e.g., a server system 112, a source device 102, or an electronic device 120) having a control circuit and a memory storing instructions for execution by the control circuit. In some embodiments, the method 550 is performed by executing instructions stored in the memory of the computing system (e.g., memory 314).

[0090] The system receives video data from a video bitstream (e.g., an encoded video sequence) which includes multiple blocks, including a first block, and a first syntax element (e.g., indicating stx_flag) (552). The system determines a secondary transformation kernel type value for the first block based on the first syntax element (554).

[0091] Following the determination that the quadratic transformation kernel type has a first value (556), the system determines a quadratic transformation set identifier based on a second syntax element from the video bitstream (e.g., indicating stx_set_flag) (558), and performs an inverse quadratic transformation (e.g., inverse quadratic transformation 412) on the first block using the determined quadratic transformation kernel type and the determined quadratic transformation set identifier (560). The inverse quadratic transformation corresponds to the quadratic transformation (e.g., quadratic forward transformation 406) performed during the encoding of the video data.

[0092] Following the decision that the quadratic transformation kernel type has a second value, the system refrains from performing an inverse quadratic transformation on the first block (562).

[0093] Figures 5A and 5B show several logical stages in a specific order, but the order-independent stages may be rearranged, and the other stages may be combined or separated. Several reordering or other groupings not specifically mentioned will be obvious to those skilled in the art, and therefore the ordering and groupings presented herein are not exhaustive. Furthermore, it should be recognized that these stages may be implemented in hardware, firmware, software, or any combination thereof.

[0094] In some embodiments, the selected transformation set and / or kernel type are signaled (e.g., within the set) by encoding the primary and / or secondary transformation set ID and / or secondary kernel type. For example, a decoding technique includes determining a secondary transformation kernel type value for a first block based on a first syntax element. In this example, in response to the secondary transformation kernel type having a first value (e.g., non-zero), a secondary transformation set identifier is determined based on a second syntax element from the video bitstream, and an inverse quadratic transformation is performed on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier.

[0095] In some embodiments, the kernel type is signaled first, followed by the conversion set. In some embodiments, the primary conversion type includes at least one inseparable conversion. In some embodiments, the secondary conversion type includes at least one inseparable conversion. In some embodiments, the conversion set includes a mixture of primary-only and primary and secondary conversion combination types. In some embodiments, the kernel types for primary and secondary conversions are signaled congruently. In some embodiments, the kernel sets for primary and secondary conversions are signaled congruently. In some embodiments, signaling the kernel type also signals whether primary and / or secondary conversions are used. In some embodiments, if the kernel type signals that primary and / or secondary conversions are disabled, the conversion set ID is not signaled.

[0096] In some embodiments, a flag is first signaled to indicate whether a primary and / or secondary transformation is applied, and when the flag is signaled with a value indicating that a primary and / or secondary transformation is applied, a kernel type is further signaled to indicate which of the transformation kernels is used, and this kernel type does not include the option of not applying a primary and / or secondary transformation. In some embodiments, the method of mapping a set context to a transformation set ID is derived from any possible coding context. In some embodiments, the coding context includes any possible combination or combination thereof of previously coded modes, such as an intra-prediction mode, or a primary transformation type, or a primary transformation set ID, or a secondary transformation type, or a secondary transformation set ID, or a coded block size, or a transformed block size.

[0097] In some embodiments, grouping of encoded texts is predefined to form an index for a mapping function or table context_to_probable_set. In some embodiments, the most probable set is derived from the encoded contexts predefined and implemented in the mapping function or table. In some embodiments, a particular set of primary and secondary transform kernel types is selected from transform kernels designed or already available to work best for a given group of encoded contexts, for example, to decorrelate best or have the best video compression efficiency. In some embodiments, for each predefined group of encoded contexts, there is currently one or a subgroup of transform sets that may be applicable for the block.

[0098] In some embodiments, the signaling of set IDs is performed by arithmetic coding. In some embodiments, the default or initial probability model (e.g., CDF) for the arithmetic coding of the most likely quadratic transformed set ID is set as approximately 100% probability, or as high as possible acceptable in the video coding system. In some embodiments, the default or initial probability model (e.g., CDF) for the arithmetic coding of the remaining less likely quadratic transformed set IDs is set as approximately 0% probability, or as low as possible acceptable in the video coding system. The approximately 0% probability is achieved by associating a minimum range value (e.g., 4) for the symbol representing one of these remaining less likely quadratic transformed set IDs. In various embodiments, the default probability model (e.g., CDF) is adaptive or non-adaptive. In some embodiments, if the video coding system can select adaptive or non-adaptive mode for entropy coding of set symbols, the non-adaptive mode is selected, for example, to spend the fewest possible bits to code the set symbol.

[0099] In some embodiments, the signaling of set IDs may be performed by variable-length coding (also known as VLC) or Huffman coding. In some embodiments, the codeword length of the most likely transformation set is as short as possible. In some embodiments, the codeword length of the less likely transformation sets is longer than the codeword length of the most likely set.

[0100] In some embodiments, signaling a quadratic transformation begins with a binary flag (e.g., most_probable_flag) indicating whether a set is most likely or not. For example, Figure 6 shows a check (602) to determine if a set is the most likely set. As shown in Figure 6, if the flag is set to false (e.g., 0), the most likely symbols are not encoded (606). If the flag is set to true (e.g., 1), the most likely set is encoded (604). For example, one might consider only one most likely transformation set and encode the set using most_probable_flag.

[0101] In some embodiments, when the most probable flag is true, the set symbols available for coding are all the symbols in the set alphabet. In some embodiments, when the most probable flag is false, the set symbols available for coding are all the symbols in the set alphabet, excluding the most probable set determined by the set context (e.g., a mapping function or the table context_to_probable_set).

[0102] In some embodiments, if the most likely flag is true and the set symbol to be encoded is most likely, the set symbol is encoded using a probably model that has the highest possible probability for the set symbol. In some embodiments, if the most likely flag is false and the set symbol to be encoded is less likely, the set symbol is encoded using a probably model that starts with uniform default probabilities.

[0103] In some embodiments, to encode a single encoding block or transform block, one or more sets of the most likely transform sets are derived based on a predefined encoding context (e.g., intra-predictive mode), and a most likely set index is further signaled to indicate which of the transform sets is selected when the most likely flag is true. When the most likely flag is false, a remaining_set_index is further signaled to indicate which of the non-most likely transform sets is selected. In some embodiments, the most likely flag is context-encoded using arithmetic coding.

[0104] In some embodiments, a list of secondary or primary transformation sets is derived for each transformation block. Previously parsed information may be used to order the secondary or primary transformation sets in the list. In one example, the most likely secondary or primary transformation set is at the top of the list. The index to the set may be signaled using bypass, golom, or arithmetic coding. An offset may be further signaled using bypass, golom, or arithmetic coding to indicate the kernel type.

[0105] (A1) In one aspect, some embodiments include a method for video encoding (e.g., method 500). In some embodiments, the method is performed in a computing system having memory and control circuits (e.g., server system 112). In some embodiments, the method is performed in a coding module (e.g., coding module 320). In some embodiments, the method is performed in a source coding component (e.g., source coder 202), a coding engine (e.g., coding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes the steps of (i) receiving video data containing a plurality of blocks, including a first block (for example, from video source 104); (ii) determining a secondary transformation kernel type value for the first block; (iii) transmitting a first syntax element in the video bitstream indicating the secondary transformation kernel type value; (iv) in accordance with the determination that the secondary transformation kernel type has a first value (for example, 1), (a) determining a secondary transformation set identifier, (b) transmitting a second syntax element in the video bitstream indicating the secondary transformation set identifier, and (c) performing a secondary transformation on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier; and (v) in accordance with the determination that the secondary transformation kernel type has a second value (for example, 0), (a) refraining from transmitting a second syntax element, and (b) refraining from performing a secondary transformation on the first block.

[0106] (A2) In some embodiments of A1, the method further includes signaling a first block within a video bitstream. For example, conversion coefficients from a quadratic transformation are signaled within the video bitstream. As an example, the output of quantization 408 is signaled.

[0107] (A3) In some embodiments of A1 or A2, the method further comprises the steps of (i) determining a linear transformation for a first block; and (ii) transmitting a third syntax element in a video bitstream, where the third syntax element indicates a linear transformation (e.g., a linear forward transformation 404).

[0108] (A4) In some embodiments of A1 to A3, the secondary transformation is selected from the transformation set based on the determined secondary transformation set identifier.

[0109] (A5) In some embodiments of A1 to A4, the method further includes signaling a secondary transformation kernel type congruently with a primary transformation kernel type.

[0110] (A6) In some embodiments of A1 to A5, the method further includes constitutively signaling a set of secondary transformation kernels in a video bitstream together with a set of primary transformation kernels.

[0111] (A7) In some embodiments of A1 to A6, the method further includes signaling a flag in the video bitstream, the flag indicating whether a primary and / or secondary transformation is applied.

[0112] (A8) In some embodiments of A1 to A7, the second syntax element is arithmetically coded.

[0113] (B1) In another aspect, some embodiments include a method for video decoding (e.g., method 550). In some embodiments, the method is performed in a computing system having memory and control circuits (e.g., server system 112). In some embodiments, the method is performed in a coding module (e.g., coding module 320). In some embodiments, the method is performed in a parser (e.g., parser 254), a motion prediction component (e.g., motion compensation prediction unit 260), and / or an intra-prediction component (e.g., intra-picture prediction unit 262). The method includes (i) receiving video data (e.g., an encoded video sequence) from a video bitstream (e.g., a video bitstream of A1) having a plurality of blocks including a first block and a first syntax element; (ii) determining a quadratic kernel type value for the first block based on the first syntax element; (iii) in accordance with the determination that the quadratic kernel type has a first value (e.g., 1), (a) determining a quadratic set identifier based on a second syntax element from the video bitstream, and (b) performing an inverse quadratic transform (e.g., inverse quadratic transform 412) on the first block using the determined quadratic kernel type and the determined quadratic set identifier; and (iv) refraining from performing an inverse quadratic transform on the first block in accordance with the determination that the quadratic kernel type has a second value (e.g., 0). For example, the selected transformation set and / or kernel type (within the set) are explicitly signaled by encoding the primary and / or secondary transformation set ID and / or secondary kernel type. In some embodiments, the kernel type is signaled first, followed by the transformation set. As an example, signaling the kernel type can also signal whether primary and / or secondary transformations are used. In some embodiments, the inverse secondary transformation corresponds to the secondary transformation performed during the encoding of the video data.

[0114] (B2) In some embodiments of B1, the second syntax element is not included in the video bitstream, in accordance with the secondary transformation kernel type having a second value. For example, if the kernel type signals that the primary and / or secondary transformations are disabled, the corresponding transformation set ID is not signaled.

[0115] (B3) In some embodiments of B1 or B2, the method further includes determining a linear transformation (and / or an inverse linear transformation corresponding to the linear transformation) for a first block based on a third syntax element from a video bitstream.

[0116] (B4) In some embodiments of B3, the linear transformation is an inseparable transformation. For example, the linear transformation type includes at least one inseparable transformation.

[0117] (B5) In some embodiments of B1 to B4, the quadratic transformation is an inseparable transformation. For example, the quadratic transformation type includes at least one inseparable transformation.

[0118] (B6) In some embodiments of B1 to B5, the secondary transformation is selected from a set of transformations based on a determined secondary transformation set identifier.

[0119] (B7) In some embodiments of B6, the conversion set includes one or more primary-only conversion types and one or more primary and secondary conversion types. For example, the conversion set may include a mixture of primary-only and primary-+secondary conversion combination types.

[0120] (B8) In some embodiments of B1 to B7, the secondary conversion kernel type is signaled congruently in the video bitstream together with the primary conversion kernel type. For example, the kernel types for the primary and secondary conversions are signaled congruently.

[0121] (B9) In some embodiments of B1 to B8, the method further includes determining a kernel set for secondary conversion based on information from the video bitstream.

[0122] In some embodiments of (B10)B9, the kernel set for the secondary transformation is signaled congruently in the video bitstream together with the kernel set for the primary transformation. For example, the kernel sets for the primary and secondary transformations are signaled congruently.

[0123] (B11) In some embodiments of B1 to B10, (i) a flag indicating whether a primary and / or secondary transformation is applied is signaled in the video bitstream, and (ii) a secondary transformation kernel type is signaled via a first syntax element, such that the signaled flag has a first value. For example, a flag indicating whether a primary and / or secondary transformation is applied is first signaled. When this flag is signaled with a value indicating that a primary and / or secondary transformation is applied, the kernel type is further signaled to indicate which of the transformation kernels is used, and this kernel type does not include the option of not applying a primary and / or secondary transformation.

[0124] (B12) In some embodiments of B1 to B11, the secondary transformation set identifier is determined based on set context information. For example, the method of mapping the set context to the transformation set ID can be derived from any possible encoding context.

[0125] (B13) In some embodiments of B12, the set context information includes one or more of the previously coded modes, primary transform types, primary transform set identifiers, secondary transform types, secondary transform set identifiers, coded block sizes, and transform block sizes. For example, the coded context includes any possible combination or combination thereof of previously coded modes, such as an intra-predictive mode, or a primary transform type, or a primary transform set ID, or a secondary transform type, or a secondary transform set ID, or a coded block size, or a transform block size. In some embodiments, the most likely set is derived from the set context information. For example, the most likely set is derived from a coded context that is predefined and implemented in a mapping function or table.

[0126] (B14) In some embodiments of B12 or B13, the grouping of encoded texts is predefined to form an index. For example, the grouping of encoded texts is predefined to form an index for a mapping function or a table context_to_probable_set.

[0127] (B15) In some embodiments of B1 to B14, a set of transform kernel types for secondary transforms is identified based on the grouping of encoding contexts. For example, a particular set of primary and secondary transform kernel types is selected or designed from already available transform kernels to work best for a given group of encoding contexts (e.g., best decorrelation or best video compression efficiency). In some embodiments, for each predefined group of encoding contexts, there is one or a subgroup of transform sets that may be applicable to the current block.

[0128] (B16) In some embodiments of B1 to B15, the second syntax element is arithmetic coded. For example, the signaling of set IDs is done by arithmetic coding. In some embodiments, the default or initial probability model (e.g., cumulative density function (CDF)) for the arithmetic coding of the most likely quadratic transformation set ID is set as approximately 100% probability, or as high as possible acceptable in the video coding system. In some embodiments, the default or initial probability model (e.g., CDF) for the arithmetic coding of the remaining less likely quadratic transformation set IDs is set as approximately 0% probability, or as low as possible acceptable in the video coding system. For example, approximately 0% probability is achieved by associating a minimum range value (e.g., 4) for a symbol representing one of these remaining less likely quadratic transformation set IDs. In some embodiments, the default probability model (e.g., CDF) is either adaptive or non-adaptive. In some embodiments, if the video coding system can select adaptive or non-adaptive mode for entropy coding of set symbols, the system is configured to select non-adaptive mode to spend the fewest possible bits on coding the set symbols. In some embodiments, a second syntax element is Huffman coded. For example, signaling of set IDs may be performed by variable-length coding (VLC) or Huffman coding. In some embodiments, the codeword length of the most likely conversion set is the shortest possible. In some embodiments, the codeword length of less likely conversion sets is longer than the codeword length of the most likely set.

[0129] (B17) In some embodiments of B1 to B16, the method further includes determining a most probable flag from a video bitstream, which indicates whether a set is most likely. For example, signaling a quadratic transformation begins with a binary flag "most_probable_flag" indicating whether a set is most likely. As an example, a set may be coded with most_probable_flag, considering only one most likely transformation set. In some embodiments, when the most probable flag is true, the set symbols available for coding are all the symbols in the set alphabet. In some embodiments, when the most probable flag is false, the set symbols available for coding are all the symbols in the set alphabet minus the most likely set determined by the set context (e.g., a mapping function or the table context_to_probable_set). In some embodiments, if the most probable flag is true and the set symbol to be coded is most likely, the set symbol is coded using a probability model that has the best possible probability for the set symbol. In some embodiments, if the most likely flag is false and the set symbol to be coded is less likely, the set symbol is coded using a probability model starting from uniform default probabilities. In some embodiments, to code a single coded or transformed block, one or more sets of most likely transformed sets are derived based on a predefined coding context (e.g., an intra-prediction mode), and a most likely set index is further signaled to indicate which of the transformed sets is selected when the most likely flag is true. When the most likely flag is false, a remaining_set_index is further signaled to indicate which of the non-most likely transformed sets is selected. In some embodiments, the most likely flag is context-coded using arithmetic coding.

[0130] (B18) In some embodiments of B1 to B17, a secondary transformation set list is derived for each transformation block. For example, a secondary or primary transformation set list is derived for each transformation block. Previously parsed information may be used to order the secondary or primary transformation sets in the list. As an example, the most likely secondary or primary transformation set may be at the top of the list. The index to the set may be signaled using bypass, golom, or arithmetic coding. The offset may be further signaled using bypass, golom, or arithmetic coding to indicate the kernel type.

[0131] In another aspect, some embodiments include a computing system (e.g., server system 112) including a control circuit (e.g., control circuit 302) and memory coupled to the control circuit (e.g., memory 314), the memory storing one or more instruction sets configured to be executed by the control circuit, the one or more instruction sets containing instructions for performing any of the methods described herein (e.g., A1-A8 and B1-B18 above). In yet another aspect, some embodiments include a non-temporary computer-readable storage medium storing one or more instruction sets executed by the control circuit of the computing system, the one or more instruction sets containing instructions for performing any of the methods described herein (e.g., A1-A8 and B1-B18 above).

[0132] In this specification, various elements may be described using terms such as "first," "second," etc., but it should be understood that these elements should not be limited by these terms. These terms are used solely to distinguish one element from another. The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the scope of the claims. Where used in the description of embodiments and in the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural form unless the context clearly indicates otherwise. The terms “and / or” as used herein will also be understood to refer to and encompass any and all possible combinations of one or more of the enumerated items relating to the description. Where used herein, the terms “have” and / or “include” specify the presence of the described features, integers, steps, actions, elements, and / or components, but will not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof.

[0133] As used herein, the term "if..." may be interpreted, depending on the context, as meaning "when" or "on the occasion" the stated antecedent is true, or "in response to the determination that...", or "according to the determination that...", or "in response to the detection that...". Similarly, the phrase "[if] it is determined that the stated antecedent is true", or "[if] the stated antecedent is true", or "[when] the stated antecedent is true", may be interpreted, depending on the context, as meaning "when" or "in response to the determination that" the stated antecedent is true, or "according to the determination that", or "when" or "in response to the detection".

[0134] The above description has been illustrated with reference to specific embodiments for illustrative purposes. However, the above illustrative discussion is not intended to be exhaustive or to limit the claims to the exact form disclosed. In light of the above teachings, many modifications and variations are possible. The embodiments have been selected and illustrated to best illustrate the operating principle and practical application, thereby enabling those skilled in the art to implement them.

Claims

1. A video decoding method performed by a computing system having memory and one or more processors, the method being: The process involves receiving video data from a video bitstream, which includes multiple blocks containing a first block and a first syntax element; A step of determining the secondary transformation kernel type for the first block based on the first syntax element; In accordance with the determination that the aforementioned secondary transformation kernel type has a first value: A secondary transformation set identifier is determined based on a second syntax element from the aforementioned video bitstream. A step of performing an inverse quadratic transform on the first block using a determined quadratic transform kernel type and a determined quadratic transform set identifier, wherein the inverse quadratic transform corresponds to the quadratic transform performed during the encoding of the video data; In response to the determination that the quadratic transformation kernel type has a second value, the step of refraining from performing the inverse quadratic transformation on the first block. Methods that include...

2. The method according to claim 1, wherein the second syntax element is not included in the video bitstream, depending on whether the secondary conversion kernel type has the second value.

3. The method according to claim 1, further comprising the step of determining a linear transformation for the first block based on a third syntax element from the video bitstream.

4. The method according to claim 3, wherein the linear transformation is an inseparable transformation.

5. The method according to claim 1, wherein the quadratic transformation is an inseparable transformation.

6. The method according to claim 1, wherein the secondary transformation is selected from a set of transformations based on a determined secondary transformation set identifier.

7. The method according to claim 6, wherein the conversion set includes one or more primary-only conversion types and one or more primary and secondary conversion types.

8. The method according to claim 1, wherein the secondary conversion kernel type is signaled jointly in the video bitstream together with the primary conversion kernel type.

9. The method according to claim 1, further comprising the step of determining a kernel set for the secondary conversion based on information from the video bitstream.

10. The method according to claim 9, wherein the kernel set for the secondary conversion is signaled jointly in the video bitstream together with the kernel set for the primary conversion.

11. A flag indicating whether a primary and / or secondary transformation is applied is signaled in the video bitstream. Depending on whether the signaled flag has a first value, the secondary transformation kernel type is signaled via the first syntax element. The method according to claim 1.

12. The method according to claim 1, wherein the secondary transformation set identifier is determined based on set context information.

13. The method according to claim 12, wherein the set context information includes one or more of the previously coded mode, primary transformation type, primary transformation set identifier, secondary transformation type, secondary transformation set identifier, encoded block size, and transformation block size.

14. The method according to claim 12, wherein the grouping of encoded texts is predefined to form an index.

15. The method according to claim 1, wherein the set of transformation kernel types for the secondary transformation is identified based on the grouping of the encoding context.

16. The method according to claim 1, wherein the second syntax element is arithmetic-encoded.

17. The method according to claim 1, further comprising the step of determining a most likely flag from the video bitstream, wherein the most likely flag indicates whether the set is most likely.

18. The method according to claim 1, wherein a secondary transformation set list is derived for each transformation block.

19. Control circuits and; The control circuit has a memory that stores a computer program for causing the method described in any one of claims 1 to 18 to be executed. A computing system having [a certain feature].

20. A computer program for causing a control circuit to perform the method described in any one of claims 1 to 18.

21. A video encoding method performed by a computing system having memory and one or more processors, the method being: The stage of receiving video data that includes multiple blocks, including the first block; The step of determining the secondary transformation kernel type for the first block; A step of transmitting a first syntax element in a video bitstream, wherein the first syntax element indicates the secondary conversion kernel type; In accordance with the determination that the aforementioned secondary transformation kernel type has a first value: Determining the secondary transformation set identifier, The method involves transmitting a second syntax element in the video bitstream, wherein the second syntax element indicates a secondary transformation set identifier. The steps include: performing a secondary transformation on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier; In accordance with the determination that the aforementioned secondary transformation kernel type has a second value: A step of refraining from sending the second syntax element and refraining from performing the secondary transformation on the first block. Methods that include...

22. A video encoding method performed by a computing system having memory and one or more processors, the method being: The stage of receiving video data that includes multiple blocks, including the first block; The step of determining the secondary transformation kernel type for the first block; A step of determining a first syntax element, wherein the first syntax element indicates the quadratic transformation kernel type; In accordance with the determination that the aforementioned secondary transformation kernel type has a first value: Determining the secondary transformation set identifier, The second syntax element is determined, wherein the second syntax element indicates a secondary transformation set identifier. The steps include: performing a secondary transformation on the first block using the determined secondary transformation kernel type and the determined secondary transformation set identifier; In accordance with the determination that the aforementioned secondary transformation kernel type has a second value: A step of refraining from sending the second syntax element and refraining from performing the secondary transformation on the first block. Methods that include...

23. Control circuits and; The control circuit has a memory that stores a computer program for causing it to perform the method described in claim 21 or 22. A computing system having [a certain feature].

24. A computer program for causing a control circuit to perform the method according to claim 21 or 22.

Images