Block end coding system and method for 2D coefficient blocks based on 1D transform

Abstract

Various implementations described herein include methods and systems for encoding and decoding video. In one aspect, a method includes receiving a video bitstream, the video bitstream including a plurality of blocks, the plurality of blocks including a current block. The method includes parsing a plurality of one-dimensional (1D) transform coefficients corresponding to the current block from the video bitstream. The method includes parsing a maximum block end (EOB) value of the plurality of 1D transform coefficients from the video bitstream. The method includes deriving a two-dimensional (2D) transform block using the maximum EOB value and the plurality of 1D transform coefficients. The method further includes reconstructing the current block using the derived 2D transform block.

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 603,533, filed November 28, 2023, entitled “End of Block Coding for 2D Coefficients Block with 1D Transforms Only,” and is a continuation-in-place of U.S. Patent Application No. 18 / 677,739, filed May 29, 2024, entitled “Systems and Methods for End of Block Coding for 2D Coefficients Block with 1D Transforms,” each of which is incorporated herein by reference in its entirety. Technical Field

[0002] The disclosed embodiments generally relate to video encoding and decoding, including but not limited to systems and methods for encoding / decoding transformations and one-dimensional (1D) transform coefficients. Background Technology

[0003] Digital video is supported by various electronic devices, such as digital televisions, laptops or desktop computers, tablets, digital cameras, digital recording devices, digital media players, video game consoles, smartphones, video conferencing equipment, and video streaming devices. Electronic devices send and receive, or otherwise transmit, digital video data via communication networks, and / or store digital video data on storage devices. Due to the limited bandwidth capacity of communication networks and the limited memory resources of storage devices, video data may be compressed using video coding according to one or more video coding standards before transmission or storage. Video encoding and decoding can be performed by hardware and / or software on electronic / client devices or servers providing cloud services.

[0004] Video coding typically utilizes prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that take advantage of the inherent redundancy in video data. Video coding aims to compress video data into a form using a lower bitrate while avoiding or minimizing video quality degradation. Several video codec standards have been developed. For example, High-Efficiency Video Coding (HEVC / H.265) is a video compression standard designed as part of the MPEG-H project. The ITU-T and ISO / IEC released the HEVC / H.265 standard in 2013 (Revision 1), 2014 (Revision 2), 2015 (Revision 3), and 2016 (Revision 4). Versatile Video Coding (VVC / H.266) is a video compression standard designed as a successor to HEVC. The ITU-T and ISO / IEC released the VVC / H.266 standard in 2020 (Revision 1) and 2022 (Revision 2). AV1 (AOMedia Video 1) is an open video coding format designed as an alternative to HEVC. On January 8, 2019, the validation version 1.0.0 of this specification (with errata table 1) was released. Summary of the Invention

[0005] This disclosure describes a set of methods for video (image) compression, including methods for performing transforms and encoding transform coefficients. A video bitstream may include multiple (1D) transform coefficients for a specific coded block. End-of-block (EOB) values ​​of the transform coefficients can be parsed from the video bitstream, and a two-dimensional (2D) transform block can be derived using the EOB values ​​and transform coefficients. The 2D transform block can be used to reconstruct the specific block. The EOB value indicates the position (e.g., number position) of the last non-zero coefficient in the coefficient scan order of a specific line (e.g., row or column) of the 1D transform coefficients. Because all coefficients after the EOB have zero values, these coefficients can be omitted from the signal in the video bitstream without degrading video quality. In this way, encoding and signaling the EOB values(s) of the 2D transform block improves coding efficiency and reduces coding overhead (e.g., by reducing redundancy and the number of encoded bits).

[0006] According to some embodiments, a video decoding method includes: (i) receiving a video stream, the video stream including a plurality of blocks, the plurality of blocks including a current block; (ii) parsing a plurality of one-dimensional (1D) transform coefficients corresponding to the current block from the video stream; (iii) parsing a maximum end-of-block (EOB) value of the plurality of 1D transform coefficients from the video stream; (iv) deriving a two-dimensional (2D) transform block using the maximum EOB value and the plurality of 1D transform coefficients; and (v) reconstructing the current block using the derived 2D transform block.

[0007] According to some embodiments, a video coding method includes: (i) receiving video data, the video data including a plurality of blocks, the plurality of blocks including a current block; (ii) deriving a plurality of one-dimensional (1D) transform coefficients corresponding to the current block; (iii) deriving a maximum end-of-block (EOB) value of the plurality of 1D transform coefficients; (iv) transmitting the plurality of 1D transform coefficients in a video bitstream; and (v) transmitting the maximum EOB value in a video bitstream.

[0008] According to some embodiments, a method for bitstream conversion includes: (i) obtaining a source video sequence comprising a plurality of images; and (ii) performing a conversion between a bitstream of the source video sequence and a bitstream of visual media data, wherein the bitstream includes: (a) a plurality of coded images corresponding to the plurality of images; (b) a plurality of one-dimensional (1D) transform coefficients corresponding to a current block of the current image among the plurality of coded images; and (c) a maximum end-of-block (EOB) value of the plurality of 1D transform coefficients.

[0009] According to some embodiments, a computing system, such as a streaming system, server system, personal computer system, or other electronic device, is provided. The computing system includes control circuitry and memory storing one or more instruction sets. The one or more instruction sets include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder).

[0010] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more instruction sets executable by a computing system. The one or more instruction sets include instructions for performing any of the methods described herein.

[0011] Therefore, apparatus and systems using methods for encoding and decoding video are disclosed. Such methods, apparatus, and systems may complement or replace conventional methods, apparatus, and systems used for video encoding / decoding. The features and advantages described in the specification are not necessarily exhaustive, and in particular, some additional features and advantages will be apparent to those skilled in the art in light of the accompanying drawings, specification, and claims provided in this disclosure. Furthermore, it should be noted that the language used in the specification has been chosen primarily for readability and guidance purposes, and not necessarily for depicting or limiting the subject matter described herein. Attached Figure Description

[0012] To gain a more detailed understanding of this disclosure, reference can be made to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the drawings only illustrate relevant features of this disclosure and are therefore not necessarily limiting, as those skilled in the art will understand upon reading this disclosure that the specification may contain other valid features.

[0013] Figure 1 This is a block diagram illustrating an exemplary communication system according to some embodiments.

[0014] Figure 2A This is a block diagram illustrating exemplary elements of an encoder assembly according to some embodiments.

[0015] Figure 2B This is a block diagram illustrating exemplary elements of a decoder component according to some embodiments.

[0016] Figure 3 This is a block diagram illustrating an exemplary server system according to some embodiments.

[0017] Figures 4A to 4D An exemplary encoding tree structure according to some embodiments is shown.

[0018] Figures 5A to 5C Exemplary prediction blocks, residual blocks, and reconstruction blocks according to some embodiments are shown.

[0019] Figure 6 Exemplary coefficients and EOB positions for 1D transformations are shown according to some embodiments.

[0020] Figure 7 An exemplary 2D coefficient block according to some embodiments is shown.

[0021] Figure 8A An exemplary video decoding process according to some embodiments is shown.

[0022] Figure 8B An exemplary video encoding process according to some embodiments is shown.

[0023] By convention, the various features shown in the accompanying drawings are not necessarily drawn to scale, and the same reference numerals may be used to denote the same features throughout the specification and the accompanying drawings. Detailed Implementation

[0024] This disclosure describes a video / image compression technique including transform coding. The disclosed technique includes obtaining a plurality of 1D transform coefficients and a corresponding set of one or more EOB values ​​for those 1D transform coefficients. The set of EOB values ​​and the plurality of 1D transform coefficients can be used to construct a two-dimensional (2D) transform block, and the 2D transform block can then be used to reconstruct the current block. Using EOB values ​​allows trailing zero coefficients to be omitted from the bitstream without degrading video quality. Therefore, the disclosed technique can reduce coding overhead and improve coding efficiency. Example systems and devices

[0025] Figure 1 This is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m), which are communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, for example, used with video-enabled applications such as video conferencing applications, digital television applications, media storage and / or distribution applications.

[0026] Source device 102 includes a video source 104 (e.g., a camera component or media storage) and an encoder component 106. In some embodiments, the video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). The encoder component 106 generates one or more encoded video streams from the video stream. The video stream from video source 104 can have a high data volume compared to the encoded video stream 108 generated by encoder component 106. Because the encoded video stream 108 has a lower data volume (less data) compared to the video stream from video source 104, it requires less bandwidth for transmission and less storage space for storage compared to the video stream from video source 104. In some embodiments, source device 102 does not include encoder component 106 (e.g., configured to transmit uncompressed video to network 110).

[0027] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired (connected) and / or wireless communication networks. One or more networks 110 may exchange data in circuit-switched and / or packet-switched channels. Representative networks include telecommunications networks, local area networks (LANs), wide area networks (WANs), and / or the Internet.

[0028] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is a streaming server or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from source device 102). The server system 112 includes an encoder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the encoder component 114 includes an encoder component and / or a decoder component. In various embodiments, the encoder component 114 is instantiated as hardware, software, or a combination of hardware and software. In some embodiments, the encoder component 114 is configured to decode the encoded video stream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings based on the encoded video stream 108. In some embodiments, the server system 112 serves as a media-aware network element (MANE). For example, server system 112 can be configured to trim encoded video streams 108 to customize potentially different streams for one or more electronic devices 120. In some embodiments, MANE is provided separately from server system 112.

[0029] Electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode encoded video data 116 to generate an output video stream that can be displayed on a display or other type of presentation device. In some embodiments, one or more electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or includes media storage). In some embodiments, electronic device 120 is a streaming client. In some embodiments, electronic device 120 is configured to access server system 112 to obtain encoded video data 116.

[0030] The source device and / or multiple electronic devices 120 are sometimes referred to as “terminal devices” or “user devices”. In some embodiments, the source device 102 and / or one or more electronic devices 120 are instances of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing equipment, and / or other types of electronic devices.

[0031] In an exemplary operation of the communication system 100, the source device 102 transmits an encoded video stream 108 to the server system 112. For example, the source device 102 may encode an image stream acquired by the source device. The server system 112 receives the encoded video stream 108 and may decode and / or encode the encoded video stream 108 using the encoder component 114. For example, the server system 112 may apply encoding to video data, which is more advantageous for network transmission and / or storage. The server system 112 may transmit encoded video data 116 (e.g., one or more encoded video streams) to one or more electronic devices 120. Each electronic device 120 may decode the encoded video data 116 and optionally display video images.

[0032] Figure 2A This is a block diagram illustrating exemplary elements of an encoder assembly 106 according to some embodiments. The encoder assembly 106 receives video data (e.g., a source video sequence) from a video source 104. In some embodiments, the encoder assembly includes a receiver (e.g., transceiver) assembly configured to receive the source video sequence. In some embodiments, the encoder assembly 106 receives the video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder assembly 106). The video source 104 may provide a source video sequence in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8-bit, 10-bit, or 12-bit), any color space (e.g., BT.601YCrCB, or RGB), and any suitable sampling structure (e.g., YCrCb 4:2:0, or YCrCb 4:4:4). In some embodiments, the video source 104 is a storage device storing previously acquired / prepared video. In some embodiments, the video source 104 is a camera that acquires local image information as a video sequence. The video data may be provided as a plurality of individual pictures, which are given motion when viewed sequentially. An image itself can be organized into a spatial pixel array, where each pixel may include one or more samples, depending on the sampling structure, color space, etc. Those skilled in the art can readily understand the relationship between pixels and samples.

[0033] Encoder component 106 is configured to encode and / or compress images of a source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. In some embodiments, encoder component 106 is configured to perform a conversion between the source video sequence and a bitstream of visual media data (e.g., a video bitstream). Implementing an appropriate encoding rate is a function of controller 204. In some embodiments, controller 204 controls and is functionally coupled to other functional units as described below. Parameters set by controller 204 may include rate control-related parameters (e.g., image skipping, quantizer and / or rate-distortion optimization technique λ value), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. Other functions of controller 204 can be readily identified by those skilled in the art, as these functions may relate to encoder component 106 optimized for a particular system design.

[0034] In some embodiments, encoder component 106 is configured to operate within an encoding loop. In a simplified example, the encoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder 210. Decoder 210 reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (when compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream (sample data) is input to reference image memory 208. Because decoding the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of reference image memory 208 are also bit-accurately corresponding between the local encoder and the remote encoder. Thus, the encoder's prediction portion interprets the reference image samples as the same sample values ​​that the decoder will interpret during prediction.

[0035] The operation of decoder 210 can be combined with, for example, the following: Figure 2B The decoder component 122 described in detail is the same as the remote decoder. However, a brief reference is provided. Figure 2B Since symbols are available and the entropy encoder 214 and parser 254 can encode / decode symbols into an encoded video sequence in a lossless manner, the entropy decoding portion of the decoder component 122, which includes buffer memory 252 and parser 254, may not be fully implemented in the local decoder 210.

[0036] Aside from parsing / entropy decoding, the decoder techniques described herein can exist in the corresponding encoder with essentially the same functional form. For this reason, the disclosed subject matter focuses on decoder operations. Furthermore, the description of the encoder techniques can be simplified, as the encoder techniques are inverses of the decoder techniques.

[0037] As part of its operation, the source encoder 202 can perform motion-compensated predictive coding. This motion-compensated predictive coding predictively encodes the input frame, referencing one or more previously encoded frames from the video sequence designated as reference frames. In this way, the encoding engine 212 encodes the differences between pixel blocks in the input frame and pixel blocks in the reference frame, which can be selected as the prediction reference for the input frame. The controller 204 can manage the encoding operations of the source encoder 202, including, for example, setting parameters and subgroup parameters for encoding the video data.

[0038] Decoder 210 decodes encoded video data of frames that can be designated as reference frames, based on symbols created by source encoder 202. The operation of encoding engine 212 can advantageously be a lossy process. When encoded video data is processed by video decoder (… Figure 2A When decoded at (not shown), the reconstructed video sequence can be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process, which can be performed on the reference frame by a remote video decoder, and allows the reconstructed reference frame to be stored in reference image memory 208. In this way, encoder component 106 locally stores a copy of the reconstructed reference frame that shares common content (no transmission errors) with the reconstructed reference frame to be obtained by the remote video decoder.

[0039] Predictor 206 can perform a prediction search against encoding engine 212. That is, for a new frame to be encoded, predictor 206 can search the reference image memory 208 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can be used as appropriate prediction references for the new image. Predictor 206 can operate pixel-by-pixel based on sample blocks to find suitable prediction references. As determined by the search results obtained by predictor 206, the input image may have prediction parameters obtained from multiple reference images stored in reference image memory 208.

[0040] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder 214. The entropy encoder 214 performs lossless compression on the symbols generated by the various functional units according to techniques known to those skilled in the art (e.g., Huffman coding, variable-length coding, and / or arithmetic coding), thereby converting the symbols into an encoded video sequence.

[0041] In some embodiments, the output of entropy encoder 214 is coupled to a transmitter. The transmitter may be configured to buffer the encoded video sequence created by entropy encoder 214 in preparation for transmission via communication channel 218, which may be a hardware / software link to a storage device capable of storing the encoded video data. The transmitter may be configured to combine encoded video data from source encoder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown). In some embodiments, the transmitter may transmit additional data while transmitting the encoded video. Source encoder 202 may include such data as part of the encoded video sequence. Additional data may include temporal / spatial / signal-noise ratio (SNR) enhancement layers, other forms of redundant data such as redundant pictures and slices, supplementary enhancement information (SEI) messages, fragments of visual usability information (VUI) parameter sets, etc.

[0042] Controller 204 manages the operation of encoder component 106. During encoding, controller 204 can assign a specific type of encoded picture to each encoded picture, but this may affect the encoding technique applied to the corresponding picture. For example, pictures can be assigned as intra-pictures (I-pictures), predictive pictures (P-pictures), or bidirectional predictive pictures (B-pictures). Intra-pictures can be encoded and decoded without using any other frames in the sequence as prediction sources. Some video codecs allow different types of intra-pictures, including, for example, independent decoder refresh (IDR) pictures. These variations of I-pictures and their corresponding applications and characteristics are familiar to those skilled in the art and will not be described in detail here. Predictive pictures can be encoded and decoded using intra-prediction or inter-prediction, which uses at most one motion vector and reference index to predict sample values ​​for each block. Bidirectional predictive pictures can be encoded and decoded using intra-prediction or inter-prediction, which uses at most two motion vectors and reference indices to predict sample values ​​for each block. Similarly, multiple predictive images can be used to reconstruct a single block using more than two reference images and associated metadata.

[0043] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and each block is encoded sequentially. These blocks can be predictively coded with reference to other (already coded) blocks, which are determined by the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be nonpredictively coded, or blocks of an I-image can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be nonpredictively coded with reference to a previously coded reference image via spatial prediction or temporal prediction. Blocks of a B-image can be nonpredictively coded with reference to one or two previously coded reference images via spatial prediction or temporal prediction.

[0044] The captured video can be presented as multiple source images (video images) in a time-series format. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In one example, a specific image being encoded / decoded is divided into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. The motion vector points to the reference block in the reference image, and when using multiple reference images, the motion vector can have a third dimension that identifies the reference image.

[0045] Encoder component 106 can perform encoding operations according to a predetermined video coding technique or standard (such as any standard described herein). In operation, encoder component 106 can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0046] Figure 2B This is a block diagram illustrating exemplary elements of a decoder component 122 according to some embodiments. Figure 2B The decoder component 122 is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to (e.g., via a wired or wireless connection) transmit data to the display 124.

[0047] In some embodiments, decoder component 122 includes a receiver coupled to channel 218 and configured (e.g., via a wired or wireless connection) to receive data from channel 218. The receiver may be configured to receive one or more encoded video sequences to be decoded by decoder component 122. In some embodiments, decoding of each encoded video sequence is independent of decoding of other encoded video sequences. Each encoded video sequence may be received from channel 218, which may be a hardware / software link leading to a storage device storing the encoded video data. The receiver may receive encoded video data and other data, such as encoded audio data and / or auxiliary data streams, that can be forwarded to their respective user entities (not depicted). The receiver may separate the encoded video sequences from other data. In some embodiments, the receiver receives additional (redundant) data when receiving encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by decoder component 122 to decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0048] According to some embodiments, decoder component 122 includes buffer memory 252, parser 254 (sometimes also called entropy decoder), scaler / inverse transform unit 258, intra-frame image prediction unit 262, motion compensation prediction unit 260, aggregator 268, loop filter unit 256, reference image memory 266, and current image memory 264. Decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. In some embodiments, decoder component 122 is implemented at least partially in software.

[0049] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to prevent network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is located between the output of channel 218 and decoder component 122. In some embodiments, a separate buffer memory is located outside decoder component 122 (e.g., to prevent network jitter), in addition to buffer memory 252 located inside decoder component 122 (e.g., configured to handle playback timing). Buffer memory 252 may not be needed or may be small when receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network. Buffer memory 252 may be required for use on packet-switched networks such as the Internet. Buffer memory 252 may be relatively large and / or have an adaptive size and may be implemented at least partially in an operating system or similar element outside of decoder component 122.

[0050] Parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. These symbols may include, for example, information for managing the operation of decoder component 122, and / or information for controlling a presentation device such as display 124. Control information for the presentation device may be in the form of, for example, supplementary enhancement information (SEI) messages or video availability information (VUI) parameter set fragments (not depicted). Parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 254 may extract a subset of parameters from the encoded video sequence for at least one subset of pixels in a subgroup for use in the video decoder, based on at least one parameter corresponding to a group. Subgroups may include group of pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The parser 254 can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0051] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of symbol 270 may involve multiple different units. Which units are involved and how they are involved can be controlled by parser 254 through subgroup control information parsed from the encoded video sequence. For clarity, the flow of such subgroup control information between parser 254 and the various units described below is not depicted.

[0052] The decoder component 122 may be conceptually subdivided into multiple functional units, and in some embodiments, these units interact closely with each other and may be at least partially integrated with each other. However, for clarity, the conceptually subdivided functional units are retained herein.

[0053] The scaler / inverse transform unit 258 receives quantized transform coefficients as symbols 270 and control information (e.g., which transform to use, block size, quantization factor, and / or quantization scaling matrix) from the parser 254. The scaler / inverse transform unit 258 can output blocks including sample values, which can be input to the aggregator 268. In some cases, the output samples of the scaler / inverse transform unit 258 belong to intra-coded blocks, i.e., blocks that do not use predictive information from previously reconstructed images but can use predictive information from previously reconstructed portions of the current image. Such predictive information can be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 can use surrounding reconstructed information extracted from the current (partially reconstructed) image from the current image memory 264 to generate blocks of the same size and shape as the blocks being reconstructed. The aggregator 268 can add the predictive information generated by the intra-picture prediction unit 262 to the output sample information provided by the scaler / inverse transform unit 258 based on each sample.

[0054] In other cases, the output samples of the scaler / inverse transform unit 258 belong to blocks of inter-frame coding and potential motion compensation. In this case, the motion compensation prediction unit 260 can access the reference image memory 266 to extract samples for prediction. After motion compensation is performed on the extracted samples according to the symbols 270 belonging to the block, these samples can be added by the aggregator 268 to the output of the scaler / inverse transform unit 258 (in this case, referred to as residual samples or residual signals) to generate output sample information. The extraction of prediction samples by the motion compensation prediction unit 260 from the address in the reference image memory 266 can be controlled by motion vectors. These motion vectors can be provided to the motion compensation prediction unit 260 in the form of symbols 270, which may have, for example, X components, Y components, and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory 266 when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0055] The output samples of aggregator 268 can undergo various loop filtering techniques in loop filter unit 256. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video stream and available to loop filter unit 256 as symbols 270 from parser 254. The video compression techniques may also respond to metadata obtained during decoding of previous (in decoding order) portions of the encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values. The output of loop filter unit 256 may be a sample stream that can be output to a presentation device (e.g., display 124) and stored in reference picture memory 266 for future inter-frame picture prediction.

[0056] Once reconstructed, certain encoded images can be used as reference images for future predictions. Once the encoded images have been fully reconstructed and the encoded images (via, for example, parser 254) are identified as reference images, the current reference images can become part of the reference image memory 266, and new current image memory can be reallocated before reconstructing subsequent encoded images begins.

[0057] Decoder component 122 can perform decoding operations according to a predetermined video compression technique that can be recorded in a standard (e.g., any standard described herein). The encoded video sequence may conform to the syntax specified by the video compression technique or standard used (as specified in the video compression technique documentation or standard, particularly in the configuration file of the video compression technique or standard) in the sense that the encoded video sequence follows the syntax of the video compression technique or standard. Furthermore, the complexity of the encoded video sequence may be within the range defined by the hierarchy of the video compression technique or standard in order to conform to some video compression techniques or standards. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, megasamples per second), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0058] Figure 3 This is a block diagram illustrating a server system 112 according to some embodiments. The server system 112 includes control circuitry 302, one or more network interfaces 304, memory 314, a user interface 306, and one or more communication buses 312 for interconnecting these components. In some embodiments, the control circuitry 302 includes one or more processors (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and / or a DPU (Data Processing Unit)). In some embodiments, the control circuitry includes one or more field-programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application-specific integrated circuits).

[0059] Network interface 304 can be configured to connect to one or more communication networks (e.g., wireless networks, wired networks, and / or optical networks). Communication networks can be local area networks, wide area networks, metropolitan area networks, vehicle and industrial networks, real-time networks, latency-tolerant networks, etc. Examples of communication networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., cable or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Such communication can be one-way receiving (e.g., broadcast television), one-way transmitting (e.g., CANBus connected to certain CANBus devices), or bidirectional (e.g., connecting to other computer systems using a local area network or wide area network). Such communication may include communication to one or more cloud computing networks.

[0060] User interface 306 includes one or more output devices 308 and / or one or more input devices 310. Input devices 310 may include one or more of a keyboard, mouse, touchpad, touchscreen, data glove, joystick, microphone, scanner, camera, etc. Output devices 308 may include one or more of an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0061] Memory 314 may include high-speed random access memory (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), DDR RAM (Double Data Rate Random Access Memory), and / or other solid-state random access memory devices) and / or non-volatile memory (e.g., one or more disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). Optionally, memory 314 includes one or more storage devices disposed remotely from control circuitry 302. Memory 314, or alternatively, the non-volatile solid-state storage devices within memory 314 include non-transitory computer-readable storage media. In some embodiments, memory 314 or the non-transitory computer-readable storage media of memory 314 stores programs, modules, instructions, and data structures, or subsets or supersets thereof: • Operating system 316, which includes programs for handling various basic system services and for performing hardware-related tasks; ● Network communication module 318, which is used to connect server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connection); ● Encoding / decoding module 320, which performs various functions related to encoding and / or decoding data (e.g., video data). In some embodiments, encoding / decoding module 320 is an instance of encoder component 114. Encoding / decoding module 320 includes, but is not limited to, one or more of the following: Decoding module 322, which performs various functions related to decoding encoded data, such as those previously described for decoder component 122; and Encoding module 340, which performs various functions related to encoding data, such as those previously described for encoder component 106; and • Image storage 352 is used to store images and image data, for example, for use with encoding / decoding module 320. In some embodiments, image storage 352 includes one or more of reference image storage 208, buffer storage 252, current image storage 264, and reference image storage 266.

[0062] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform various functions previously described for the parser 254), a transformation module 326 (e.g., configured to perform various functions previously described for the scaler / inverse transformation unit 258), a prediction module 328 (e.g., configured to perform various functions previously described for the motion compensation prediction unit 260 and / or the intra-frame image prediction unit 262), and a filter module 330 (e.g., configured to perform various functions previously described for the loop filter 256).

[0063] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform various functions previously described for the source encoder 202 and / or encoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described for the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes... Figure 3 A subset of the modules shown. For example, both decoding module 322 and encoding module 340 use a shared prediction module.

[0064] Each of the modules identified above and stored in memory 314 corresponds to a set of instructions for performing the functions described herein. The modules identified above (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules; therefore, subsets of these modules can be combined or otherwise rearranged in various embodiments. For example, optionally, the codec module 320 does not include separate decoding and encoding modules, but uses the same set of modules to perform both sets of functions. In some embodiments, memory 314 stores a subset of the modules and data structures identified above. In some embodiments, memory 314 stores additional modules and data structures not described above.

[0065] Although Figure 3 A server system 112 according to some embodiments is shown, but Figure 3 This is intended more as a functional description of various features that may exist in one or more server systems, rather than a structural diagram of the embodiments described herein. In practice, items shown individually may be combined, and some items may be separated. For example, Figure 3 Some items shown individually can be implemented on a single server, and a single item can be implemented by one or more servers. The actual number of servers used to implement server system 112, and how features are distributed among the servers, will vary depending on the implementation method and, optionally, in part, on the amount of data traffic processed by the server system during peak usage periods and during average usage periods. Example encoding and decoding techniques

[0066] The encoding and decoding processes and techniques described below can be performed on the devices and systems described above (e.g., source device 102, server system 112, and / or electronic device 120).

[0067] Now let's introduce block partitioning. Figures 4A-4D An example encoded tree structure according to some embodiments is shown. Figure 4A As shown in the first coding tree structure (400), some coding methods use a 4-way partitioning tree starting from 64×64 levels and going down to 4×4 levels, for example, utilizing some additional constraints for 8×8 blocks. Figure 4A In this context, partitions designated as "R" are recursively defined because the same partitioning tree is repeated at lower proportions until the lowest level is reached. For example... Figure 4B As shown in the example coding tree structure (402), some coding methods extend the partition tree to a 10-way structure and increase the maximum size starting from 128×128 (e.g., sometimes referred to as a superblock). The second coding tree structure includes 4:1 / 1:4 rectangular partitions that are not present in the first coding tree structure. Figure 4BThe partitioning type with 3 sub-partitions in the second row is called a T-partition. In addition to the code block size, the code tree depth can also be defined to indicate the partitioning depth from the root node.

[0068] As an example, a Coding Tree Unit (CTU) can be partitioned into Coding Units (CUs) using a quadtree structure represented as a coding tree to accommodate various local characteristics. In some embodiments, a decision is made at the CU level regarding whether to use inter-frame (temporal) or intra-frame (spatial) prediction to encode a picture region. Depending on the PU partitioning type, each CU can be further partitioned into one, two, or four Prediction Units (PUs). Within a PU, the same prediction process can be applied, and relevant information can be sent to the decoder based on the PU. After obtaining residual blocks by applying prediction processing based on the PU partitioning type, the CUs can be partitioned into Transform Units (TUs) according to another quadtree structure similar to the CU's coding tree.

[0069] A quadtree partitioning structure with nested multi-type trees can be used to replace the concept of multiple partition unit types, employing binary and ternary partitions. In the coding tree structure, the CU can have a square or rectangular shape. The CTU is initially partitioned by a quadtree structure. The leaf nodes of the quadtree can be further partitioned using a multi-type tree structure. For example... Figure 4C As shown in the third coding tree structure (404), the multi-type tree structure includes four partition types. The leaf nodes of the multi-type trees are called CUs unless the CU is too large for the maximum transform length. This means that CUs, PUs, and TUs can have the same block size in a quadtree coding block structure with nested multi-type trees. Figure 4D An example of a block partitioning of CTU(406) is shown, which illustrates an example quadtree.

[0070] The coding tree scheme supports the ability for luma and chroma to have separate block tree structures. In some cases, for P and B slices, the luma and chroma CTBs within a single CTU have the same coding tree structure. However, for I slices, luma and chroma can have separate block tree structures. When the separate block tree mode is applied, the luma CTB is partitioned into CUs via one coding tree structure, and the chroma CTB is partitioned into chroma CUs via another coding tree structure. This means that a CU in an I slice can include or consist of coding blocks for the luma component or coding blocks for both chroma components, and a CU in a P or B slice can always include or consist of coding blocks for all three color components, unless the video is monochrome.

[0071] The transform block is now introduced. To support extended coding block partitioning, multiple transform sizes (e.g., ranging from 4 to 64 points per dimension) and transform shapes (e.g., squares or rectangles with width / height ratios of 2:1 / 1:2 and 4:1 / 1:4) can be used.

[0072] The two-dimensional transform process may involve the use of a hybrid transform kernel (e.g., composed of different one-dimensional transforms for each dimension of the coded residual block). The primary one-dimensional transform may include at least one of the following: a) 4-point, 8-point, 16-point, 32-point, and 64-point Discrete Cosine Transform (DCT-2); b) 4-point, 8-point, and 16-point Asymmetric Discrete Sine Transform (DST-4, DST-7) and their inverted versions; or c) 4-point, 8-point, 16-point, and 32-point Identity Transform. Table 1 lists the basis functions for DCT-2 and Asymmetric DST, such as those used in AV1. Table 1 Examples of primary transformation basis functions

[0073] The availability of hybrid transform kernels can be based on transform block size and prediction mode. Example dependencies are listed in Table 2 below, where “→” and “↓” represent horizontal and vertical dimensions, and “√” and “×” indicate the availability of kernels for that block size and prediction mode. IDTX (or IDT) stands for Identity Transformation. Table 2 Availability of Hybrid Transform Cores

[0074] For chroma components, transform type selection can be performed implicitly. For intra-frame prediction residuals, such as those specified in Table 3, the transform type can be selected based on the intra-frame prediction mode. For inter-frame prediction residuals, the transform type can be selected based on the transform type selection of the co-occurring luma block. Therefore, for chroma components, it may not be necessary to transmit the transform type in the bitstream. Table 3 Transform type selection for chroma intra-frame prediction residuals Intra-frame prediction Vertical Transformation Horizontal transformation DC_PRED DCT DCT V_PRED ADST DCT H_PRED DCT ADST D45_PRED DCT DCT D135_PRED ADST ADST D113_PRED ADST DCT D157_PRED DCT ADST D203_PRED DCT ADST D67_PRED ADST DCT SMOOTH_PRED ADST ADST SMOOTH_V_PRED ADST DCT SMOOTH_H_PRED DCT ADST PAETH_PRED ADST ADST

[0075] We will now present example encoding and decoding using prediction blocks and residual blocks. Figure 5A The calculation of a prediction block according to some embodiments is illustrated. Figure 5A In the example, intra-frame prediction is performed on the current block 502 to generate prediction block 504. In some embodiments, inter-frame prediction is performed to generate prediction blocks. The current block 502 includes a set of samples (e.g., pixel blocks), and prediction block 504 includes a set of predictions corresponding to the set of samples. Figure 5B The calculation of residual blocks according to some embodiments is illustrated. For example... Figure 5BAs shown, the prediction block 504 is subtracted from the current block 502 to generate a residual block 506 that includes the set of residuals. For example, the corresponding difference between each sample and its corresponding prediction is calculated. Figure 5C The calculation of the reconstructed block according to some embodiments is shown. For example... Figure 5C As shown, residual block 506 undergoes one or more transforms and quantizations to generate a set of residual coefficients. This set of residual coefficients can be sent from the encoder component to the decoder component. The set of residual coefficients is inversely quantized and inversely transformed to generate reconstructed residual block 508. Reconstructed residual block 508 is combined with prediction block 504 (e.g., the reconstruction residuals of reconstructed residual block 508 are added to the predictions of prediction block 504) to generate reconstructed block 510 corresponding to the current block 502.

[0076] Some embodiments include switching the probabilistic model used during entropy coding. Entropy coding can utilize context and is capable of efficiently encoding symbols. Some embodiments include coefficient coding techniques in which the structure of coefficient symbols is transformed to be derived in a more context-aware manner.

[0077] In the following text, the term "row-based EOB" indicates that the EOB applies to 2D coefficient blocks formed using horizontal 1D transformations (e.g., corresponding to...). Figure 7 (Line 704 in the text). The term "column-based EOB" indicates that the EOB is applicable to 2D coefficient blocks formed using vertical 1D transformations (e.g., corresponding to...). Figure 7 (Column 706 in the table). The term "1D EOB" indicates the EOB of a 1D transform. The term "2D EOB" indicates the EOB of a 2D coefficient block. Furthermore, the "1D Skip Flag" indicates whether at least one non-zero coefficient exists in the 1D transform coefficients. The value of the flag can be predefined to indicate whether all coefficients are zero. As an example, the flag is true (equal to 1) if all coefficients are zero. Similarly, the "2D Skip Flag" indicates whether at least one non-zero coefficient exists in a 2D coefficient block.

[0078] Coefficient encoding can be performed by scanning quantized transform coefficients in a 2D block using a predefined scan order (e.g., zigzag or diagonal scan) and converting the scanned coefficients into a 1D vector. In addition to scanning the quantized transform coefficients, multiple syntaxes can be encoded. Exemplary syntaxes may include: the position of the last non-zero coefficient (also referred to herein as the end of block (EOB)), whether the coefficients at each scanned coefficient position (e.g., saliency plot) are non-zero, the coefficient level, and / or the coefficient sign value.

[0079] EOB information is part of the coefficient encoding and can be implemented in various ways. If the coefficient encoding begins at the first coefficient position (e.g., predefined in a 2D block), typically on the upper left, then the EOB indicates where the encoding of the coefficient symbol should end. Conversely, if the coefficient encoding ends at the last coefficient position (e.g., predefined in a 2D block), typically on the lower right, then the EOB indicates where the encoding of the coefficient symbol should begin.

[0080] A 1D transform for a residual block can refer to applying a 1D transform along one direction (e.g., horizontal or vertical), while a 2D transform for a residual block can refer to applying a 1D transform along multiple directions (e.g., horizontal and vertical) (e.g., sequentially). In some embodiments, the resulting transform coefficients are encoded using a 2D coefficient encoding method. Some embodiments include an EOB encoding method derived from the 1D transform for the 2D coefficient block and the context of the 2D coefficient block.

[0081] EOB can be represented in several ways. Typically, EOB can be represented as a numbered position, for example, immediately following the last non-zero coefficient in the coefficient scan order. Figure 6 Here is an example showing 8-point quantized coefficients [5,0,4,0,3,0,0,0] arranged in a one-dimensional row, where the last non-zero coefficient (represented by element 602) is 3, and its position in the 8-point quantized coefficient vector is 4 (represented by element 604). EOB can then be represented as 5 (e.g., EOB indicates the position following the last non-zero coefficient in the coefficient scan order). By this convention, EOB = 0 indicates that there are no non-zero coefficients in the transform block after position 4. It is worth noting that after the transform, the coefficients may or may not be quantized.

[0082] Figure 7 An exemplary 2D coefficient block 702, comprising eight rows 704 (e.g., rows 704-1 and 704-2) and eight columns 706 (e.g., columns 706-1 and 706-2) according to some embodiments, is shown. Each of the rows 704 and each of the columns 706 includes a corresponding set of 1D transform coefficients (e.g., corresponding coefficient lines). The last non-zero coefficient in each row 704 is located at... Figure 7 The text in bold indicates this. The corresponding EOB for each line 704 is the position next to (e.g., to its right) the last non-zero coefficient, and... Figure 7 The cells shown are shaded. Figure 7 The right side shows an 8×1 column, where each cell in the column indicates the EOB position in the corresponding row. Figure 7There are eight EOBs, one EOB per line of 704. In some embodiments, only the largest EOB among all EOBs of each 1D transform coefficient encoded together in the 2D coefficient block is encoded. For example, in Figure 7 In the table, the maximum EOB 708 is located at row number 5 (where rows are counted starting from the initial value 0), and the EOB is 6.

[0083] although Figure 7 The EOB corresponding to row 704 of block 702 is shown, but it will be apparent to those skilled in the art that similar examples can be constructed for 1D transformations in other directions (e.g., column 706 of block 702 could be used for the vertical direction).

[0084] Figure 8A This is a flowchart illustrating a video decoding method 800 according to some embodiments. Method 800 can be executed on a computing system (e.g., server system 112, source device 102, or electronic device 120) including control circuitry and memory, the memory storing instructions executed by the control circuitry. In some embodiments, method 800 is executed by executing instructions stored in the computing system's memory (e.g., memory 314).

[0085] The system receives (802) a video stream, which includes multiple blocks, including the current block. The system parses (804) multiple 1D transform coefficients corresponding to the current block from the video stream. The system parses (806) the maximum EOB value of the multiple 1D transform coefficients from the video stream. The system uses the maximum EOB value and the multiple 1D transform coefficients to derive (808) a 2D transform block (e.g., a 2D coefficient block). The system uses the derived 2D transform block to reconstruct (810) the current block. For example, the 2D coefficient block, composed of 1D transforms, is encoded based on the maximum EOB of the 1D transform coefficients in the 2D coefficient block. In some embodiments, the 2D coefficient block is encoded using only 1D transforms based on the maximum EOB of the 1D transform coefficients in the 2D coefficient block. In some embodiments, the 2D EOB of the 2D coefficient block is determined as the maximum value of the 1D EOB across all lines (e.g., rows or columns) of the 2D coefficient block.

[0086] In some embodiments, the EOB is represented based on the position of the last non-zero coefficient in the 1D transform coefficients (e.g., as described above regarding...). Figure 7(As described). One example is including the position immediately following the last non-zero coefficient in the coefficient scan order. In some embodiments, when entropy coding is performed on an EOB, the context for selecting the probabilistic model includes the encoded adjacent transform blocks. In some embodiments, the 1D transform direction employed in the current 2D coefficient block implicitly derives whether to select the context from the upper adjacent transform block, or the left adjacent transform block, or the upper-left adjacent transform block. In some embodiments, the intra-prediction mode employed in the current 2D coefficient block implicitly derives whether to select the context from the upper adjacent transform block, or the left adjacent transform block, or the upper-left adjacent transform block. In some embodiments, when entropy coding is performed on an EOB, the context for selecting the probabilistic model includes the size of the current 2D coefficient block. In some embodiments, when entropy coding is performed on a row-based EOB, the context for selecting the probabilistic model depends on the block width (or height). In some embodiments, when entropy coding is performed on a column-based EOB, the context for selecting the probabilistic model depends on the block height (or width).

[0087] In some embodiments, a set of multiple EOBs is encoded for multiple rows (or columns) in a 2D coefficient block. In some embodiments, for each line, the difference between its associated EOB value and a reference EOB value is encoded. In some embodiments, for selected lines, the difference between its associated EOB value and a reference EOB value is encoded. In one example, the reference EOB is the EOB value associated with the previous (in encoding order) line (row or column), and the EOB associated with the first (in encoding order) line is explicitly encoded without referencing other EOB values. In another example, the reference EOB is the maximum EOB value among all lines, where the maximum EOB value is explicitly encoded without referencing other EOB values.

[0088] In some embodiments, whether a single EOB value or multiple EOB values ​​are used, they are sent via high-level syntax (e.g., sequence-level flags, picture-level flags, sub-picture-level flags, slice-level flags, tile-level flags) or block-level syntax (e.g., maximum coding block-level flag, maximum coding block line-level flag, coding block-level flag, transform block-level flag).

[0089] In some embodiments, given a line-based EOB (e.g., a row EOB or a column EOB), when encoding coefficients within a line, coefficients are scanned based on the line, and coefficients from the EOB position to the end of the line are not encoded. In some embodiments, 2D coefficient blocks are encoded using only 1D transforms, for example, by encoding a set of all “1D skip flags” from each line in the 2D coefficient block, where each skip flag indicates the presence of at least one non-zero coefficient in each 1D transform coefficient.

[0090] In some embodiments, after decoding all 1D skip flags in a 2D coefficient block, only lines of 1D coefficients with a corresponding flag value of false or 0 (or any other predefined value indicating the presence of at least one non-zero coefficient) are further decoded for non-zero coefficients. In some embodiments, if a 2D skip flag is implemented for the 2D coefficient block and all 1D skip flags are true, then the 1D skip flags are not encoded. For example, the mere presence of a 2D skip flag can signal the absence of non-zero coefficients in the 2D coefficient block.

[0091] In some embodiments, when entropy encoding is performed on the 1D skip flag set, the context used to select the probabilistic model includes information from encoded neighboring transform blocks. In some embodiments, the decision to select the context from the top, left, and / or top-left neighboring transform blocks is implicitly derived based on the 1D transform direction employed in the current 2D coefficient block. In some embodiments, the decision to select the context from the top, left, and / or top-left neighboring transform blocks is implicitly derived based on the intra-prediction mode employed in the current 2D coefficient block.

[0092] In some embodiments, when entropy encoding is performed on a 1D skip flag set, the context of the selection probability model includes the size of the current 2D coefficient block. In some embodiments, when row-based EOBs are entropy encoded, the context of the selection probability model depends on the block width (and / or height) of the current block. In some embodiments, when column-based EOBs are entropy encoded, the context of the selection probability model depends on the block height (and / or width) of the current block.

[0093] In some embodiments, if the 1D skip flag of the 1D transform coefficient is true, then the 1DEOB of the same 1D coefficient is not encoded, and the 1D EOB of the same 1D coefficient is implicitly derived to 0 on both the encoder and decoder sides.

[0094] Figure 8B This is a flowchart illustrating a video encoding method 850 according to some embodiments. Method 850 can be executed on a computing system (e.g., server system 112, source device 102, or electronic device 120) including control circuitry and memory, the memory storing instructions executed by the control circuitry. In some embodiments, method 850 is executed by executing instructions stored in the computing system's memory (e.g., memory 314).

[0095] The system receives (852) video data, which comprises multiple blocks, including the current block. The system derives (854) multiple 1D transform coefficients corresponding to the current block. The system derives (856) the maximum EOB value of the multiple 1D transform coefficients. The system transmits the multiple 1D transform coefficients in the video stream (858). The system transmits the maximum EOB value in the video stream (860). As previously mentioned, the encoding process can mirror the decoding process described herein (e.g., using transform coefficients and / or EOB values). For the sake of brevity, these details are not repeated here.

[0096] Although Figure 8A and Figure 8B Multiple logical stages are shown in a specific order, but stages that are not in order can be reordered, and other stages can be combined or split. Some reorderings or other groupings not specifically mentioned will be obvious to those skilled in the art, and therefore the ordering and groupings given in this disclosure are not exhaustive. Furthermore, it should be recognized that these stages can be implemented in hardware, firmware, software, or any combination thereof.

[0097] Now let's turn to some exemplary embodiments:

[0098] (A1) In one aspect, some embodiments include a video decoding method (e.g., method 800). In some embodiments, the method is performed on a computing system (e.g., server system 112) including memory and control circuitry. In some embodiments, the method is performed on an encoding / decoding module (e.g., encoding / decoding module 320). In some embodiments, the method is performed on a source encoding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving a video bitstream comprising a plurality of blocks, the plurality of blocks including a current block; (ii) parsing a plurality of 1D transform coefficients corresponding to the current block from the video bitstream; (iii) parsing a maximum EOB value of the plurality of 1D transform coefficients from the video bitstream; (iv) deriving a 2D transform block (e.g., a 2D coefficient block) using the maximum EOB value and the plurality of 1D transform coefficients; and (v) reconstructing the current block using the derived 2D transform block. For example, a 2D coefficient block consisting only of 1D transforms is encoded based on the maximum EOB of the 1D transform coefficients in the 2D coefficient block.

[0099] (A2) In some embodiments of A1, the maximum EOB value is the 2D maximum EOB value for the 2D transform block, wherein the maximum EOB value is derived using the corresponding 1D EOB values ​​of the set of lines (e.g., rows or columns) of the current block. As an example, the lines of the current block can be the rows or columns of the current block. For example, the 2D EOB of the 2D coefficient block is determined to be the maximum 1D EOB among all rows (or columns).

[0100] (A3) In some embodiments of A2, each corresponding 1D EOB value indicates the position of the last non-zero coefficient in the corresponding line of the line set. For example, the EOB can be indicated based on the position of the last non-zero coefficient in the 1D transform coefficients. In one example, the EOB indicates the position following the last non-zero coefficient in the coefficient scan order.

[0101] (A4) In some embodiments of A3, transform coefficients located after the last non-zero coefficient are not transmitted in the bitstream. For example, given a row (or column) based EOB, when encoding coefficients within a row (or column), coefficients are scanned based on the row (or column), and coefficients from the EOB position to the end of the row (or column) are not encoded.

[0102] (A5) In some embodiments of any of A2-A4, the 1DEOB value in the corresponding EOB value is an incremental value indicating the difference between the maximum EOB value of the corresponding line of the current block and a reference EOB value. For example, for each (or selected) row (or column), the difference between its associated EOB value and a reference EOB value is encoded.

[0103] (A6) In some embodiments of A5, the reference EOB value is the EOB value of a previous line in the line set. For example, the reference EOB is the EOB value associated with a previous (in coded order) row (or column), and the EOB associated with the first (in coded order) row (or column) is explicitly encoded without referencing other EOB values.

[0104] (A7) In some embodiments of A5 or A6, the reference EOB value is the maximum EOB value of the 2D transform block. For example, the reference EOB is the maximum EOB value across all rows (or columns), and the maximum EOB value is explicitly encoded without referencing other EOB values.

[0105] (A8) In some embodiments of any of A1-A7, parsing the maximum EOB value includes entropy decoding of the maximum EOB value using the context of the encoded adjacent transform blocks. For example, when entropy encoding the EOB, the context for selecting the probability model includes, but is not limited to, the encoded adjacent transform blocks.

[0106] (A9) In some embodiments of A8, the context is selected based on the transformation type of the transformation used to generate the plurality of 1D transformation coefficients. For example, the decision to select a context from the top, left, or upper-left adjacent transformation block can be implicitly derived based on the 1D transformation direction used in the current 2D coefficient block. As an example, the transformation type can be horizontal, vertical, or other types of 1D transformation (e.g., in any direction).

[0107] (A10) In some embodiments of A8 or A9, the context is selected based on the intra-prediction mode used to encode the current block. For example, the decision to select a context from the top, left, or upper-left adjacent transform block can be implicitly derived based on the intra-prediction mode employed in the current 2D coefficient block. As an example, the spatial orientation of the intra-prediction mode can be used to select the context (e.g., eight primary spatial orientations, each of which can provide an implicit context without signaling notification).

[0108] (A11) In some embodiments of any of A8-A10, the context is selected based on the size of the 2D transform block. For example, when entropy coding is performed on EOB, the context for selecting the probabilistic model includes, but is not limited to, the size of the current 2D coefficient block.

[0109] (A12) In some embodiments of A11, the size of the 2D transform block includes the block width. For example, when entropy coding is performed on row-based EOBs, the context for selecting the probabilistic model depends on the block width (or height). In some embodiments, the maximum EOB value corresponds to a row, and the size corresponds to the row length (e.g., the number of transform coefficients per row in the current block).

[0110] (A13) In some embodiments of A11 or A12, the size of the 2D transform block includes the block height. For example, when entropy encoding a column-based EOB, the context for selecting the probabilistic model depends on the block height (or width). In some embodiments, the maximum EOB value corresponds to a column, and the size corresponds to the column length (e.g., the number of transform coefficients per column in the current block).

[0111] (A14) In some embodiments of any of A1-A13, the maximum EOB value is a first maximum EOB value corresponding to a first subset of the plurality of 1D transform coefficients. The method further includes: parsing a second maximum EOB value corresponding to a second subset of the plurality of 1D transform coefficients, wherein the 2D transform block is derived using the first maximum EOB value and the second maximum EOB value. For example, the set of multiple EOBs is encoded for multiple rows (or columns) in a 2D coefficient block.

[0112] (A15) In some embodiments of A14, the method: determines whether to use one or more maximum EOB values ​​to derive the 2D transform block based on indicators transmitted in the video bitstream. For example, the use of a single EOB value or multiple EOB values ​​can be signaled in a high-level syntax or block-level syntax (e.g., maximum coded block level flag, maximum coded block row (or column) level flag, coded block level flag, or transform block level flag).

[0113] (A16) In some embodiments of any of A1-A15, the method further includes: parsing from the video bitstream a set of transform skip flags corresponding to lines of the current block, each transform skip flag in the set indicating whether the corresponding line of the current block includes at least one non-zero transform coefficient. The 2D transform block is derived based on the set of transform skip flags and the maximum EOB value. For example, a 2D coefficient block consisting of 1D transforms is encoded using a set of "1D skip flags" from each row (or column) of the 2D coefficient block, wherein the skip flags indicate whether at least one non-zero coefficient exists in each 1D transform coefficient. In some embodiments, the set of transform skip flags is entropy-coded. In some embodiments, the context for selecting the probability model includes information from encoded neighboring transform blocks. In some embodiments, a decision to select a context from a top-side, left-side, or top-left neighboring transform block can be implicitly derived based on the 1D transform direction employed in the current 2D coefficient block. In some embodiments, a decision to select a context from a top-side, left-side, or top-left neighboring transform block can be implicitly derived based on the intra-frame prediction mode employed in the current 2D coefficient block. In some embodiments, when entropy encoding a 1D skip flag set, the context of the selection probability model includes, but is not limited to, the size of the current 2D coefficient block. For example, when entropy encoding a row-based EOB, the context of the selection probability model depends on the block width (or height). As another example, when entropy encoding a column-based EOB, the context of the selection probability model depends on the block height (or width). In some embodiments, if the 1D skip flag of a 1D transform coefficient is true, 1D EOBs with the same 1D coefficient are not encoded, and 1D EOBs with the same 1D coefficient are implicitly derived to 0 on both the encoder and decoder sides.

[0114] (A17) In some embodiments of A16, deriving the 2D transform block based on the transform skip flag set includes only decoding the lines of the current block where the transform skip flag set indicates that at least one transform coefficient is non-zero. For example, after decoding all 1D skip flags in the 2D coefficient block, only rows (or columns) of 1D coefficients whose flag value is false or 0 (or any other predefined value indicating the presence of at least one non-zero coefficient) are further decoded for non-zero coefficients.

[0115] (A18) In some embodiments of A16 or A17, the transform skip flag set includes 2D skip flags for the 2D transform block. The 2D skip flags indicate whether the 2D transform block includes at least one non-zero transform coefficient. For example, if a 2D skip flag implementation exists for the 2D coefficient block, and all 1D skip flags are true, then no 1D skip flags are encoded. Instead, the method uses existing 2D skip flags that can signal the absence of non-zero coefficients in the 2D coefficient block.

[0116] (B1) On the other hand, some embodiments include a video coding method (e.g., method 850). In some embodiments, the method is performed on a computing system including a memory and one or more processors. The method includes: (i) receiving video data comprising a plurality of blocks, the plurality of blocks including a current block; (ii) deriving a plurality of 1D transform coefficients corresponding to the current block; (iii) deriving a maximum EOB value of the plurality of 1D transform coefficients; (iv) transmitting the plurality of 1D transform coefficients in a video bitstream; and (v) transmitting the maximum EOB value in the video bitstream.

[0117] (C1) On the other hand, some embodiments include a visual media data processing method. In some embodiments, the method is executed on a computing system including a memory and one or more processors. The method includes: (i) obtaining a source video sequence comprising a plurality of images; and (ii) performing a conversion between the source video sequence and a bitstream of visual media data, wherein the bitstream includes: (a) a plurality of coded images corresponding to the plurality of images; (b) a plurality of 1D transform coefficients corresponding to a current block of the current image among the plurality of coded images; and (c) a maximum EOB value of the plurality of 1D transform coefficients.

[0118] (D1) On the other hand, some embodiments include a video decoding method. In some embodiments, the method is performed on a computing system including a memory and one or more processors. The method includes: (i) receiving a video bitstream comprising a plurality of blocks, the plurality of blocks including a current block; (ii) parsing a plurality of 1D transform coefficients corresponding to the current block from the video bitstream; (iii) parsing a set of transform skip flags corresponding to the plurality of 1D transform coefficients from the video bitstream, and indicating which rows for the current block include at least one non-zero transform coefficient; (iv) deriving a 2D transform block (e.g., a 2D coefficient block) using the plurality of 1D transform coefficients and the set of transform skip flags; and (v) reconstructing the current block using the derived 2D transform block.

[0119] (D2) In some embodiments of D1, the transform skip flag set includes one or more 1D transform skip flags. Each 1D transform skip flag indicates whether the corresponding line of the current block includes at least one non-zero transform coefficient.

[0120] (D3) In some embodiments of D1 or D2, the 2D transform block is derived using the plurality of 1D transform coefficients, the transform skip flag set, and at least one EOB value for the plurality of 1D transform coefficients.

[0121] (D4) In some embodiments of any of D1-D3, deriving the 2D transform block based on the transform skip flag set includes only decoding the line of the current block that indicates the transform skip flag set includes at least one non-zero transform coefficient.

[0122] (D5) In some embodiments of any of D1-D4, the transform skip flag set includes a 2D skip flag for a 2D transform block. The 2D skip flag indicates whether the 2D transform block includes at least one non-zero transform coefficient.

[0123] (D6) In some embodiments of any of D1-D5, the method further includes entropy coding of the transform skip flag set based on the coding information.

[0124] (D7) In some embodiments of D6, the encoding information includes: information of one or more adjacent transform blocks; the 1D transform direction of the current block; the intra-prediction mode of the current block; and the size of the 2D transform block.

[0125] On the other hand, some embodiments include a computing system (e.g., server system 112) that includes a control circuitry system (e.g., control circuitry system 302) and a memory (e.g., memory 314) coupled to the control circuitry system. The memory stores one or more sets of instructions configured to be executed by the control circuitry system, including instructions for performing any of the methods described herein (e.g., A1-A18, B1, C1, and D1-D7 above).

[0126] In another aspect, some embodiments include a non-transitory computer-readable storage medium that stores one or more sets of instructions executed by a control circuitry of a computing system, the set of instructions including instructions for performing any of the methods described herein (e.g., A1-A18, B1, C1, and D1-D7 above).

[0127] Unless otherwise stated, any syntax elements described herein (e.g., flags and indicators) may be High-Level Syntax (HLS). As used herein, HLS is transmitted at a level higher than the block level. For example, HLS may correspond to sequence level, frame level, slice level, or tile level. As another example, HLS elements may be transmitted in Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), Adaptation Parameter Set (APS), slice header, picture header, tile header, and / or CTU header.

[0128] It should be understood that while the terms “first,” “second,” etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the claims. As used in the description of embodiments and the appended claims, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations of one or more of the listed related items, and includes any and all possible combinations of one or more of the listed related items. It should also be further understood that, when used in this specification, the terms “comprising” and / or “including” specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0129] As used herein, the term "when" can be interpreted, depending on the context, as meaning "if," "after," "in response to determining," "according to determining," or "in response to detecting" that the prerequisite is true. Similarly, depending on the context, the phrases "if it is determined (that the prerequisite is true)," "if (that the prerequisite is true)," or "when (that the prerequisite is true)" can be interpreted as meaning "after determining that the prerequisite is true," "in response to determining that the prerequisite is true," "according to determining that the prerequisite is true," "after detecting that the prerequisite is true," or "in response to detecting that the prerequisite is true." As used herein, N refers to a variable. Unless explicitly stated otherwise, different instances of N may refer to the same number (e.g., the same integer value, such as the number 2) or different numbers. For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. In view of the above teachings, many modifications and variations are possible. The embodiments were chosen and described in order to best explain the operating principles and practical applications, thereby enabling others skilled in the art to implement them.

Claims

1. A video decoding method, executed on a computing system having memory and one or more processors, the method comprising: Receive a video stream, the video stream comprising multiple blocks, the multiple blocks including the current block; Parse multiple one-dimensional (1D) transform coefficients corresponding to the current block from the video stream; The maximum end-of-block (EOB) value of the plurality of 1D transform coefficients is parsed from the video stream; The two-dimensional (2D) transform block is derived using the maximum EOB value and the plurality of 1D transform coefficients; and Reconstruct the current block using the exported 2D transform block.

2. The method according to claim 1, wherein, The maximum EOB value is the 2D maximum EOB value for the 2D transform block, and wherein the maximum EOB value is derived using the corresponding 1D EOB value of the line set of the current block.

3. The method according to claim 2, wherein, Each corresponding 1D EOB value indicates the position of the last non-zero coefficient in the corresponding line of the line set.

4. The method according to claim 3, wherein, Transform coefficients located after the last non-zero coefficient are not sent in the bitstream.

5. The method according to claim 2, wherein, The 1D EOB value in the corresponding EOB value is an incremental value indicating the difference between the maximum EOB value of the corresponding line of the current block and the reference EOB value.

6. The method according to claim 5, wherein, The reference EOB value is the EOB value of a previous line in the line set.

7. The method according to claim 5, wherein, The reference EOB value is the maximum EOB value of the 2D transform block.

8. The method according to claim 1, wherein, Parsing the maximum EOB value involves entropy decoding of the maximum EOB value using the context of the encoded adjacent transform blocks.

9. The method according to claim 8, wherein, The context is selected based on the transformation type of the transformation used to generate the plurality of 1D transformation coefficients.

10. The method according to claim 8, wherein, The context is selected based on the intra-prediction mode used to encode the current block.

11. The method according to claim 8, wherein, The context is selected based on the size of the 2D transform block.

12. The method according to claim 11, wherein, The size of the 2D transform block includes the block width.

13. The method according to claim 11, wherein, The size of the 2D transform block includes the block height.

14. The method according to claim 1, wherein, The maximum EOB value is a first maximum EOB value corresponding to a first subset of the plurality of 1D transform coefficients; as well as The method further includes: resolving a second maximum EOB value corresponding to a second subset of a plurality of 1D transform coefficients, wherein the 2D transform block is derived using the first maximum EOB value and the second maximum EOB value.

15. The method of claim 14, further comprising: Whether to use one or more maximum EOB values ​​to derive the 2D transform block is determined based on the indicators sent in the video stream.

16. The method according to claim 1, further comprising: The transform skip flag set corresponding to the line of the current block is parsed from the video stream, and each transform skip flag in the transform skip flag set indicates whether the corresponding line of the current block includes at least one non-zero transform coefficient; as well as The 2D transform block is derived based on the transform skip flag set and the maximum EOB value.

17. The method according to claim 16, wherein, Deriving the 2D transform block based on the transform skip flag set includes decoding only the lines of the current block that indicate at least one transform coefficient of the transform skip flag set as non-zero.

18. The method according to claim 16, wherein, The transform skip flag set includes a 2D skip flag for the 2D transform block, the 2D skip flag indicating whether the 2D transform block includes at least one non-zero transform coefficient.

19. A computing system, comprising: Control circuit; Memory; as well as One or more instruction sets stored in the memory and configured to be executed by the control circuitry, the one or more instruction sets including instructions for the following operations: Receive video data, the video data comprising multiple blocks, the multiple blocks including the current block; Derive multiple one-dimensional (1D) transformation coefficients corresponding to the current block; Derive the maximum end-of-block (EOB) value of the plurality of 1D transform coefficients; Send the plurality of 1D transform coefficients in the video stream; as well as The maximum EOB value is sent in the video stream.

20. A non-transitory computer-readable storage medium storing one or more instruction sets configured to be executed by a computing device including control circuitry and memory, the one or more instruction sets including instructions for: Obtain a source video sequence, which includes multiple images; as well as Perform the conversion between the source video sequence and the bitstream of visual media data, wherein, The bitstream includes: Multiple encoded images corresponding to the multiple images; Multiple one-dimensional (1D) transform coefficients corresponding to the current block of the current image in the plurality of coded images; and The maximum end-of-block (EOB) value of the plurality of 1D transform coefficients.

Images