Method, apparatus, system and storage medium for video encoding and decoding

Abstract

Methods, apparatuses, systems, and storage medium for video encoding and decoding are provided. A method for video decoding includes receiving a coded video bitstream including a plurality of syntax elements; determining, by an arithmetic coding engine, a first maximum alphabet size for arithmetic coding, the first maximum alphabet size being determined based on hardware constraints; determining a second maximum alphabet size that is smaller than the first maximum alphabet size; and decoding the plurality of syntax elements included in the coded video bitstream based on the determined second maximum alphabet size, wherein each of the plurality of syntax elements is entropy coded with an alphabet size that is smaller than or equal to the determined second maximum alphabet size.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 358,309, filed July 5, 2022, and U.S. Application No. 17 / 983,876, filed November 9, 2022, the disclosures of which are incorporated herein by reference in their entirety. Technical Field

[0003] Embodiments of this disclosure relate to advanced video coding techniques, and more specifically, to methods, apparatus, systems, and storage media for video encoding and decoding. Background Technology

[0004] AOMedia Video 1 (AV1) is an open video coding format designed for video transmission over the Internet. It was developed by the Open Media Consortium (AOMedia) as a successor to VP9. AOMedia is a consortium founded in 2015 that includes semiconductor companies, video-on-demand providers, video content producers, software development companies, and web browser vendors. Many components of the AV1 project are derived from previous research by consortium members. Individual contributors began experimenting with technology platforms several years ago: Xiph / Mozilla's Daala released code in 2010, Google's experimental VP9 evolution project VP10 was announced on September 12, 2014, and Cisco's Thor released it on August 11, 2015. Building on the VP9 code, AV1 incorporates additional technologies, several of which were developed in these experimental forms. The first version 0.1.0 of the AV1 reference codec was released on April 7, 2016. The alliance released the AV1 bitstream specification and reference software-based encoders and decoders on March 28, 2018. On June 25, 2018, a confirmed version 1.0.0 of the specification was released. On January 8, 2019, a validated version 1.0.0 was released, which included errata table 1. The AV1 bitstream specification includes reference video codecs. Summary of the Invention

[0005] According to an embodiment, a method for video decoding is provided. The method includes: receiving an encoded video bitstream comprising a plurality of syntax elements; determining a first maximum alphabet size for arithmetic encoding by an arithmetic coding engine, the first maximum alphabet size being determined based on hardware constraints; determining a second maximum alphabet size smaller than the first maximum alphabet size; and decoding the plurality of syntax elements included in the encoded video bitstream based on the determined second maximum alphabet size, wherein each of the plurality of syntax elements is entropy encoded with an alphabet size less than or equal to the determined second maximum alphabet size. A method for video encoding is also provided, comprising: determining a first maximum alphabet size for arithmetic encoding by an arithmetic coding engine, the first maximum alphabet size being determined based on hardware constraints; determining a second maximum alphabet size smaller than the first maximum alphabet size; and encoding a plurality of syntax elements based on the determined second maximum alphabet size, wherein each of the plurality of syntax elements is entropy encoded with an alphabet size less than or equal to the determined second maximum alphabet size; and including the encoded plurality of syntax elements in the encoded video bitstream. A method for storing a video stream is also provided, wherein the video stream is stored on a non-volatile computer-readable storage medium, the video stream being generated according to the video encoding method of this application, or decoded according to the video decoding method of this application.

[0006] According to one or more embodiments, the second maximum alphabet size is an integer value between 2 and 15, including 2 and 15.

[0007] According to one or more embodiments, the plurality of syntax elements includes a first syntax element and a second syntax element, wherein the first syntax element is a set index indicating which set a selected symbol belongs to, and the second syntax element is an index indicating which symbol in the selected set is used for encoding or decoding.

[0008] According to one or more embodiments, the symbol is a level of motion vector difference, the alphabet size of the first syntax element is 2, and the total number of levels of motion vector difference in each set is 3 and 8, respectively.

[0009] According to one or more embodiments, the symbol is a level of motion vector difference, and the alphabet size of the first syntax element is 3, and the total number of levels of motion vector difference in each set is 3, 4 and 4 respectively.

[0010] According to one or more embodiments, each of the symbols indicates the absolute value of a scaling parameter from luminance to chrominance CfL mode.

[0011] According to one or more embodiments, each of the symbols indicates a mode of luminance intra-frame mode encoding.

[0012] According to one or more embodiments, each of the symbols indicates a mode of chroma intra-frame mode coding.

[0013] According to one or more embodiments, each of the symbols indicates a wedge-shaped wave pattern.

[0014] According to one or more embodiments, each of the symbols is a block end eob value.

[0015] According to an embodiment, a system is provided. The system includes: at least one memory configured to store computer program code; and at least one processor configured to access the computer program code and operate according to the instructions of the computer program code to perform the method according to this application. A video decoding apparatus is also provided, characterized in that the apparatus includes: a receiving module for receiving an encoded video bitstream including a plurality of syntax elements; a first determining module for determining a first maximum alphabet size for arithmetic encoding, the first maximum alphabet size being determined based on hardware constraints; a second determining module for determining a second maximum alphabet size smaller than the first maximum alphabet size; and a decoding module for decoding the plurality of syntax elements included in the encoded video bitstream based on the determined second maximum alphabet size, wherein each of the plurality of syntax elements is encoded with an alphabet size entropy less than or equal to the determined second maximum alphabet size. A video encoding apparatus is also provided, the apparatus comprising: a first determining module, wherein an arithmetic coding engine determines a first maximum alphabet size for arithmetic coding, the first maximum alphabet size being determined based on hardware constraints; a second determining module, wherein a second maximum alphabet size is determined to be smaller than the first maximum alphabet size; and an encoding module, configured to: encode a plurality of syntax elements based on the determined second maximum alphabet size, wherein each of the plurality of syntax elements is encoded with an alphabet size entropy less than or equal to the determined second maximum alphabet size; and include the encoded plurality of syntax elements in an encoded video bitstream.

[0016] According to one or more embodiments, the second maximum alphabet size is an integer value between 2 and 15, including 2 and 15.

[0017] According to one or more embodiments, the plurality of syntax elements includes a first syntax element and a second syntax element, wherein the first syntax element is a set index indicating which set a selected symbol belongs to, and the second syntax element is an index indicating which symbol in the selected set is used for encoding or decoding.

[0018] According to one or more embodiments, the symbol is a level of motion vector difference, the alphabet size of the first syntax element is 2, and the total number of levels of motion vector difference in each set is 3 and 8, respectively.

[0019] According to one or more embodiments, the symbol is a level of motion vector difference, and the alphabet size of the first syntax element is 3, and the total number of levels of motion vector difference in each set is 3, 4 and 4 respectively.

[0020] According to one or more embodiments, each of the symbols indicates the absolute value of a scaling parameter from luminance to chrominance CfL mode.

[0021] According to one or more embodiments, each of the symbols indicates a mode of luminance intra-frame mode encoding.

[0022] According to one or more embodiments, each of the symbols indicates a mode of chroma intra-frame mode coding.

[0023] According to one or more embodiments, each of the symbols indicates a wedge-shaped wave pattern.

[0024] According to an embodiment, a non-volatile computer-readable medium is provided, storing computer code. The computer code is configured to, when executed by at least one processor, cause the at least one processor to perform the method according to this application. Attached Figure Description

[0025] Other features, properties, and various advantages of the subject matter will become more apparent from the following detailed description and accompanying drawings.

[0026] Figure 1 This is a simplified block diagram of a communication system according to an embodiment.

[0027] Figure 2 This is a simplified block diagram of a communication system according to an embodiment.

[0028] Figure 3 This is a simplified block diagram of the decoder according to an embodiment.

[0029] Figure 4 This is a schematic diagram of a simplified block diagram of an encoder according to an embodiment.

[0030] Figure 5 This is an example flowchart of a process for decoding binary numbers according to an embodiment.

[0031] Figure 6 The adjacent blocks of the current block are shown in the luma intra-frame mode coding according to one embodiment.

[0032] Figure 7 This is a schematic diagram of the luminance-to-chrominance (CfL) prediction process.

[0033] Figure 8 It is a block diagram of computer code according to one embodiment.

[0034] Figure 9 This is a schematic diagram of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0035] The following detailed description of the exemplary embodiments is with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0036] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. Modifications and variations are possible according to this disclosure, or may be obtained from practice of the embodiments. Furthermore, one or more features or components of one embodiment may be incorporated into or combined with another embodiment (or one or more features of another implementation). Moreover, it will be understood from the flowcharts and operational descriptions provided below that one or more operations may be omitted, one or more operations may be added, one or more operations may be performed simultaneously (at least partially), and the order of one or more operations may be switched.

[0037] It is evident that the systems and / or methods described herein can be implemented in various forms of hardware, software, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit these implementations. It should be understood that software and hardware can be designed to implement the systems and / or methods based on the description herein.

[0038] The features described below can be used individually or in any combination in any order. Furthermore, embodiments can be implemented using processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-transient computer-readable medium.

[0039] Unless explicitly stated otherwise, no element, action, or instruction used herein should be construed as critical or necessary. Furthermore, as used herein, the terms “a” and “an” are intended to include one or more items and are used interchangeably with “one or more.” If used for only one item, the term “a” or similar language is used. Additionally, as used herein, the terms “has,” “have,” “having,” “include,” “including,” etc., are intended to be open-ended terms. Furthermore, unless explicitly stated otherwise, the word “based on” means “at least partially based on.” Moreover, expressions such as “at least one of A and B” or “at least one of A or B” should be understood to include only A, only B, or both A and B.

[0040] Figure 1 A simplified block diagram of a communication system 100 according to an embodiment of the present disclosure is shown. The communication system 100 may include at least two terminals 110 and 120 interconnected via a network 150. For unidirectional data transmission, the first terminal 110 may encode video data locally for transmission to the other terminal 120 via the network 150. The second terminal 120 may receive the encoded video data from the other terminal from the network 150, decode the encoded data, and display the recovered video data. Unidirectional data transmission is common in media service applications, etc.

[0041] Figure 1 A second pair of terminals 130 and 140 are shown, provided to support bidirectional transmission of encoded video, which may occur, for example, during a video conference. For bidirectional data transmission, each of terminals 130 and 140 can encode video data acquired at a local location for transmission to the other terminal via network 150. Each of terminals 130 and 140 can also receive encoded video data sent by the other terminal, can decode the encoded data, and can display the recovered video data on a local display device.

[0042] exist Figure 1 In the embodiments described, terminals 110, 120, 130, and 140 may be servers, personal computers, smartphones, and / or other types of terminals. For example, terminals 110-140 may be laptop computers, tablet computers, media players, and / or dedicated video conferencing equipment. Network 150 refers to any number of networks that transmit encoded video data between terminals 110, 120, 130, and 140, including, for example, wired (connected) and / or wireless communication networks. Communication network 150 may exchange data in circuit-switched and / or packet-switched channels. This network may include telecommunications networks, local area networks, wide area networks, and / or the Internet. For the purposes of this discussion, unless explained below, the architecture and topology of network 150 may be irrelevant to the operation of the present application.

[0043] As an example, Figure 2 The diagram illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0044] like Figure 2As shown, the streaming system 200 may include an acquisition subsystem 213, which may include a video source 201 and an encoder 203. The video source 201 may be, for example, a digital camera, and may be configured to create, for example, an uncompressed video sample stream 202. The uncompressed video sample stream 202 can provide a higher data volume compared to an encoded video stream and can be processed by the encoder 203 coupled to the video source 201. The encoder 203 may include hardware, software, or a combination thereof to implement or implement aspects of the disclosed subject matter as described in more detail below. An encoded video stream 204 may be stored on a streaming server 205 for future use; the encoded video stream 204 may include a lower data volume compared to the sample stream. One or more streaming clients 206 may access the streaming server 205 to retrieve a video stream 209, which may be, for example, a copy of the encoded video stream 204.

[0045] In this embodiment, the streaming server 205 can also be used as a Media-Aware Network Element (MANE). For example, the streaming server 205 can be configured to trim encoded video bitstreams 204 to customize potentially different bitstreams for one or more streaming clients 206. In this embodiment, the MANE can be provided separately from the streaming server 205 in the streaming system 200.

[0046] Streaming client 206 may include video decoder 210 and display 212. Video decoder 210 may, for example, decode video stream 209, which is an incoming copy 204 of the encoded video stream, and create outgoing video sample stream 211 that can be presented on display 212 or another presentation device (not shown). In some streaming systems, video streams 204 and 209 may be encoded according to certain video coding / compression standards. Examples of such standards include, but are not limited to, ITU-T Recommendation H.265. A video coding standard, informally referred to as Universal Video Coding (VVC), is under development. Embodiments of this disclosure can be used in the context of VVC.

[0047] Figure 3 An example functional block diagram of a video decoder 210 attached to a display 212 according to an embodiment of the present disclosure is shown.

[0048] The video decoder 210 may include a channel 312, a receiver 310, a buffer memory 315, an entropy decoder / parser 320, a scaler / inverse transform unit 351, an intra-frame image prediction unit 352, a motion compensation prediction unit 353, an aggregator 355, a loop filter unit 356, a reference image memory 357, and a current image memory (358). In at least one embodiment, the video decoder 210 may include an integrated circuit, a series of integrated circuits, and / or other electronic circuits. The video decoder 210 may also be partially or wholly embodied in software running on one or more CPUs having associated memory.

[0049] In this and other embodiments, receiver 310 may receive one or more encoded video sequences for decoding by decoder 210, one encoded video sequence at a time, wherein the decoding of each encoded video sequence is independent of other encoded video sequences. Encoded video sequences may be received from channel 312, which may be a hardware / software link to a storage device storing the encoded video data. Receiver 310 may receive encoded video data and other data, such as encoded audio data and / or auxiliary data streams, which may be forwarded to their respective user entities (not shown). Receiver 310 may separate the encoded video sequences from other data. To combat network jitter, buffer memory 315 may be coupled between receiver 310 and entropy decoder / resolver 320 (hereinafter referred to as the "resolver"). Buffer memory 315 may not be used or may be very small when receiver 310 is receiving data from a store / forward device with sufficient bandwidth and controllability, or from a peer-to-peer network. Buffer memory 315 may be necessary for use on best-effort packet networks such as the Internet, may be relatively large, and may have an adaptive size.

[0050] Video decoder 210 may include parser 320 to reconstruct symbols 321 from an encoded video sequence. Categories of these symbols may include, for example, information for managing the operation of video decoder 210, and potential information for controlling display devices such as display 212, which may be coupled to... Figure 2The decoder is shown. Control information for the display device may be, for example, a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). Parser 320 can perform parsing / entropy decoding on the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow principles well known to those skilled in the art, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. Parser 320 can extract a subgroup parameter set from the encoded video sequence for at least one subgroup of pixels in the subgroups used in the video decoder, based on at least one parameter corresponding to a group. Subgroups may include Group of Pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. Parser 320 can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0051] Parser 320 can perform entropy decoding / parsing operations on the video sequence received from buffer memory 315 to create symbol 321.

[0052] The reconstruction of symbol 321 may involve multiple different units, depending on the type of encoded video frame or its portion (e.g., inter-frame and intra-frame, inter-block and intra-block) and other factors. Which units are involved, and how they are involved, can be controlled by subgroup control information parsed by the parser from the encoded video sequence, 320. For clarity, the flow of such subgroup control information between the parser 320 and the multiple units is not described below.

[0053] In addition to the functional blocks already mentioned, decoder 210 can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide them into the functional units described below.

[0054] One unit is a scaler / inverse transform unit 351. Scaler / inverse transform unit 351 receives quantization transform coefficients as symbols 321 from parser 320, along with control information including the transform mode used, block size, quantization factor, and quantization scaling matrix. Scaler / inverse transform unit 351 can output blocks containing sample values, which can be input to aggregator 355.

[0055] In some cases, the output samples of the scaler / inverse transform unit 351 may belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed images, but can use predictive information from previously reconstructed portions of the current image. Such predictive information may be provided by the intra-picture prediction unit 352. In some cases, the intra-picture prediction unit 352 uses surrounding reconstructed information extracted from the current (partially reconstructed) image in the current image memory 358 to generate surrounding blocks of the same size and shape as the block being reconstructed. In some cases, the aggregator 355 adds the predictive information generated by the intra-picture prediction unit 352 to the output sample information provided by the scaler / inverse transform unit 351 based on each sample.

[0056] In other cases, the output samples of the scaler / inverse transform unit 351 may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit 353 can access the reference image memory 357 to extract samples for prediction. After motion compensation is performed on the extracted samples according to symbol 321, these samples can be added by the aggregator 355 to the output of the scaler / inverse transform unit 351 (referred to in this case as residual samples or residual signals) to generate output sample information. The motion compensation unit's retrieval of predicted samples from the address in the reference image memory 357 can be controlled by motion vectors. The motion vectors are used by the motion compensation prediction unit 353 in the form of symbol 321, which, for example, includes X, Y, and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory 357 when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0057] The output samples of aggregator 355 can be employed by various loop filtering techniques in loop filter unit 356. Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video bitstream, and these parameters can be used as symbols 321 from parser 320 in loop filter unit 356. However, in other embodiments, video compression techniques may also respond to metadata acquired during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0058] The output of the loop filter unit 356 can be a sample stream, which can be output to, for example, a display 212 and stored in a reference image memory 357 for subsequent inter-frame image prediction.

[0059] Once fully reconstructed, certain encoded images can be used as reference images for future predictions. Once the encoded images have been fully reconstructed and are identified as reference images (e.g., by parser 320), the current reference image can become part of the reference image memory 357, and a new current image memory can be reallocated before the reconstruction of subsequent encoded images begins.

[0060] The video decoder 210 can perform decoding operations according to a predetermined video compression technique, such as that documented in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard used; in this sense, it follows the syntax of the video compression technique or standard, as specified in the video compression technique document or standard, and specifically in the summary document therein. To conform to certain video compression techniques or standards, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum image size, maximum frame rate, maximum reconstruction sampling rate (measured, for example, megasamples per second), maximum reference image size, etc. In some cases, the limitations set by the hierarchy can be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0061] In this embodiment, receiver 310 may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by video decoder 210 to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0062] Figure 4 An example functional block diagram of a video encoder 203 associated with a video source 201 according to an embodiment of the present disclosure is shown.

[0063] The video encoder 203 may include, for example, an encoder as a source encoder 430, an encoding engine 432, a (local) decoder 433, a reference image memory 434, a predictor 435, a transmitter 440, an entropy encoder 445, a controller 450, and a channel 460.

[0064] The video encoder 203 can receive video samples from the video 201 (which is not part of the encoder), and the video source 201 can acquire video images to be encoded by the video encoder 203.

[0065] Video source 201 can provide a source video sequence in the form of a digital video sample stream, which will be encoded by video encoder 203. This digital video sample stream can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb4:2:0, YCrCb4:4:4). In a media service system, video source 201 can be a storage device storing previously prepared video. In a video conferencing system, video source 201 can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed sequentially. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples, depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0066] According to an embodiment, the video encoder 203 can encode and compress images of a source video sequence into an encoded video sequence 443 in real time or under any other time constraints required by the application. Implementing an appropriate encoding rate is a function of the controller 450. The controller 450 also controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller 450 may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. Other functions of the controller 450 can be readily identified by those skilled in the art, as they may relate to the video encoder 203 optimized for a particular system design.

[0067] Some video encoders operate within an encoding loop readily recognized by those skilled in the art. In a simplified description, the encoding loop may include the encoding portion of the source encoder 430 (responsible for creating symbols based on the input image to be encoded and the reference image) and a (local) decoder 433 embedded in the video encoder 203. The decoder 433 creates sample data that the (remote) decoder also creates (because in some video compression techniques, any compression between the symbols and the encoded video stream is lossless). The reconstructed sample stream is input to a reference image memory 434. Since the decoding of the symbol stream produces bit-precise results independent of the decoder's location (local or remote), the contents of the reference image memory also correspond bit-precisely between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction portion are exactly the same sample values ​​that the decoder will "see" during prediction. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is well known to those skilled in the art.

[0068] The operation of the “local” decoder 433 can be combined with, for example, the above. Figure 3 The “remote” decoder 210 described in detail is the same. However, when symbols are available and the entropy encoder 445 and the parser 320 are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the decoder 210, including the channel 312, receiver 310, buffer memory 315, and parser 320, may not be fully implemented in the local decoder 433.

[0069] It can be observed that any decoder technique other than parsing / entropy decoding, which exists in the decoder, must also exist in the corresponding encoder in essentially the same functional form. For this reason, the subject matter disclosed focuses on decoder operation. The description of encoder techniques can be simplified, as encoder techniques are inverses of the fully described decoder techniques. More detailed descriptions are only required in certain areas, and are provided below.

[0070] As part of its operation, the source encoder 430 can perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence designated as "reference frames." In this manner, the encoding engine 432 encodes the differences between pixel blocks in the input frame and pixel blocks in the reference frame, which can be selected as a prediction reference for the input frame.

[0071] The local decoder 433 can decode encoded video data of a frame that can be designated as a reference frame, based on symbols created by the source encoder 430. The operation of the encoding engine 432 can be a lossy process. When the encoded video data can be decoded by the video decoder ( Figure 4 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local decoder 433 replicates the decoding process, which can be performed by the video decoder on the reference frame, and allows the reconstructed reference frame to be stored in the reference image memory 434. In this way, the video encoder 203 can locally store a copy of the reconstructed reference frame that shares common content (no transmission errors) with the reconstructed reference frame to be acquired by the remote video decoder.

[0072] Predictor 435 can perform a prediction search against encoding engine 432. That is, for a new frame to be encoded, predictor 435 can search the reference image memory 434 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. Predictor 435 can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by predictor 435, it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory 434.

[0073] The controller 450 can manage the encoding operations of the source encoder 430, including, for example, setting parameters and subgroup parameters for encoding video data.

[0074] The outputs of all the aforementioned functional units can be entropy encoded in the entropy encoder 445. The entropy encoder performs lossless compression on the symbols generated by the various functional units using techniques known to those skilled in the art, such as Huffman coding, variable-length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0075] Transmitter 440 can buffer the encoded video sequence created by entropy encoder 445, thereby preparing it for transmission via communication channel 460, which may be a hardware / software link to a storage device storing the encoded video data. Transmitter 440 can combine the encoded video data from source encoder 430 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0076] The controller 450 manages the operation of the video encoder 203. During encoding, the controller 450 can assign a specific encoded picture type to each encoded picture, but this may affect the encoding technique applicable to the corresponding picture. For example, pictures can typically be assigned as intra-frame pictures (I-pictures), predictive pictures (P-pictures), or bidirectional predictive pictures (B-pictures).

[0077] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art are familiar with variations of I-pictures and their corresponding applications and characteristics.

[0078] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0079] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0080] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined based on the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or the blocks can be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be non-predictively coded with reference to a previously coded reference image via spatial prediction or temporal prediction. Blocks of a B-image can be non-predictively coded with reference to one or two previously coded reference images via spatial prediction or temporal prediction.

[0081] The video encoder 203 can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder 203 can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0082] In this embodiment, transmitter 440 may transmit additional data while transmitting encoded video. Source encoder 430 may include such data as part of the encoded video sequence. Additional data may include temporal / spatial / SNR enhancement layers, redundant images and slices, other forms of redundant data, Supplementary Enhancement Information (SEI) messages, fragments of Visual Usability Information (VUI) parameter sets, etc.

[0083] [Context-Adaptive Arithmetic Coding Engines in HEVC and VVC]

[0084] The context-adaptive arithmetic coding (CABAC) engines in HEVC and VVC can use a table-based probability transition process between 64 different representative probabilistic states. In HEVC, the range representing the state of the coding engine, ivlCurrRange, can be quantized into a set of four values ​​before computing a new range. HEVC state transitions can be implemented by approximating the value of ivlCurrRange*pLPS(pStateIdx) using a table containing all 64×4 8-bit pre-computed values, where pLPS is the probability of the least probable symbol (LPS) and pStateIdx is the index of the current state. Furthermore, decoding decisions can be implemented using a pre-computed lookup table (LUT). First, ivlLpsRange can be obtained using the LUT, as shown in Equation 1 below. Then, ivlLpsRange can be used to update ivlCurrRange and compute the output binVal.

[0085] ivlLpsRange=rangeTabLps[pStateIdx][qRangeIdx](Equation 1)

[0086] In VVC, probabilities can be linearly represented by probability indices pStateIdx. Therefore, all computations can be performed using equations without LUT operations. To improve the accuracy of probability estimation, a multi-hypothesis probability update model can be applied. The pStateIdx used in the interval subdivision of the binary arithmetic encoder can be a combination of two probabilities, pStateIdx0 and pStateIdx1. These two probabilities can be associated with each context model and can be updated independently with different adaptive rates. The adaptive rates of pStateIdx0 and pStateIdx1 for each context model can be pre-trained based on statistics of the associated binary numbers. The probability estimate pStateIdx can be the average of the estimates from the two hypotheses.

[0087] Figure 5 An embodiment of the process (500) for decoding a single binary decision is illustrated. Process 500 may begin at operation 502 to determine the value of the variable ivlCurrRange. At operation 504, if the variable ivlCurrRange is less than or equal to the variable ivlOffset, the process proceeds to operation 506 to update the values ​​of variables binVal, ivlOffset, and ivlCurrRange. If the variable ivlCurrRange is not less than the value of the variable ivlOffset, the process proceeds to operation 508 to update the value of the variable binVal. The process proceeds from operation 506 or operation 508 to operation 510 to update variables pStateIdx0 and pStateIdx1. The process proceeds to operation 512 to execute the RenormD procedure.

[0088] As performed in HEVC, VVC CABAC can also have a quantization parameter (QP)-related initialization procedure called at the beginning of each slice. Given initial values ​​for the brightness QP used for the slice, the initial probability state of the context model (denoted as preCtxState) can be derived, as shown in Equations 2 to 4 below:

[0089] m = slopeIdx × 5 - 45 (Equation 2)

[0090] n = (offsetIdx << 3) + 7 (Equation 3)

[0091] preCtxState=Clip3(1,127,((m×(QP-32))>>4)+n) (Equation 4)

[0092] The slopeIdx and offsetIdx can be limited to 3 bits, and the total initial value can be represented with 6 bits of precision. The probability state preCtxState can directly represent the probability in the linear domain. Therefore, referring to Equations 5 and 6 below, preCtxState may only require appropriate shift operations before being input into the arithmetic coding engine, and it stores the logarithmic-to-linear domain mapping and a 256-byte table.

[0093] pStateIdx0=preCtxState<<3 (Equation 5)

[0094] pStateIdx1=preCtxState<<7 (Equation 6)

[0095] [Multi-symbol arithmetic coding in AV1]

[0096] In AV1, an M-ary arithmetic coding engine can be used for entropy coding of syntax elements. Each syntax element can be associated with an alphabet of M elements, where M can be any integer value between 2 and 16. The input to the encoding can be M-ary symbols, and the encoding context can include a set of M probabilities represented by a cumulative distribution function (CDF). The probabilities can be updated after each syntax element is encoded / parsed. The cumulative distribution function can be an array of M 15-bit integers as shown in Equation 7 below:

[0097] C = [c0, c1, ..., c (M-2) ,2 15 (Equation 7)

[0098] Where c n / 32768 is the probability that the sign is less than or equal to n.

[0099] The probability update can be performed using the following equation 8:

[0100]

[0101] Where α is a probability update rate adapted based on the number of times the symbol has been decoded (e.g., at most 32), and m is the index of the element in the CDF.

[0102] This adaptive α allows for faster probability updates at the start of encoding / parsing syntax elements. The M-ary arithmetic coding process can follow a conventional arithmetic coding engine design. However, only the highest 9 significant bits of the 15-bit probability value can be input to the arithmetic encoder / decoder. The probability update rate α associated with a symbol can be calculated based on the number of occurrences of the associated symbol when parsing the bitstream, and the value of α can be reset at the beginning of a frame or tile using the following Equation 9:

[0103]

[0104] According to Equation 9, the probability update rate has a large value at the beginning and then saturates after 32 occurrences.

[0105] [Multi-hypothesis arithmetic coding]

[0106] A multi-hypothesis probabilistic model can be provided for encoding M-ary symbols. For example, one approach could include modifications to the AV1 arithmetic coding engine related to multi-hypothesis estimation. For instance, AV1 uses a data-adaptive model for probability updates, where the fewer occurrences of a syntax element, the higher the update rate, and the more observations, the lower the update rate. However, the engine uses only a single probability model. A multi-hypothesis estimation, where each syntax element maintains two or more probability tables with different update rates, can provide additional compression efficiency. Therefore, a multi-hypothesis probabilistic model can be implemented with two update rates as shown in Equations 10-11 below:

[0107]

[0108] α2 is modeled with faster updates, while α1 is modeled with slower updates. The final probabilistic model can be computed as a linear combination of the hypotheses. According to an embodiment, the average of the two hypotheses can be used. To accommodate symbolic statistics of changes across frames / tiles, alpha_1 and alpha_2 can be signaled at the frame / tile level.

[0109] [Intra-frame Luminosity Coding]

[0110] For a luma intra-coded block, all available intra-prediction modes can be reordered and inserted into the mode list before signaling, where the intra-prediction modes of two adjacent blocks can be involved in this reordering process. The positions of the two adjacent blocks are as follows: Figure 6 shown. Specifically, Figure 6 The diagram shows the current block 520, as well as the upper block 522 and the left block 524, which are adjacent blocks to the current block 520.

[0111] Specifically, the reordering process can be defined in the following three steps: (1) always first place the DC, SMOOTH, SMOOTH_V, SMOOTH_H, and PAETH modes in the mode list; (2) for each directional intra-prediction mode in adjacent blocks, derive 9 modes by adding offsets [0,-1,+1,-2,+2,-3,+3,-4,+4] to the directional modes, and add them to the mode list if they have not been added yet; and (3) if the mode list is not full, insert the predefined default mode into the mode list.

[0112] Following the reordering process described above, the intra-prediction modes can be divided into five mode sets based on the mode index. Each mode set can contain 5, 8, 16, 16, or 16 modes. Table 1 below illustrates the mapping between the reordered mode indices and the associated mode set indices.

[0113] Table 1: Mapping between schema index and schema set index

[0114]

[0115]

[0116] To signal the intra-prediction mode, a first syntax can be signaled to indicate the mode set index of the intra-prediction mode, and a second syntax can be signaled to indicate the mode index in the mode set.

[0117] According to the embodiment, incremental angles are allowed only when the block size in AV1 is equal to or greater than 8×8, while incremental angles are allowed for all block sizes in AI Music Creativity (AIMC). According to the embodiment, for both vertical and horizontal modes, offset-based intra prediction refinement (ORIP) can always be enabled for zero-reference lines and disabled for non-zero-reference lines.

[0118] [Chroma Intra-Frame Mode Coding]

[0119] Since the texture in the chroma component is typically smoother than that in the luma component, for each chroma block, 14 intra-prediction modes can be adaptively selected and included in only one set of intra-modes.

[0120] Specifically, the following aspects may be included in the embodiments:

[0121] (1) The context used to signal the chromaticity mode encoding can depend on whether the mode set index of the corresponding luminance mode is zero.

[0122] (2) The chroma nominal modes can be reordered based on the corresponding luma intra-prediction modes. Luma-to-chroma (CfL) modes can be added to the chroma intra-prediction mode list first. If the mode list is a directional intra-prediction mode list, the nominal modes of the corresponding luma intra-prediction modes can be added to the mode list. The remaining 12 nominal modes can be added to the mode list in the following order [DC, SMOOTH, SMOOTH_V, SMOOTH_H, PAETH, VERTICAL, HORIZONTAL, D45_PRED, D67_PRED, D113_PRED, D135_PRED, D157_PRED, D203_PRED].

[0123] (3) For the chroma component, the incremental angle can be derived from the corresponding luma mode without signaling. Specifically, if the chroma intra-prediction mode is a directional mode and equal to the intra-prediction mode of the corresponding luma block, the incremental angle of the chroma block can be set to be equal to the incremental angle of the corresponding luma block. Otherwise, the incremental angle of the chroma block can be set to zero.

[0124] [Chromaticity predicted based on luminance]

[0125] For the chroma component, in addition to 56 directional modes and 5 non-directional modes, CfL can also be provided. CfL is a chroma-only intra-prediction mode that models chroma pixels as a linear function of overlapping reconstructed luminance pixels. CfL prediction can be expressed as follows: Equation 12:

[0126] CfL(α)=α×L AC +DC (Equation 12)

[0127] Among them, L AC α represents the AC (alternating current) contribution of the luminance component, α represents the scaling parameter of the linear model, and DC represents the DC (direct current) contribution of the chrominance component.

[0128] Figure 7 This is a schematic diagram of the CfL prediction process 530. (Reference) Figure 7Following Equation 12, the reconstructed luminance pixels are subsampled to chroma resolution, and then the average value is subtracted to form the AC contribution. To approximate the chroma AC component from the AC contribution, the decoder does not need to compute a scaling parameter as in some background techniques. AV1CfL can determine the scaling parameter α based on the original chroma pixels and signal the scaling parameter in the bitstream. This reduces decoder complexity and produces more accurate predictions. The DC contribution of the chroma component can be computed using an intra-frame DC mode, which is sufficient for most chroma content and has a well-established, fast implementation.

[0129] When CfL mode is selected, the joint flag of the scaling factors for the U and V components can be signaled first. A scaling factor flag can be negative, zero, or positive. Additionally, the (zero, zero) combination may not be allowed in CfL mode because it leads to "DC" prediction. Therefore, there are a total of 8 (3*3-1=8) possible combinations of the flags for the two scaling factors. Thus, the joint flag may require 8 values. The joint flag can be signaled using only one context.

[0130] Regarding the signaling used for the magnitude of the scaling parameter, 16-value symbols can be used to represent values ​​ranging from 0 to 2 in steps of 1 / 8. 16-value symbols fully utilize the capabilities of the multi-symbol entropy encoder. The context for signaling the scaling parameter depends on the value of the joint flag.

[0131] Motion Vector Difference Encoding in AV1

[0132] AV1 allows for 1 / 8 pixel motion vector precision (or accuracy), and the following syntax elements can be used to signal the motion vector difference in reference frame list 0 or reference frame list 1.

[0133] The syntax element `mv_joint` can specify which components of the motion vector difference are non-zero. For example, a value of 0 can indicate that there is no non-zero MVD along the horizontal or vertical direction; a value of 1 can indicate that there is a non-zero MVD only along the horizontal direction; a value of 2 can indicate that there is a non-zero MVD only along the vertical direction; and a value of 3 can indicate that there is a non-zero MVD along both the horizontal and vertical directions.

[0134] The syntax element mv_sign can specify whether the motion vector difference is positive or negative.

[0135] The syntax element `mv_class` can specify the level of motion vector difference. As shown in Table 2 below, a higher level may mean that the motion vector difference has a larger amplitude.

[0136] Table 2: Amplitude Levels of Motion Vector Difference

[0137]

[0138]

[0139] The syntax element mv_bit can specify the integer part of the offset between the motion vector difference and the starting amplitude for each MV level.

[0140] The syntax element mv_fr can specify the first two fractions of the motion vector difference.

[0141] The syntax element mv_hp can specify the third fraction of the motion vector difference.

[0142] [Problems with multi-symbol arithmetic coding in AV1]

[0143] Regarding multi-symbol arithmetic coding in AV1, the M-ary arithmetic coding engine used in AV1 can support syntax elements of alphabet size M, where M can be any integer value between 2 and 16. In hardware, the arithmetic coding engine should be able to handle the worst case (alphabet size 16), which leads to increased chip area and reduced throughput.

[0144] [Example Implementation]

[0145] The embodiments of this disclosure can be used individually or in any combination in any order. Furthermore, each of the embodiments of this disclosure (e.g., methods, encoders, and decoders) can be implemented by processing circuitry (e.g., one or more processors or one or more integrated circuits). In one example, one or more processors execute a program stored on a non-volatile computer-readable medium.

[0146] Embodiments of this disclosure can reduce the complexity of the arithmetic coding engine by supporting a reduced alphabet size N (N-ary arithmetic coding engine), where N <= M. For this purpose, it may be required that the alphabet size of all entropy-coded syntax elements be <= N.

[0147] Embodiments of this disclosure may set the maximum alphabet size to be less than the hardware-imposed constraint.

[0148] Embodiments of this disclosure may limit the alphabet size of entropy-encoded syntax elements to less than or equal to N.

[0149] According to the embodiments, N can be any integer value between 2 and 16, such as 2, 4 or 8.

[0150] According to an embodiment, the syntax element `mv_class` defined in AV1 can have an alphabet size of 11. According to an embodiment, `mv_class` can specify the level of motion vector difference. If N is less than 16 (e.g., 4 or 8), new syntax elements `mv_class_set` and `mv_class_idx` can be defined to replace `mv_class`, where the alphabet size of `mv_class_set` and `mv_class_idx` can be less than or equal to N. This can be implemented by grouping the symbols of `mv_class` into multiple sets and providing an index (e.g., `mv_class_set`) for each symbol in one set. According to an embodiment, another index (e.g., `mv_class_idx`) can be provided / signaled to indicate which candidate in the selected set is being used for encoding and decoding. According to an embodiment, the encoder can perform the above functions (e.g., defining syntax elements based on the value of N to replace `mv_class`, grouping the symbols of `mv_class` into multiple sets, and providing an index for each symbol in the set), entropy-encode the syntax elements to obtain the encoded bitstream, and send the encoded bitstream to the decoder. According to an embodiment, the decoder can receive an encoded bitstream sent by the encoder, obtain syntax elements (and their symbols) from the encoded bitstream (e.g., by entropy decoding), and decode the video data of the encoded bitstream based on the syntax elements.

[0151] According to one or more embodiments, the alphabet size of mv_class_set is set (e.g., by the encoder) to 2, and the number of MV levels grouped in each mv_class_set is 3 and 8, respectively.

[0152] According to one or more embodiments, the alphabet size of mv_class_set is set (e.g., by the encoder) to 3, and the number of MV levels grouped in each mv_class_set is 3, 4, and 4, respectively.

[0153] According to an embodiment, the syntax elements cfl_alpha_u and cfl_alpha_v defined in AV1 have an alphabet size of 16. The syntax element cfl_alpha_u may contain the absolute value of alpha minus 1 for the U component, and the syntax element cfl_alpha_v may contain the absolute value of alpha minus 1 for the V component. According to an embodiment, "alpha" may be a scaling parameter used for the CfL mode (e.g., α in Equation 12). If N is less than 16, such as 4 or 8, new syntax elements cfl_alpha_u_set and cfl_alpha_u_idx may be defined to replace cfl_alpha_u, and new syntax elements cfl_alpha_v_set and cfl_alpha_v_idx may be defined to replace cfl_alpha_v, wherein the alphabet size of each of the new syntax elements is less than or equal to N. According to an embodiment, the encoder may perform the above functions (e.g., defining syntax elements to replace cfl_alpha_u and cfl_alpha_v based on the value of N) and send the encoded bitstream to the decoder. According to an embodiment, the decoder can receive an encoded bitstream sent by the encoder, obtain syntax elements (and their symbols) from the encoded bitstream (e.g., by entropy decoding), and decode the video data of the encoded bitstream based on the syntax elements.

[0154] According to one or more embodiments, candidate CfL alpha values ​​can be grouped into different sets (i.e., sets of CfL alpha values). Set indices (e.g., cfl_alpha_u_set, cfl_alpha_v_set) can be signaled to indicate which set a selected CfL alpha value belongs to, and another index (e.g., cfl_alpha_u_idx and cfl_alpha_v_idx) can be signaled to indicate which candidate in the selected set is being used for encoding and decoding. According to embodiments, the encoder can perform the above functions (e.g., defining syntax elements to replace cfl_alpha_u and cfl_alpha_v with values ​​based on N, grouping candidate CfL alpha values ​​into multiple sets), entropy-encode the syntax elements to obtain an encoded bitstream, and send the encoded bitstream to the decoder. According to embodiments, the decoder can receive the encoded bitstream sent by the encoder, obtain syntax elements (and their symbols) from the encoded bitstream (e.g., through entropy decoding), and decode the video data of the encoded bitstream based on the syntax elements.

[0155] According to an embodiment, candidate CfL alpha values ​​are defined from one or more of a predefined set of CfL alpha values.

[0156] According to embodiments, (e.g., by an encoder or decoder) candidate CfL alpha values ​​from one or more sets of CfL alpha values ​​are determined using encoded information, including but not limited to neighboring reconstructed sample values ​​of luminance and chrominance, alpha values ​​used in neighboring blocks, and block size / shape. In one example, one or more sets of CfL alpha values ​​define a set of the most probable CfL alpha values.

[0157] According to embodiments, (e.g., by an encoder or decoder) candidate CfL alpha values ​​in one or more sets of CfL alpha values ​​are determined based on the associated alpha value. For example, when two sets exist, one set may be configured to include all CfL alpha values ​​less than a threshold, and the remaining CfL alpha values ​​may be included in the other set. According to embodiments, the encoder and / or decoder may perform the above-described functions.

[0158] According to an embodiment, the set indexes (e.g., cfl_alpha_u_set, cfl_alpha_v_set) are context-encoded.

[0159] According to an embodiment, the indexes of which candidates in the selected set are being used (e.g., cfl_alpha_u_idx and cfl_alpha_v_idx) are encoded without any context (i.e., bypass encoding or encoding with fixed probability).

[0160] According to an embodiment, the set index can be determined based on encoded information, such as a previous CfL mode set index, or the luminance-reconstructed pixels and chrominance-reconstructed pixels of adjacent samples. In this case, the set index can be implicitly signaled. According to an embodiment, the encoder and / or decoder can determine the set index.

[0161] According to one or more embodiments, the alphabet size of cfl_alpha_u_set (and cfl_alpha_v_set) is set (e.g., by the encoder) to 2 (or 4), and the number of candidate CfL alpha values ​​in each set is 8 or 4.

[0162] According to an embodiment, the syntax elements mode_set_index and mode_idx associated with luma intra-frame mode coding have a alphabet size of <= N.

[0163] According to one or more embodiments, the value of mode_set_index is set (e.g., by the encoder) to 8. The number of modes in the first set is set (e.g., by the encoder) to 5, and the number of modes in the other sets is set (e.g., by the encoder) to 8.

[0164] According to an embodiment, the syntax elements uv_mode_set_index and uv_mode_idx associated with chroma intra-frame mode coding have a alphabet size of <= N.

[0165] According to one or more embodiments, the value of uv_mode_set_index is set (e.g., by the encoder) to 2, the number of modes in the first set is set (e.g., by the encoder) to 6, and the number of modes in the second set is set (e.g., by the encoder) to 8.

[0166] According to an embodiment, when signaling the wedge pattern index (i.e., wedge_idx), all possible wedge patterns can be grouped into different sets, and the number of candidate wedge patterns in each set may be less than or equal to N.

[0167] According to one or more embodiments, a signal can be sent to the first syntax element wedgelet_set to indicate which set of wedge patterns (i.e., the set of wedge patterns) is from the selected wedge pattern.

[0168] According to one or more embodiments, a second syntax wedgelet_idx can be signaled to indicate which candidate from the selected set of wedge patterns is chosen for encoding and decoding.

[0169] According to one or more embodiments, one or more sets of wedge patterns are generated (e.g., by an encoder or decoder) based on encoded information (including but not limited to adjacent reconstructed sample values, wedge patterns used in adjacent blocks, and block size / shape).

[0170] According to an embodiment, one or more sets of wedge wave patterns define a set of the most probable wedge wave patterns.

[0171] According to an embodiment, when the number of samples in the residual block is greater than a threshold T, multiple syntax elements are used to signal the eob value. The syntax element eob can be a variable indicating the index of the end of the block. According to an embodiment, the encoder can determine whether the number of samples in the residual block is greater than the threshold T, and based on the determination that the number of samples in the residual block is greater than the threshold T, signal the eob value in the encoded bitstream using multiple syntax elements.

[0172] According to one or more embodiments, the value of T is equal to 2^(N-1).

[0173] According to one or more embodiments, the range of eob values ​​is first classified into different groups, and the group index, i.e., eob_group_idx, is signaled first, and then (e.g., in the encoded bitstream) the selected entry of the selected group is further signaled to indicate the eob value. According to embodiments, the encoder can perform the above functions. According to embodiments, the decoder can receive the encoded bitstream sent by the encoder, obtain syntax elements from the encoded bitstream (e.g., via entropy decoding), and decode the video data of the encoded bitstream based on the syntax elements.

[0174] According to an embodiment, at least one processor and a memory storing computer program instructions (i.e., computer code) may be provided. When executed by the at least one processor, the computer program instructions may implement an encoder or decoder and may perform any number of functions described in this disclosure, including functions performed by an encoder or decoder.

[0175] For example, refer to Figure 8 At least one processor may implement encoder 702 or decoder 704. Computer program instructions may include, for example, encoding (or decoding) code 710, configured to cause at least one processor to encode (or decode) at least one image received according to embodiments of the present disclosure. Encoding (or decoding) code 710 may include, for example, alphabet determination code 720, grouping code 730, definition code 740, arithmetic encoding (or decoding) code 750, determination code 760, and transmission code 790. While example code causing certain functions to be performed is described herein, it should be understood that embodiments of the present disclosure may include other code for performing any other functions described herein.

[0176] According to embodiments of this disclosure, alphabet determination code 720 can be configured to cause at least one processor to determine a maximum alphabet size for arithmetic encoding (and / or decoding). For example, alphabet determination code 720 can be configured to cause at least one processor to determine a first maximum alphabet size for arithmetic encoding, the first maximum alphabet size being determined based on hardware constraints; and to determine a second maximum alphabet size smaller than the first maximum alphabet size based on the first maximum alphabet size.

[0177] Grouping code 730 can be configured to cause at least one processor to group symbols related to video coding into sets. According to embodiments, grouping can be based on determining that the maximum alphabet size is less than or equal to a predetermined value. For example, the symbols can be categories of motion vector differences, indications of the absolute value of scaling parameters for CfL modes, modes of luma intra-frame mode coding, modes of chroma intra-frame mode coding, wedge patterns, or eob values. According to embodiments, grouping can be used to group symbols of syntax elements (e.g., mv_class, cfl_alpha_u, and cfl_alpha_v) to replace the syntax elements.

[0178] According to embodiments of this disclosure, definition code 740 can be configured to cause at least one processor to define symbol-related syntax elements. According to embodiments, the definition can be based on determining that the maximum alphabet size is less than or equal to a predetermined value. According to embodiments, the newly defined syntax element can replace the existing syntax element.

[0179] According to embodiments of this disclosure, arithmetic encoding (or decoding) code 750 can be configured to cause at least one processor to perform arithmetic encoding on syntax elements to obtain an encoded video stream or to perform arithmetic decoding on syntax elements. For example, syntax elements can be encoded (or decoded) based on a determined second maximum alphabet size, wherein each of a plurality of syntax elements is entropy encoded (or decoded) with an alphabet size less than or equal to the determined second maximum alphabet size.

[0180] According to embodiments of this disclosure, determination code 760 can be configured to cause at least one processor to determine a set, a symbol and / or index in one or more sets based on information (e.g., encoded information).

[0181] According to embodiments of the present disclosure, when encoder 702 is provided, computer program instructions may further include, for example, transmission code 790, which may be configured to cause at least one processor to transmit a bitstream including the encoded image to a decoder (e.g., decoder 704). According to embodiments of the present disclosure, encoder 702 may provide information (e.g., syntax elements and their symbols) in the bitstream, such as the information described in the present disclosure.

[0182] The techniques described above can be implemented as computer software that uses computer-readable instructions and is physically stored on one or more computer-readable media. For example, Figure 9 A computer system 900 suitable for implementing embodiments of the disclosed subject matter is shown.

[0183] Computer software can be coded using any suitable machine code or computer language. Machine code or computer language can be created through assembly, compilation, linking or similar mechanisms to produce code that includes instructions that can be executed directly by a computer's central processing unit (CPU), graphics processing unit (GPU) or through interpretation, microcode execution, etc.

[0184] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0185] Figure 9 The components shown for the computer system 900 are exemplary in nature and are not intended to impose any limitation on the scope or functionality of the computer software implementing the embodiments of this disclosure. The configuration of the components should also not be construed as having any dependency or requirement on any component or combination thereof illustrated in the exemplary embodiments of the computer system 900.

[0186] Computer system 900 may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through, for example, tactile input (such as keystrokes, swipes, or movement with a data glove), audio input (such as speech or tapping), visual input (such as gestures), or olfactory input (not shown). Human-machine interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (such as speech, music, or ambient sound), images (such as scanned images or photographic images acquired from a still image camera), and video (such as two-dimensional video or three-dimensional video, including stereoscopic video).

[0187] The input human-machine interface device may include one or more of the following (only one of each is depicted): keyboard 901, mouse 902, touchpad 903, touch screen 910, data glove or joystick 905, microphone 906, scanner 907, camera 908.

[0188] Computer system 900 may also include certain human-machine interface (HMI) output devices. Such HMI output devices can stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. These HMI output devices may include tactile output devices (e.g., tactile feedback from a touchscreen 910, a data glove, or a joystick 905, but may also include tactile feedback devices that are not used as input devices). For example, these devices may be audio output devices (such as speakers 909, headphones (not depicted)), visual output devices (such as screens 910, including CRT screens, LCD screens, plasma screens, OLED screens, each with or without touchscreen input capabilities, each with or without tactile feedback capabilities—some of which can output two-dimensional or more than three-dimensional visual output in a manner such as stereoscopic output; virtual reality glasses (not depicted), holographic displays, and smoke canisters (not depicted)), and printers (not depicted).

[0189] The computer system 900 may also include human-accessible storage devices and their associated media, such as a CD / DVD ROM / RW 920 including media 921 such as CD / DVD, a thumb drive 922, a removable hard disk drive or solid-state drive 923, conventional magnetic media such as magnetic tapes and floppy disks (not depicted), and devices based on dedicated ROM / ASIC / PLD such as security dongles (not depicted).

[0190] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the presently disclosed subject matter does not include transmission media, carrier waves, or other volatile signals.

[0191] Computer system 900 may also include interfaces to one or more communication networks. These networks may be wireless, wired, or optical. They may also be local, wide area, metropolitan area, vehicular and industrial, real-time, latency-tolerant, etc. Examples of networks include local area networks (LANs) such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., wired or wireless wide area digital TV networks including cable TV, satellite TV, and terrestrial broadcast TV, and vehicular and industrial networks including CAN buses. Some networks typically require external network interface adapters (such as, for example, USB ports of computer system 900) to be attached to certain general-purpose data ports or peripheral buses 949; other networks are typically integrated into the core of computer system 900 by being attached to system buses as described below (e.g., Ethernet interfaces to PC computer systems or cellular network interfaces to smartphone computer systems). Using any of these networks, computer system 900 can communicate with other entities. Such communication can be unidirectional (receive only, e.g., broadcasting TV), unidirectional (transmit only, e.g., CANbus to certain CANbus devices), or bidirectional, e.g., to other computer systems using local area digital networks or wide area digital networks. Such communication can include communication to cloud computing environments. Certain protocols and protocol stacks can be used on each of the networks and network interfaces described above.

[0192] The aforementioned human-machine interface device, human-accessible storage device, and network interface 954 can be attached to the core 940 of the computer system 900.

[0193] Core 940 may include one or more central processing units (CPUs) 941, graphics processing units (GPUs) 942, dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) 943, hardware accelerators 944 for certain tasks, etc. These devices, along with read-only memory (ROM) 945, random access memory 946, and internal mass storage 947 such as internal non-user-accessible hard disk drives (SD drives) or SSDs, can be connected via system bus 948. In some computer systems, system bus 948 may be accessed as one or more physical connectors to allow for expansion by adding CPUs, GPUs, etc. Peripheral devices may be attached directly or via peripheral bus 949 to core system bus 948. Peripheral bus architectures include PCI, USB, etc. A graphics adapter 950 may be included in core 940.

[0194] The CPU 941, GPU 942, FPGA 943, and accelerator 944 can execute certain instructions, and combinations of these instructions can constitute the aforementioned computer code. This computer code can be stored in ROM 945 or RAM 946. Transitional data can also be stored in RAM 946, while permanent data can be stored, for example, in internal mass storage 947. Fast storage and retrieval of any memory device can be enabled by using cache memory, which can be closely associated with one or more CPUs 941, GPUs 942, mass storage 947, ROM 945, RAM 946, etc.

[0195] Computer-readable media may have computer code thereon for performing various computer-implemented operations. The media and computer code may be those specifically designed and constructed for the purposes of this disclosure, or they may be of types known and available to those skilled in the art of computer software.

[0196] As a non-limiting example, the architecture corresponding to computer system 900, particularly core 940, can provide functionality as a result of one or more processors (including CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, and certain memories of core 940 having non-volatile properties, such as internal mass storage 947 or ROM 945. Software implementing various embodiments of this disclosure can be stored in such devices and executed by core 940. Depending on specific needs, the computer-readable medium may include one or more memory devices or chips. The software can cause core 940, and particularly the processors therein (including CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM 946 and modifying such data structures according to software-defined processes. Alternatively or as an alternative, the computer system can provide functionality as a result of logical hard-wired connections or otherwise embodied in circuitry (e.g., accelerator 944), which can replace or operate with the software to execute specific processes or specific portions of specific processes described herein. Where appropriate, references to software may include logic, and vice versa. Where appropriate, references to computer-readable media may include circuitry (such as integrated circuits (ICs)) storing software for execution, circuitry embodying logic for execution, or both. This disclosure includes any suitable combination of hardware and software.

[0197] While this disclosure describes several non-limiting exemplary embodiments, variations, arrangements, and various alternative equivalents exist, all of which fall within the scope of this disclosure. Therefore, those skilled in the art will be able to design numerous systems and methods that, while not expressly shown or described herein, embody the principles of this disclosure and are therefore within its spirit and scope.

Claims

1. A method for video decoding, characterized in that, The method includes: Receives an encoded video stream that includes multiple syntax elements; The arithmetic coding engine determines the first maximum alphabet size M for arithmetic coding, and the first maximum alphabet size is determined based on hardware constraints; A second maximum alphabet size is determined to be less than or equal to the first maximum alphabet size, wherein the second maximum alphabet size is determined to be less than or equal to an integer value N, where N is the reduced alphabet size and N is less than or equal to M; and Based on the determined second maximum alphabet size, the plurality of syntax elements included in the encoded video bitstream are decoded, wherein each of the plurality of syntax elements is encoded with an alphabet size entropy less than or equal to the determined second maximum alphabet size. The plurality of syntax elements include a first syntax element and a second syntax element. The first syntax element is a set index indicating which set a selected symbol belongs to, and the second syntax element is an index indicating which symbol in the selected set is used for encoding or decoding. Determining the second maximum alphabet size includes: when N is determined to be less than 16, determining the second maximum alphabet size by replacing the original syntax element used to indicate the symbol with the first syntax element and the second syntax element.

2. The method according to claim 1, characterized in that, The second maximum alphabet size is an integer value between 2 and 16, inclusive.

3. The method according to claim 1, characterized in that, The symbol is the level of the motion vector difference, and The alphabet size of the first syntax element is 2, and the total number of levels of motion vector differences in each set is 3 and 8, respectively.

4. The method according to claim 1, characterized in that, The symbol is the level of the motion vector difference, and The alphabet size of the first syntax element is 3, and the total number of levels of motion vector differences in each set is 3, 4, and 4, respectively.

5. The method according to claim 1, characterized in that, Each symbol indicates the absolute value of the scaling parameter from luminance to chrominance CfL mode.

6. The method according to claim 1, characterized in that, Each of the symbols indicates a mode of luminance intra-frame mode coding.

7. The method according to claim 1, characterized in that, Each of the symbols indicates a mode of chroma intra-frame mode coding.

8. The method according to claim 1, characterized in that, Each of the symbols indicates a wedge-shaped wave pattern.

9. The method according to claim 1, characterized in that, Each of the symbols represents the block end eob value.

10. A video encoding method, characterized in that, include: The arithmetic coding engine determines the first maximum alphabet size M for arithmetic coding, and the first maximum alphabet size is determined based on hardware constraints; A second maximum alphabet size is determined to be less than or equal to the first maximum alphabet size, wherein the second maximum alphabet size is determined to be less than or equal to an integer value N, where N is the reduced alphabet size and N is less than or equal to M; and Based on the determined second maximum alphabet size, multiple syntax elements are encoded, wherein each of the multiple syntax elements is encoded with an alphabet size entropy that is less than or equal to the determined second maximum alphabet size. Include the encoded syntax elements in the encoded video stream; The plurality of syntax elements include a first syntax element and a second syntax element. The first syntax element is a set index indicating which set a selected symbol belongs to, and the second syntax element is an index indicating which symbol in the selected set is used for encoding or decoding. Determining the second maximum alphabet size includes: when N is determined to be less than 16, determining the second maximum alphabet size by replacing the original syntax element used to indicate the symbol with the first syntax element and the second syntax element.

11. A video encoding and decoding system, comprising: At least one memory is configured to store computer program code; as well as At least one processor is configured to access the computer program code and operate in accordance with the instructions of the computer program code to perform the method as claimed in any one of claims 1-10.

12. A video decoding apparatus, characterized in that, The device includes: The receiving module receives the encoded video stream, which includes multiple syntax elements; The first determining module determines the first maximum alphabet size M for arithmetic encoding, the first maximum alphabet size being determined based on hardware constraints; The second determining module determines a second maximum alphabet size that is less than or equal to the first maximum alphabet size. The second maximum alphabet size is determined to be less than or equal to an integer value N, where N is the reduced alphabet size and N is less than or equal to M. The decoding module decodes the plurality of syntax elements included in the encoded video stream based on the determined second maximum alphabet size, wherein each of the plurality of syntax elements is encoded with an alphabet size entropy less than or equal to the determined second maximum alphabet size. The plurality of syntax elements include a first syntax element and a second syntax element. The first syntax element is a set index indicating which set a selected symbol belongs to, and the second syntax element is an index indicating which symbol in the selected set is used for encoding or decoding. The second determining module determines the second maximum alphabet size by: when determining that N is less than 16, determining the second maximum alphabet size by replacing the original syntax element used to indicate the symbol with the first syntax element and the second syntax element.

13. A video encoding apparatus, characterized in that, The device includes: The first determining module determines the first maximum alphabet size M for arithmetic encoding by the arithmetic encoding engine. The first maximum alphabet size is determined based on hardware constraints. The second determining module determines a second maximum alphabet size that is less than or equal to the first maximum alphabet size. The second maximum alphabet size is determined to be less than or equal to an integer value N, where N is the reduced alphabet size and N is less than or equal to M. An encoding module is configured to: encode a plurality of syntax elements based on a determined second maximum alphabet size, wherein each of the plurality of syntax elements is encoded with an alphabet size entropy less than or equal to the determined second maximum alphabet size; and include the encoded plurality of syntax elements in an encoded video stream. The plurality of syntax elements include a first syntax element and a second syntax element. The first syntax element is a set index indicating which set a selected symbol belongs to, and the second syntax element is an index indicating which symbol in the selected set is used for encoding or decoding. The second determining module determines the second maximum alphabet size by: when determining that N is less than 16, determining the second maximum alphabet size by replacing the original syntax element used to indicate the symbol with the first syntax element and the second syntax element.

14. A non-volatile computer-readable medium storing computer code configured to, when executed by at least one processor, cause the at least one processor to perform the method as described in any one of claims 1-10.

15. A method for storing video streams, characterized in that, The video encoding method of claim 10 is used to generate a video stream and to store the video stream.

16. A method for transmitting a video stream, characterized in that, The video encoding method of claim 10 is used to generate a video stream and transmit the video stream.

Images