Video coding and decoding method, electronic device and computer readable storage medium

HK40098690BActive Publication Date: 2026-09-04TENCENT AMERICA LLC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
HK62024086664
Authority / Receiving Office
HK · HK
Patent Type
Patents
Current Assignee / Owner
Priority Date
2023-03-23
Filing Date
2024-02-05
Publication Date
2026-09-04
Estimated Expiration
2043-03-28

AI Technical Summary

Technical Problem

Existing video encoding and decoding technologies, when using spatial and temporal redundancy for compression, struggle to effectively utilize the spatially adjacent block information of the current block for context encoding, resulting in low encoding efficiency.

Method used

By determining the decoding parameters of the spatially adjacent blocks of the current block in the video decoder, a context-based adaptive binary arithmetic coding (CABAC) context model is constructed based on these parameters, and decoding is performed using the spatially adjacent block information of the current block.

Benefits of technology

It improves the efficiency of video encoding, reduces the amount of data, and enhances the encoding quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

A video coding method, an electronic device and a storage medium are provided. In the method, coded information of a current block is received from a coded video bitstream. The coded information includes a context coded syntax element associated with the current block. The syntax element indicates a decoding parameter of the current block. A context-based adaptive binary arithmetic coding (CABAC) context model associated with the context coded syntax element is determined based on decoding parameters of two or more spatial neighboring blocks of the current block. The context coded syntax element is decoded based on the determined CABAC context model associated with the context coded syntax element. The current block is reconstructed based on the decoded syntax element.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] References merged

[0002] This application claims priority to U.S. Patent Application No. 18 / 125,631, filed March 23, 2023, entitled "CABAC CONTEXT MODELING WITH INFORMATION FROM SPATIAL NEIGHBORS," which claims priority to U.S. Provisional Application No. 63 / 328,093, filed April 6, 2022, entitled "CABAC Context Modeling with Information from Spatial Neighbors." The disclosure of the earlier applications is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure describes embodiments that generally relate to video encoding and decoding. Background Technology

[0004] The background description provided herein is intended to provide a general overview of the context of this disclosure. With respect to the work described in this background section and to various aspects of this specification that may not qualify as prior art at the time of submission, the work of the currently designated inventors is neither expressly nor implicitly acknowledged as prior art to this disclosure.

[0005] Image / video compression helps transmit image / video files across different devices, storage, and networks while minimizing quality degradation. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. In one example, a video codec can use a technique called intra-frame prediction, which compresses images based on spatial redundancy. For example, intra-frame prediction can use reference data from the reconstructed current image for sample prediction. In another example, a video codec can use a technique called inter-frame prediction, which compresses images based on temporal redundancy. For example, inter-frame prediction can use motion compensation to predict samples in the current image from a previously reconstructed image. Motion compensation is typically indicated by motion vectors (MV). Summary of the Invention

[0006] This disclosure provides methods and apparatus for video encoding / decoding. In some examples, the apparatus for video decoding includes receiving circuitry and processing circuitry.

[0007] According to one aspect of this disclosure, a video decoding method performed in a video decoder is provided. In this method, encoded information of a current block can be received from an encoded video bitstream. The encoded information may include context-coded syntax elements associated with the current block. The syntax elements may indicate decoding parameters for the current block. A context-based adaptive binary arithmetic coding (CABAC) context model associated with the context-coded syntax elements can be determined based on the decoding parameters of two or more spatially adjacent blocks of the current block. The context-coded syntax elements can be decoded based on the determined CABAC context model associated with the context-coded syntax elements. The current block can be reconstructed based on the decoded syntax elements.

[0008] In some embodiments, two or more spatially adjacent blocks of the current block may include the upper right adjacent block, the upper top adjacent block, the upper left adjacent block, the left adjacent block, and the lower left adjacent block.

[0009] In some embodiments, two or more spatially adjacent blocks of the current block may include a column of adjacent blocks to the left of the current block and a row of adjacent blocks to the top of the current block.

[0010] In the example, the CABAC context model can be identified as one of four candidate context models assigned to the first binary number associated with the syntax element.

[0011] In the example, the CABAC context model can be determined as one of the four candidate context models based on (i) the number of spatial neighboring blocks of the current block is equal to or greater than a threshold, and (ii) the merged sub-block flag associated with each spatial neighboring block in the spatial neighboring block is false, or the inter-frame affine flag associated with each spatial neighboring block in the spatial neighboring block is false, or the local illumination compensation flag associated with each spatial neighboring block in the spatial neighboring block is false.

[0012] In some embodiments, the threshold can be equal to 4.

[0013] In the example, the CABAC context model can be determined as the fourth context model among the four candidate context models based on (i) the number of spatial neighboring blocks of the current block is equal to 4, and (ii) the merged sub-block flag associated with each spatial neighboring block in the spatial neighboring block is false, or the inter-frame affine flag associated with each spatial neighboring block in the spatial neighboring block is false, or the local illumination compensation flag associated with each spatial neighboring block in the spatial neighboring block is false.

[0014] In response to (i) the number of spatial neighbor blocks of the current block is not equal to or greater than a threshold, or (ii) the merged sub-block flag associated with each spatial neighbor block in the spatial neighbor block is true, or the inter-frame affine flag associated with each spatial neighbor block in the spatial neighbor block is true, or the local illumination compensation flag associated with each spatial neighbor block in the spatial neighbor block is true, a CABAC context model can be selected from the first candidate context model, the second candidate context model, or the third candidate context model among four candidate context models based on one of the first decoding parameters of the left neighbor block or the second decoding parameters of the upper neighbor block.

[0015] The CABAC context model can be selected based on (i) the availability of the left neighboring block, the first decoding parameter of the left neighboring block, or (ii) the availability of the upper neighboring block, the second decoding parameter of the upper neighboring block, and the spatial neighboring block.

[0016] In some embodiments, the syntax element may be one of the following: inter-frame affine flag, merged sub-block flag, local illumination compensation flag, non-inter-frame flag, coding unit skip flag, prediction mode flag, or inter-frame matrix-based intra-frame prediction (MIP) flag.

[0017] According to another aspect of this disclosure, an apparatus is provided. The apparatus includes processing circuitry. The processing circuitry can be configured to perform any of the described methods for video decoding / encoding.

[0018] This disclosure also provides a non-volatile computer-readable medium storing instructions that, when executed by a computer for video decoding, cause the computer to perform a method for video decoding / encoding. Attached Figure Description

[0019] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, wherein:

[0020] Figure 1 It is a schematic illustration of an exemplary block diagram of a communication system (100).

[0021] Figure 2 This is a schematic illustration of an exemplary block diagram of a decoder.

[0022] Figure 3 This is a schematic illustration of an exemplary block diagram of an encoder.

[0023] Figure 4 This is an exemplary flowchart of decoding binary numbers according to some embodiments of the present disclosure based on context-based adaptive binary arithmetic coding (CABAC).

[0024] Figure 5The example locations of the spatial neighbors of the current block are shown.

[0025] Figure 6 The example locations of the spatial neighbors of the current block are shown.

[0026] Figure 7 A flowchart outlining a decoding process according to some embodiments of this disclosure is shown.

[0027] Figure 8 A flowchart outlining the coding process according to some embodiments of this disclosure is shown.

[0028] Figure 9 This is a schematic illustration of an exemplary computer system according to an embodiment. Detailed Implementation

[0029] Figure 1 Block diagrams of video processing systems (100) in some examples of this application are shown. The video processing system (100) is an example application of the subject matter disclosed in this application, specifically a video encoder and video decoder in a streaming environment. The subject matter disclosed in this application is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc.

[0030] The video processing system (100) includes an acquisition subsystem (113) that may include a video source (101) such as a digital camera, which creates an uncompressed video image stream (102). In an embodiment, the video image stream (102) includes samples captured by a digital camera. The video image stream (102) is depicted as a thick line to emphasize the high data volume of the video image stream compared to encoded video data (104) (or encoded video bitstream). The video image stream (102) may be processed by an electronic device (120) that includes a video encoder (103) coupled to the video source (101). The video encoder (103) may include hardware, software, or a combination of hardware and software to implement or carry out aspects of the disclosed subject matter as described in more detail below. Compared to the video image stream (102), the encoded video data (104) (or the encoded video bitstream) is depicted as a thin line to emphasize the lower data volume of the encoded video data, which can be stored on a streaming server (105) for future use. One or more streaming client subsystems, such as Figure 1Client subsystems (106) and (108) can access a streaming server (105) to retrieve copies (107) and (109) of encoded video data (104). Client subsystem (106) may include, for example, a video decoder (110) in an electronic device (130). The video decoder (110) decodes the incoming copy (107) of the encoded video data and produces an output video picture stream (111) that can be displayed on a display (112) (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data (104), (107), and (109) (e.g., video streams) may be encoded according to certain video coding / compression standards. Examples of such standards include ITU-T H.265. In embodiments, the video coding standard under development is informally referred to as Versatile Video Coding (VVC), and this application can be used in the context of the VVC standard.

[0031] It should be noted that electronic devices (120) and (130) may include other components (not shown). For example, electronic device (120) may include a video decoder (not shown), and electronic device (130) may also include a video encoder (not shown).

[0032] Figure 2 A block diagram of an exemplary video decoder (210) is shown. The video decoder (210) may be disposed in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., receiving circuitry). The video decoder (210) may be used in place of... Figure 1 The video decoder (110) in the embodiment.

[0033] The receiver (231) may receive one or more encoded video sequences to be decoded by the video decoder (210). In one embodiment, one encoded video sequence is received at a time, wherein the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. The encoded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device storing the encoded video data. The receiver (231) may receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that may be forwarded to their respective user entities (not indicated). The receiver (231) may separate the encoded video sequences from other data. To prevent network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter referred to as "parser (220)"). In some applications, the buffer memory (215) is part of the video decoder (210). In other cases, the buffer memory (215) may be located outside the video decoder (210) (not indicated). In other cases, an external buffer (not shown) may be provided for the video decoder (210) to prevent network jitter, for example, and another buffer (215) may be configured internally for, for example, handling broadcast timing. When the receiver (231) receives data from a store / forward device with sufficient bandwidth and controllability or from an isochronous synchronization network, the buffer (215) may not be necessary, or it may be made smaller. Of course, for use on packet networks such as the Internet, a buffer (215) may also be required; this buffer may be relatively large and adaptive in size, and may be at least partially implemented in the operating system or a similar component (not shown) external to the video decoder (210).

[0034] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the encoded video sequence. These symbols may include information for managing the operation of the video decoder (210) and potential information for controlling a display device (212) (e.g., a display screen), which is not part of the electronic device (230) but may be coupled to it, such as... Figure 2As shown in the figure. The control information for the display device may be a parameter set fragment (not shown) of Supplemental Enhancement Information (SEI message) or Video Usability Information (VUI). The parser (220) may parse / entropy decode the received encoded video sequence. The encoding of the encoded video sequence may be based on video coding techniques or standards and may follow various principles, including variable-length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser (220) may extract a subgroup parameter set of at least one subgroup of pixels in the subgroup of pixels in the encoded video sequence for use in the video decoder based on at least one parameter corresponding to a group. The subgroup may include Group of Pictures (GOP), picture, tile, slice, macroblock, coding unit (CU), block, transform unit (TU), prediction unit (PU), etc. The parser (220) can also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0035] The parser (220) can perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to create symbols (221).

[0036] Depending on the type of encoded video frames or a subset of encoded video frames (e.g., inter-frame and intra-frame frames, inter-frame and intra-frame blocks) and other factors, the reconstruction of the symbol (221) may involve multiple different units. Which units are involved and how they are involved can be controlled by the subgroup control information parsed from the encoded video sequence by the parser (220). For brevity, the flow of such subgroup control information between the parser (220) and the various units described below is not described.

[0037] In addition to the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into several functional units as described below. In practical embodiments operating under commercial constraints, many of these units interact closely with each other and can be integrated with one another. However, for the purposes of describing the disclosed subject matter, it is appropriate to conceptually subdivide it into the functional units described below.

[0038] The first unit is the scaler / inverse transform unit (251). The scaler / inverse transform unit (251) receives quantization transform coefficients as symbols (221) and control information from the parser (220), including the transform method used, block size, quantization factor, quantization scaling matrix, etc. The scaler / inverse transform unit (251) can output a block containing sample values, which can be input into the aggregator (255).

[0039] In some cases, the output samples of the scaler / inverse transform unit (251) may belong to an intra-coded block. An intra-coded block is a block that does not use predictive information from a previously reconstructed image, but can use predictive information from a previously reconstructed portion of the current image. Such predictive information may be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) uses reconstructed information extracted from the current picture buffer (258) to generate a surrounding block of the same size and shape as the block being reconstructed. For example, the current picture buffer (258) buffers a partially reconstructed current image and / or a fully reconstructed current image. In some cases, the aggregator (255) adds the predictive information generated by the intra-picture prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) based on each sample.

[0040] In other cases, the output samples of the scaler / inverse transform unit (251) may belong to inter-frame coding and latent motion compensation blocks. In this case, the motion compensation prediction unit (253) can access the reference image memory (257) to extract samples for prediction. After motion compensation of the extracted samples according to the symbols (221), these samples can be added by the aggregator (255) to the output of the scaler / inverse transform unit (251) (referred to as residual samples or residual signals in this case) to generate output sample information. The motion compensation prediction unit (253) can obtain the predicted samples from the address in the reference image memory (257) under motion vector control, and the motion vector is available to the motion compensation prediction unit (253) in the form of the symbols (221), which, for example, include X, Y and reference image components. Motion compensation may also include interpolation of sample values ​​extracted from the reference image memory (257) when using subsample precise motion vectors, motion vector prediction mechanisms, etc.

[0041] The output samples of the aggregator (255) can be employed by various loop filtering techniques in the loop filter unit (256). Video compression techniques may include in-loop filtering techniques controlled by parameters included in the encoded video sequence (also referred to as the encoded video stream), and these parameters can be used as symbols (221) from the parser (220) in the loop filter unit (256). Video compression techniques may also respond to metadata obtained during decoding of a previous (in decoding order) portion of the encoded image or encoded video sequence, and to previously reconstructed and loop-filtered sample values.

[0042] The output of the loop filter unit (256) can be a sample stream, which can be output to a display device (212) and stored in a reference image memory (257) for subsequent inter-frame image prediction.

[0043] Once fully reconstructed, some of the encoded images can be used as reference images for future predictions. For example, once the encoded images corresponding to the current image have been fully reconstructed and the encoded images (by, for example, the parser (220)) are identified as reference images, the current image buffer (258) can become part of the reference image memory (257), and a new current image buffer can be reallocated before the reconstruction of subsequent encoded images begins.

[0044] The video decoder (210) can perform decoding operations according to a predetermined video compression technique, such as that specified in the ITU-T H.265 standard. The encoded video sequence may conform to the syntax specified by the video compression technique or standard in the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the configuration file recorded in the video compression technique or standard. Specifically, the configuration file may select certain tools from all available tools in the video compression technique or standard as the only tools available under said configuration file. For compliance, the complexity of the encoded video sequence is also required to be within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, maximum frame rate, maximum reconstruction sampling rate (measured in megasamples per second, for example), maximum reference picture size, etc. In some cases, the limitations set by the hierarchy can be further limited by the Hypothetical Reference Decoder (HRD) specification and the metadata managed by the HRD buffer, which is represented by signals in the encoded video sequence.

[0045] In this embodiment, the receiver (231) may receive additional (redundant) data along with the encoded video. The additional data may be a portion of the encoded video sequence. The additional data may be used by the video decoder (210) to properly decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant images, forward error correction codes, etc.

[0046] Figure 3 A block diagram of an exemplary video encoder (303) is shown. The video encoder (303) is disposed in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., transmission circuitry). The video encoder (303) can be used in place of... Figure 1 The video encoder (103) in the embodiment.

[0047] The video encoder (303) can obtain data from the video source (301) (not) Figure 3 In one embodiment, a portion of the electronic device (320) receives video samples, the video source being capable of capturing video images to be encoded by a video encoder (303). In another embodiment, the video source (301) is a portion of the electronic device (320).

[0048] A video source (301) can provide a sequence of source videos encoded by a video encoder (303) in the form of a digital video sample stream, which can have any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, etc.), any color space (e.g., BT.601YCrCb, RGB, etc.), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media service system, the video source (301) can be a storage device storing previously prepared video. In a video conferencing system, the video source (301) can be a camera that captures local image information as a video sequence. Video data can be provided as multiple individual pictures, which are given motion when viewed in sequence. The pictures themselves can be constructed as spatial pixel arrays, where each pixel can include one or more samples depending on the sampling structure, color space, etc., used. Those skilled in the art can easily understand the relationship between pixels and samples. The following focuses on describing samples.

[0049] According to an embodiment, the video encoder (303) can encode and compress images of a source video sequence into an encoded video sequence (343) in real time or under any other required time constraints. Implementing an appropriate encoding rate is a function of the controller (350). In some embodiments, the controller (350) controls and is functionally coupled to other functional units described below. For simplicity, coupling is not shown in the figures. Parameters set by the controller (350) may include rate control related parameters (image skipping, quantizer, λ value of rate-distortion optimization techniques, etc.), image size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) may be used with other suitable functions related to the video encoder (303) optimized for a particular system design.

[0050] In some embodiments, the video encoder (303) operates within an encoding loop. As a simplified description, in an embodiment, the encoding loop may include a source encoder (330) (e.g., responsible for creating symbols, such as a symbol stream, based on the input image to be encoded and a reference image) and a (local) decoder (333) embedded within the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data. The reconstructed sample stream (sample data) is input to a reference image memory (334). Since decoding of the symbol stream produces bit-accurate results regardless of the decoder's location (local or remote), the contents of the reference image memory (334) also correspond bit-accurately between the local and remote encoders. In other words, the reference image samples "seen" by the encoder's prediction section are exactly the same sample values ​​that the decoder will "see" during decoding when using the prediction. This fundamental principle of reference image synchronization (and the drift that occurs when synchronization cannot be maintained, for example, due to channel errors) is also used in some related techniques.

[0051] The operation of the “local” decoder (333) can be combined with, for example, the above-mentioned... Figure 2 The video decoder (210) is described in detail as the same as the "remote" decoder. However, a further brief reference is provided. Figure 2 When symbols are available and the entropy encoder (345) and parser (220) are able to encode / decode the symbols into an encoded video sequence without loss, the entropy decoding portion of the video decoder (210), including the buffer (215) and parser (220), may not be fully implemented in the local decoder (333).

[0052] In the embodiments, any decoder technology other than parsing / entropy decoding present in the decoder also exists in the corresponding encoder in the same or substantially the same functional form. Therefore, this application focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is the inverse of the fully described decoder technology. More detailed descriptions are provided in certain areas.

[0053] During operation, in some embodiments, the source encoder (330) may perform motion-compensated predictive coding. The motion-compensated predictive coding predictively encodes the input image, referencing one or more previously encoded images from the video sequence designated as "reference images." In this manner, the encoding engine (332) encodes the differences between pixel blocks of the input image and pixel blocks of the reference image, which may be selected as a predictive reference for the input image.

[0054] The local video decoder (333) can decode encoded video data of a picture that can be designated as a reference picture, based on symbols created by the source encoder (330). The operation of the encoding engine (332) can be a lossy process. When the encoded video data can be decoded by the video decoder (333), Figure 3 When the source video sequence (not shown) is decoded, the reconstructed video sequence can typically be a copy of the source video sequence with some errors. The local video decoder (333) replicates the decoding process, which can be performed by the video decoder on the reference image, and allows the reconstructed reference image to be stored in the reference image memory (334). In this way, the video encoder (303) can locally store a copy of the reconstructed reference image that shares the same content (no transmission errors) as the reconstructed reference image to be obtained by the remote video decoder.

[0055] The predictor (335) can perform a prediction search against the encoding engine (332). That is, for a new image to be encoded, the predictor (335) can search in the reference image memory (334) for sample data (as candidate reference pixel blocks) or certain metadata, such as reference image motion vectors, block shapes, etc., that can serve as appropriate prediction references for the new image. The predictor (335) can operate pixel-by-pixel based on the sample blocks to find suitable prediction references. In some cases, based on the search results obtained by the predictor (335), it can be determined that the input image may have prediction references obtained from multiple reference images stored in the reference image memory (334).

[0056] The controller (350) can manage the encoding operations of the source encoder (330), including, for example, setting parameters and subgroup parameters for encoding video data.

[0057] The outputs of all the above-mentioned functional units can be entropy encoded in the entropy encoder (345). The entropy encoder (345) performs lossless compression on the symbols generated by the various functional units according to techniques such as Huffman coding, variable length coding, and arithmetic coding, thereby converting the symbols into an encoded video sequence.

[0058] The transmitter (340) can buffer the encoded video sequence created by the entropy encoder (345) in preparation for transmission via a communication channel (360), which may be a hardware / software link to a storage device that will store the encoded video data. The transmitter (340) can combine the encoded video data from the video encoder (303) with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown).

[0059] The controller (350) manages the operation of the video encoder (303). During encoding, the controller (350) can assign a specific encoded image type to each encoded image, but this may affect the encoding techniques applicable to the corresponding images. For example, images can typically be assigned to any of the following image types:

[0060] An intra-frame picture (I-picture) is a picture that can be encoded and decoded without using any other pictures in the sequence as a prediction source. Some video codecs allow different types of intra-frame pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those skilled in the art will understand variations of I-pictures and their corresponding applications and characteristics.

[0061] A predictive image (P-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most one motion vector and a reference index to predict sample values ​​for each block.

[0062] A bidirectional predictive image (B-image) can be an image that can be encoded and decoded using intra-frame prediction or inter-frame prediction, which uses at most two motion vectors and a reference index to predict sample values ​​for each block. Similarly, multiple predictive images can use more than two reference images and associated metadata to reconstruct a single block.

[0063] Source images are typically spatially subdivided into multiple sample blocks (e.g., 4×4, 8×8, 4×8, or 16×16 sample blocks), and encoded block by block. These blocks can be predictively coded with reference to other (already coded) blocks, determined based on the coding assignments of the corresponding images applied to the blocks. For example, blocks of an I-image can be non-predictively coded, or the blocks can be predictively coded (spatial or intra-frame prediction) with reference to already coded blocks of the same image. Pixel blocks of a P-image can be predictively coded with reference to a previously coded reference image via spatial or temporal prediction. Blocks of a B-image can be predictively coded with reference to one or two previously coded reference images via spatial or temporal prediction.

[0064] The video encoder (303) can perform encoding operations according to a predetermined video coding technique or standard, such as ITU-T H.265 Recommendation. In operation, the video encoder (303) can perform various compression operations, including predictive coding operations that utilize temporal and spatial redundancy in the input video sequence. Therefore, the encoded video data can conform to the syntax specified by the video coding technique or standard used.

[0065] In this embodiment, the transmitter (340) may transmit additional data while transmitting encoded video. The source encoder (330) may include such data as part of the encoded video sequence. The additional data may include other forms of redundant data such as temporal / spatial / SNR enhancement layers, redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0066] The acquired video can serve as multiple source images (video images) presented in a time series. Intra-frame image prediction (often simplified to intra-frame prediction) utilizes spatial correlations within a given image, while inter-frame image prediction utilizes (temporal or other) correlations between images. In an embodiment, a specific image being encoded / decoded is segmented into blocks, referred to as the current image. When a block in the current image resembles a reference block in a previously encoded and still buffered reference image in the video, the block in the current image can be encoded using a vector called a motion vector. This motion vector points to the reference block in the reference image, and when multiple reference images are used, the motion vector may have a third dimension that identifies the reference image.

[0067] In some embodiments, bidirectional prediction techniques can be used in inter-frame image prediction. According to bidirectional prediction, two reference images are used, such as a first reference image and a second reference image, both preceding the current image in the video in decoding order (but possibly past and future in display order). A block in the current image can be encoded using a first motion vector pointing to a first reference block in the first reference image and a second motion vector pointing to a second reference block in the second reference image. Specifically, the block can be predicted using a combination of the first and second reference blocks.

[0068] In addition, merging mode techniques can be used in inter-frame image prediction to improve coding efficiency.

[0069] According to some embodiments disclosed in this application, predictions such as inter-frame image prediction and intra-frame image prediction are performed on a block-by-block basis. For example, according to the HEVC standard, images in a video image sequence are segmented into coding tree units (CTUs) for compression. The CTUs in the images have the same size, such as 64×64 pixels, 32×32 pixels, or 16×16 pixels. Generally, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Furthermore, each CTU can be further subdivided into one or more coding units (CUs) using a quadtree. For example, a 64×64 pixel CTU can be subdivided into one 64×64 pixel CU, or four 32×32 pixel CUs, or sixteen 16×16 pixel CUs. In embodiments, each CU is analyzed to determine the prediction type used for the CU, such as inter-frame prediction or intra-frame prediction. Furthermore, depending on temporal and / or spatial predictability, the CU is divided into one or more prediction units (PUs). Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In embodiments, prediction operations in encoding (encoding / decoding) are performed on a per-prediction-block basis. Taking a luma prediction block as an example, a prediction block includes a matrix of pixel values ​​(e.g., luma values), such as 8×8 pixels, 16×16 pixels, 8×16 pixels, 16×8 pixels, etc.

[0070] It should be noted that any suitable technology can be used to implement the video encoders (103) and (303) and the video decoders (110) and (210). In one embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more integrated circuits. In other embodiments, the video encoders (103) and (303) and the video decoders (110) and (210) can be implemented using one or more processors that execute software instructions.

[0071] This disclosure includes embodiments related to entropy coding. For example, a context model of context-based adaptive binary arithmetic coding (CABAC) associated with the current block can be determined based on information from spatially neighboring blocks of the current block.

[0072] Local Illumination Compensation (LIC) is an inter-frame prediction technique that models the local illumination variations between the current block and its predicted blocks. Local illumination variations can be modeled as a function of the local illumination variations between the current block template and the reference block template. The parameters of this function can be represented by a scale α and an offset β, which can form a linear equation α*p[x]+β to compensate for the illumination variations, where p[x] is the reference sample pointed to by the MV at position x on the reference image. Since α and β can be derived based on the current block template and the reference block template, no signaling overhead is required for the current block template and the reference block template. However, it may be necessary to signal the use of LIC for Advanced Motion Vector Prediction (AMVP) mode.

[0073] The local illumination compensation previously proposed outside of VVC can be used for single-prediction inter-frame CU, but the following modifications are required:

[0074] (1) Intra-frame neighbor samples can be used in the derivation of LIC parameters;

[0075] (2) Disable LIC for blocks with fewer than 32 luminance samples;

[0076] (3) LIC parameters can be derived for both non-sub-block mode and affine mode based on template block samples corresponding to the current CU, rather than partial template block samples corresponding to the first upper left 16x16 unit; and

[0077] (4) Samples of the reference block template are generated by using block MV-based motion compensation (MC). The block MV may not be rounded to integer pixel precision.

[0078] For example, in VVC, compared to the design in HEVC, CABAC can include the following main changes:

[0079] (1) Core CABAC engine;

[0080] (2) Separate the residual coding structure used for transform blocks and transform skip blocks; and

[0081] (3) Context modeling for transformation coefficients.

[0082] The CABAC engine in HEVC uses a table-based probability transition process between 64 distinct representative probability states. In HEVC, the current interval range (e.g., ivlCurrRange) representing the state of the coding engine can be quantized into a set of four values ​​before computing a new interval range. HEVC state transitions can be implemented using a table containing all 64x4 8-bit pre-computed values ​​to approximate the value of ivlCurrRange*pLPS(pStateIdx), where pLPS is the probability of the least likely symbol (LPS) and pStateIdx is the index of the current state. Furthermore, decoding decisions can be implemented using a pre-computed lookup table (LUT). For example, the LPS range (e.g., ivlLpsRange) can be determined based on the LUT, as shown in equation (1) below. The LPS range ivlLpsRange can be further used to update the current interval range ivlCurrRange and compute the output binary value (e.g., binVal).

[0083] ivlLpsRange=rangeTabLps[pStateIdx][qRangeIdx] Equation (1)

[0084] For example, in VVC, probabilities can be linearly represented by the probability index pStateIdx. Therefore, all computations can be performed using equations without LUT operations. To improve the accuracy of probability estimation, a multi-hypothesis probability update model can be applied. The pStateIdx used in the interval segmentation of the binary arithmetic encoder can be a combination of two probabilities, pStateIdx0 and pStateIdx1. These two probabilities can be associated with each context model in CABAC and can be further updated independently with different adaptive rates. The adaptive rates of pStateIdx0 and pStateIdx1 for each context model can be pre-trained based on statistics of the associated binary numbers. The probability estimate of pStateIdx can be the average of the estimates from the two hypotheses (e.g., pStateIdx0 and pStateIdx1).

[0085] Figure 4An exemplary flowchart (400) for decoding a single binary decision (e.g., decoding a binary number) in a VVC is shown. As shown in flowchart (400), at (S410), input variables of a context table (e.g., ctxTable) and a context index (e.g., ctxIdx) can be received. In some embodiments, the input variables may also include the current interval range (e.g., ivlCurrRange) and the interval offset (e.g., ivlOffset). At (S420), variables of the quantization range index (e.g., qRangeIdx), the probability state (e.g., pState), the value of the maximum possible symbol (e.g., valMps), and the minimum possible symbol interval range (e.g., ivlLpsRange) can be determined, and ivlCurrRange can be updated to ivlCurrRange-ivlLpsRange. At (S430), when ivlOffset is less than ivlCurrRange, as shown in (S450), the binary value (e.g., binVal) can be determined as valMps. When ivlOffset is equal to or greater than ivlCurrRange, as shown in (S440), the binary value (e.g., binVal) can be determined as 1-valMps, ivlOffset can be decremented by ivlCurrRange, and ivlCurrRange can be set to be equal to ivlLpsRange. At (S460), pStateIdx0 and pStateIdx0 can be updated based on the parameters shift0, shift1, and the binary value. At (S370), renormalization processing (e.g., RenormD) in the decoder can be performed based on the determined pStateIdx0 and pStateIdx0.

[0086] Similar to HEVC, VVC CABAC can also have a quantization parameter (QP) dependent initialization process invoked at the beginning of each strip. Given the initial value of the brightness QP of the strip, the initial probability state of the context model (denoted as preCtxState) can be derived using equations (2) to (4) as follows.

[0087] m = slopeIdx × 5 – 45 Equation (2)

[0088] n = (offsetIdx << 3) + 7 Equation (3)

[0089] preCtxState = Clip3(1, 127, ((m × (QP - 32)) >> 4) + n) Equation (4)

[0090] In this context, slopeIdx and offsetIdx can be restricted to 3 bits, and the total initial value is represented by 6 bits of precision. The initial probability state preCtxState of the context model directly represents the probability in the linear domain. Therefore, preCtxState may only require appropriate shift operations before it is input into the arithmetic coding engine (e.g., CABAC). Accordingly, the logarithmic-to-linear domain mapping and the 256-byte table can be saved (or skipped). pStateIdx0 and pStateIdx1 can be further determined based on preCtxState, as shown in the following equations (5) and (6):

[0091] pStateIdx0 = preCtxState << 3 Equation (5)

[0092] pStateIdx1 = preCtxState << 7 Equation (6)

[0093] In the CABAC decoding process, the input to the decoding process can be all binary strings of the syntax element. The binary strings of the syntax element can be obtained based on the binaryization process. The output of the decoding process can be the value of the syntax element. The decoding process can specify how to parse each binary number of the binary string for each syntax element. After parsing each binary number, the resulting binary string (or the parsed binary string) can be compared with all binary strings of the syntax element obtained based on the binaryization process. The result of the comparison can be determined as follows: (1) If the binary string (or the resulting binary string) is equal to one of the binary strings, the corresponding value of the syntax element is the output. (2) Otherwise (e.g., the binary string is not equal to one of the binary strings), the next bit is parsed. When parsing each binary number, the variable binIdx is incremented by 1, starting from setting binIdx to equal 0 for the first binary number. The parsing of each binary number is specified by two ordered steps: (1) The derivation process of ctxTable, ctxIdx, and bypassFlag can be invoked with binIdx as input and ctxTable, ctxIdx, and bypassFlag as output. (2) The arithmetic decoding process can be invoked using ctxTable, ctxIdx and bypassFlag as inputs and the value of the binary number as output.

[0094] During the derivation of ctxTable, ctxIdx, and bypassFlag, the input to the derivation process can be the position of the current binary number in the binary string. The position of the current binary number in the binary string can be represented as a binary number index (e.g., binIdx). The output of the derivation process can be ctxTable, ctxIdx, and bypassFlag. The values ​​of ctxTable, ctxIdx, and bypassFlag can be derived based on the entries (or values) of the binIdx of the corresponding syntax elements in Table 1, as follows.

[0095] Table 1. Assigning ctxInc to syntax elements with context-encoded binary numbers

[0096]

[0097]

[0098] If an entry in Table 1 is not equal to "Bypass", "End", or "None", the value of binIdx can be decoded by calling the DecodeDecision procedure. As shown in Table 1, the variable ctxInc can be specified by the corresponding entry in Table 1. When more than one value is listed for binIdx in Table 1, the allocation procedure for ctxInc for the corresponding binIdx can be further specified in the entry given in parentheses. For the current value of initType and the current syntax element, the variable ctxIdxOffset can be set to the minimum value equal to ctxIdx. ctxIdx can be set to the sum of ctxInc and ctxIdxOffset. bypassFlag can be set to 0.

[0099] If the entry in Table 1 equals "bypass", the value of binIdx can be decoded by calling the DecodeBypass procedure. Therefore, ctxTable can be set to 0. ctxIdx can be set to 0. bypassFlag can be set to 1.

[0100] If the entry in Table 1 equals "End", the value of binIdx can be decoded by calling the DecodeTerminate procedure. Therefore, ctxTable can be set to 0. ctxIdx can be set to 0. bypassFlag can be set to 0.

[0101] If an entry in Table 1 is equal to "None", then the corresponding syntax element will not have a value for binIdx.

[0102] During the derivation of ctxInc using the left-hand and top-hand syntax elements, the input to the derivation can be the luminance position (x0, y0), specifying the top-left luminance sample of the current luminance block relative to the top-left sample of the current image, the color component cIdx, the current coding quadtree depth cqtDepth, the dual-tree channel type chType, the width cbWidth and height cbHeight of the current coding block in the luminance sample, and variables derived in the coding tree semantics. Variables can include allowSplitBtVer, allowSplitBtHor, allowSplitTtVer, allowSplitTtHor, and allowSplitQt. The output of the derivation can be the context index ctxInc.

[0103] In some embodiments, the position (xNbL, yNbL) can be set to equal (x0-1, y0), and the derivation process for the availability of neighboring blocks can be invoked based on setting the position (xCurr, yCurr) to equal (x0, y0), setting the neighboring position (xNbY, yNbY) to equal (xNbL, yNbL), setting checkPredModeY to equal FALSE, and setting cIdx as input. The output of the derivation process can be assigned to availableL.

[0104] In some embodiments, the position (xNbA, yNbA) can be set to equal (x0, y0-1), and the derivation process for the availability of neighboring blocks can be invoked based on setting the position (xCurr, yCurr) to equal (x0, y0), setting the adjacent position (xNbY, yNbY) to equal (xNbA, yNbA), setting checkPredModeY to equal FALSE, and setting cIdx as input. The output of the derivation process can be assigned to availableA.

[0105] In the example, the position (xCtb, yCtb) can be set to equal (x0>>CtbLog2SizeY, y0>>CtbLog2SizeY), the position (xCtbA, yCtbA) can be set to equal (xNbA>>CtbLog2SizeY, yNbA>>CtbLog2SizeY), and the position (xCtbL, yCtbL) can be set to equal (xNbL>>CtbLog2SizeY, yNbL>>CtbLog2SizeY).

[0106] The allocation of ctxInc can be specified using condL and condA as specified in Table 2. Table 2 shows the specification of ctxInc using left and top syntax elements. As shown in Table 2, for the syntax elements alf_ctb_flag[cIdx][xCtb][yCtb], alf_ctb_cc_cb_idc[xCtb][yCtb], alf_ctb_cc_cr_idc[xCtb][yCtb], split_qt_flag, split_cu_flag, cu_skip_flag[x0][y0], pred_mode_ibc_flag[x0][y0], intra_mip_flag, inter_affine_flag[x0][y0], and merge_subblock_flag[x0][y0], ctxInc can be determined by equation (7) as follows:

[0107] ctxInc=(condL&&availableL)+(condA&&availableA)+ctxSetIdx*3 Equation (7)

[0108] For the syntax elements pred_mode_flag[x0][y0] and non_inter_flag, ctxInc can be determined using equation (8) as follows:

[0109] ctxInc=(condL&&availableL)||(condA&&availableA) Equation (8)

[0110] Table 2. Specification of ctxInc using left and top syntax elements

[0111]

[0112]

[0113] In the relevant examples, it may be inefficient to derive the CABAC context model for the current block (such as for inter_affine_flag or merge_subblock_flag) using only information from the blocks above and to the left of the current block.

[0114] In this disclosure, the context model for some syntax elements used for the current block may depend on information from two or more spatially adjacent blocks. Syntax elements may include inter-frame affine flags (e.g., inter_affine_flag) and / or sub-block merge flags (e.g., merge_subblock_flag) and / or lic_flags (e.g., flags indicating whether a lighting compensation tool is used for a particular block / region), etc. In some embodiments, syntax elements may be shown in Table 2.

[0115] For simplicity and clarity, embodiments of this disclosure may be provided based on the inter_affine_flag as an exemplary syntax element.

[0116] In this disclosure, one or more additional context increments (e.g., ctxInc) can be used to determine the syntax element of the current block. ctxInc can indicate the context index of the context model (e.g., ctxIdx). For example, ctxIdx can be set to be equal to the sum of ctxInc and ctxIdxOffset. ctxIdxOffset can be a fixed value that depends on the type of the syntax element and the type of the stripes (I, B, and P). The context increment (e.g., ctxInc) can be calculated using information such as the size of the current block, the type of the current block, and information about the blocks adjacent to the current block. Accordingly, the context model of the syntax element can be derived from the information of the M spatially adjacent blocks of the current block, where M can be a positive integer.

[0117] In this embodiment, M can be equal to 5. The five spatial neighbors of the current block can be as follows: Figure 5 As shown. Figure 5 As shown, the current block (502) may include the top-right (B0) adjacent block, the top (B1) adjacent block, the top-left (B2) adjacent block, the left (A1) adjacent block, and the bottom-left (A0) adjacent block. Each of the adjacent blocks can have various sizes. For example, Figure 5 Each adjacent block can include 4×4 luminance samples.

[0118] In an embodiment, M can be equal to the number of all sub-blocks from the bottom left (LB) position to the top left (AL) position, and from the top left (AL) position to the top right (AR) position. Figure 6 As shown in the image. Figure 6 As shown, the current block (602) may include a column of adjacent blocks to the left of the current block (602) (e.g., LB, L0, L1, L2, L3, AL) and a row of adjacent blocks above the current block (602) (AL, A3, A2, A1, A0, and AR). Each of the adjacent blocks can have various sizes. For example, Figure 6Each adjacent block can include 4×4 luminance samples.

[0119] In one embodiment, the additional ctxInc used for inter_affine_flag can be as shown in Table 3. As shown in Table 3, four values ​​of ctxInc (e.g., 0, 1, 2, and 3) can be provided to the first context-encoded binary number of inter_affine_flag (e.g., binIdx = 0).

[0120] Table 3. Assigning ctxInc to syntax elements with context-encoded binary numbers

[0121]

[0122] In another embodiment, the additional ctxInc used for merge_subblock_flag can be as shown in Table 4. As shown in Table 4, four values ​​of ctxInc (e.g., 0, 1, 2, and 3) can be provided to the first context-encoded binary number of merge_subblock_flag (e.g., binIdx = 0).

[0123] Table 4. Assigning ctxInc to syntax elements with context-encoded binary numbers

[0124]

[0125] In an embodiment, ctxInc for inter_affine_flag can be set to 3 based on a combination of the following two conditions: (1) 5 (e.g., Figure 5 (2) For each of the available spatial neighbors (A0, A1, B0, B1, and B2), N or more of them are available. In the example, N equals 4.

[0126] (MergeSubblockFlag[xNb][yNb]||InterAffineFlag[xNb][yNb])

[0127] Otherwise, if at least one of the two conditions is not met, the value of ctxInc of inter_affine_flag can be derived based on the conditions condL and condA of the left and top neighbors, respectively. condL and condA can be determined based on Table 2. ctxInc can be determined to be 0, 1, or 2 based on the following equation (9):

[0128] ctxInc = ( condL && availableL ) | | ( condA && availableA ) Equation (9)

[0129] For example, as shown in Table 2, for `inter_affine_flag`, `condL` corresponds to `MergeSubblockFlag[xNbL][yNbL]||InterAffineFlag[xNbL][yNbL]`, and `condA` corresponds to `MergeSubblockFlag[xNbA][yNbA]||InterAffineFlag[xNbA][yNbA]`. In the example, `ctxInc` can be set to 0 when `(condL&&availableL)` is satisfied and `(condA&&avaialbeA)` is not satisfied. In the example, `ctxInc` can be set to 1 when `(condL&&availableL)` is not satisfied and `(condA&&avaialbeA)` is satisfied. In the example, `ctxInc` can be set to 2 when both `(condL&&availableL)` and `(condA&&avaialbeA)` are satisfied.

[0130] In an embodiment, the ctxInc used for merge_subblock_flag can be set to 3 based on a combination of the following two conditions: (1) 5 (e.g., Figure 5 (2) For each of the available spatial neighbors (A0, A1, B0, B1, and B2), N or more of them are available. In the example, N equals 4.

[0131] (MergeSubblockFlag[xNb][yNb]||InterAffineFlag[xNb][yNb])

[0132] Otherwise, if at least one of the two conditions is not met, the value of ctxInc of merge_subblock_flag can be derived based on the conditions condL and condA of the left and top neighbors, respectively. condL and condA can be determined based on Table 2. ctxInc can be determined to be 0, 1, or 2 based on the following equation (10):

[0133] ctxInc=(condL&&availableL)||(condA&&availableA) Equation (10)

[0134] In an embodiment, the context model of the LIC flag (e.g., lic_flag) of the current block can be derived based on two or more adjacent blocks. If the value of lic_flag is true, the LIC flag can indicate that LIC should be applied to the current block. Otherwise, if the value of lic_flag is false, LIC is not applied to the current block.

[0135] As shown in Table 5, the four values ​​of ctxInc can be provided to the first context-encoded binary number of lic_flag (e.g., binIdx = 0). Table 6 shows the conditions of the left adjacent block and the upper adjacent block of the current block.

[0136] Table 5. Assigning ctxInc to syntax elements with context-encoded binary numbers

[0137]

[0138] Table 6. Specification of ctxInc using left and top syntax elements

[0139] Syntax elements condL condA ctxSetIdx lic_flag[x0][y0] LicFlag[xNbL][yNbL] LicFlag[xNbA][yNbA] 0

[0140] In the example, ctxInc for lic_flag can be set to 3 based on a combination of the following two conditions: (1) 5 (e.g., Figure 5 (1) N or more of the spatial neighbors A0, A1, B0, B1, and B2 are available. In the example, N equals 4. (2) The LIC flag of each of the available spatial neighbors is false. Otherwise, if at least one of the two conditions is not met, the ctxInc of lic_flag can be derived based on the conditions condL and condA of the left and top neighbors, respectively. condL and condA can be determined based on Table 6. ctxInc can be determined as 0, 1, or 2 based on the following equation (11):

[0141] ctxInc=(condL&&availableL)||(condA&&availableA) Equation (11)

[0142] In this embodiment, a similar context modeling approach can be applied to other syntax elements of ctxInc that are currently deduced using left and top syntax elements, such as non_inter_flag, cu_skip_flag, pred_mode_flag, and intra_mip_flag. Therefore, context models for non_inter_flag, cu_skip_flag, pred_mode_flag, and intra_mip_flag can be deduced based on two or more adjacent blocks.

[0143] Figure 7 A flowchart outlining a process (700) according to an embodiment of the present disclosure is shown. The process (700) can be used in a video decoder. In various embodiments, the process (700) is executed by processing circuitry, such as processing circuitry that performs the functions of a video decoder (110), processing circuitry that performs the functions of a video decoder (210), etc. In some embodiments, the process (700) is implemented as software instructions, so that the processing circuitry executes the process (700) when the software instructions are executed. The process begins at (S701) and proceeds to (S710).

[0144] At (S710), encoded information for the current block is received from the encoded video stream. The encoded information includes context-coded syntax elements associated with the current block. These syntax elements indicate the decoding parameters for the current block.

[0145] At (S720), a context-based adaptive binary arithmetic coding (CABAC) context model associated with the syntax elements of the context encoding is determined based on the decoding parameters of two or more spatially adjacent blocks of the current block.

[0146] At (S730), the context-encoded syntax elements are decoded based on the determined CABAC context model associated with the context-encoded syntax elements.

[0147] At (S740), the current block can be reconstructed based on the decoded syntax elements.

[0148] In some embodiments, two or more spatially adjacent blocks of the current block include the upper right adjacent block, the upper top adjacent block, the upper left adjacent block, the left side adjacent block, and the lower left adjacent block.

[0149] In some embodiments, two or more spatially adjacent blocks of the current block include a column of adjacent blocks to the left of the current block and a row of adjacent blocks to the top of the current block.

[0150] In the example, the CABAC context model is determined as one of four candidate context models assigned to the first binary number associated with the syntax element.

[0151] In the example, the CABAC context model is determined as one of the four candidate context models based on (i) the number of spatial neighboring blocks of the current block is equal to or greater than a threshold, and (ii) the merged sub-block flag associated with each spatial neighboring block in the spatial neighboring block is false, or the inter-frame affine flag associated with each spatial neighboring block in the spatial neighboring block is false, or the local illumination compensation flag associated with each spatial neighboring block in the spatial neighboring block is false.

[0152] In some embodiments, the threshold is equal to 4.

[0153] In the example, the CABAC context model is determined as the fourth context model among the four candidate context models based on (i) the number of spatial neighboring blocks of the current block is equal to 4, and (ii) the merged sub-block flag associated with each spatial neighboring block in the spatial neighboring block is false, or the inter-frame affine flag associated with each spatial neighboring block in the spatial neighboring block is false, or the local illumination compensation flag associated with each spatial neighboring block in the spatial neighboring block is false.

[0154] In response to (i) the number of spatial neighbor blocks of the current block is not equal to or greater than a threshold, or (ii) the merged sub-block flag associated with each spatial neighbor block in the spatial neighbor block is true, or the inter-frame affine flag associated with each spatial neighbor block in the spatial neighbor block is true, or the local illumination compensation flag associated with each spatial neighbor block in the spatial neighbor block is true, the CABAC context model is selected from the first candidate context model, the second candidate context model, or the third candidate context model from four candidate context models based on one of the first decoding parameters of the left neighbor block or the second decoding parameters of the upper neighbor block.

[0155] In the example, the CABAC context model is selected based on (i) the first decoding parameter of the left adjacent block in response to the availability of the left adjacent block, or (ii) the second decoding parameter of the upper adjacent block in response to the availability of the upper adjacent block.

[0156] In some embodiments, the syntax element is one of the following: inter-frame affine flag, merged sub-block flag, local illumination compensation flag, non-inter-frame flag, coding unit skip flag, prediction mode flag, or inter-frame matrix-based intra-frame prediction (MIP) flag.

[0157] Then, the process proceeds to (S799) and ends.

[0158] The process (700) can be adjusted as appropriate. One or more steps in the process (700) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0159] Figure 8A flowchart outlining a process (800) according to an embodiment of the present disclosure is shown. The process (800) can be used in a video encoder. In various embodiments, the process (800) is executed by processing circuitry, such as processing circuitry that performs the functions of a video encoder (103), processing circuitry that performs the functions of a video encoder (303), etc. In some embodiments, the process (800) is implemented as software instructions, so that the processing circuitry executes the process (800) when the software instructions are executed. The process begins at (S801) and proceeds to (S810).

[0160] At (S810), a context-based adaptive binary arithmetic coding (CABAC) context model is determined from multiple candidate context models to encode the syntax elements associated with the current block based on the encoding parameters of two or more spatially adjacent blocks of the current block.

[0161] At (S820), the syntax elements associated with the current block are context-encoded based on the defined CABAC context model. The syntax elements indicate the encoding parameters of the current block.

[0162] At (S830), encoded information for the current block is generated based on the determined CABAC context model, wherein the encoded information includes context-encoded syntax elements associated with the current block.

[0163] Then, the process proceeds to (S899) and ends.

[0164] The process (800) can be adjusted as appropriate. One or more steps in the process (800) can be modified and / or omitted. One or more additional steps can be added. Any suitable implementation order can be used.

[0165] The above-described technology can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 9 A computer system (900) is shown, which is adapted to implement certain embodiments of the disclosed subject matter.

[0166] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through decoding, microcode, etc.

[0167] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0168] Figure 9The components shown for the computer system (900) are exemplary in nature and are not intended to limit the scope or functionality of the computer software used to implement the embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any component or combination thereof shown in the exemplary embodiments of the computer system (900).

[0169] The computer system (900) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device may also be used to capture certain media, which need not be directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0170] The human-machine interface input device may include one or more of the following (only one is shown): keyboard (901), mouse (902), touchpad (903), touch screen (910), data glove (not shown), joystick (905), microphone (906), scanner (907), camera (908).

[0171] The computer system (900) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touchscreen (910), data gloves (not shown), or joystick (905), but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (909), headphones (not shown)), visual output devices (e.g., screens (910) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each with or without touchscreen input functionality, each with or without tactile feedback functionality—some of which may output two-dimensional or more three-dimensional visual outputs by means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0172] The computer system (900) may also include human-accessible storage devices and related media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) (920) or similar media (921), thumb drives (922), removable hard disk drives or solid-state drives (923), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated devices based on ROM / ASIC / PLD such as security software protectors (not shown), and so on.

[0173] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0174] The computer system (900) may also include interfaces to one or more communication networks. For example, the network may be wireless, wired, or optical. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a vehicle network and industrial network, a real-time network, a latency-tolerant network, and so on. Networks also include Ethernet, wireless LANs, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicle and industrial networks (including CANBus), and so on. Some networks typically require external network interface adapters for connection to certain general-purpose data ports or peripheral buses (949) (e.g., a USB port on the computer system (900)); other systems are typically integrated into the core of the computer system (900) via a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer system (900) can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0175] The aforementioned human-computer interface device, human-accessible storage device, and network interface can be connected to the core (940) of the computer system (900).

[0176] The core (940) may include one or more central processing units (CPU) (941), graphics processing units (GPUs) (942), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (943), task-specific hardware accelerators (944), graphics adapters (950), etc. These devices, along with read-only memory (ROM) (945), random access memory (946), internal mass storage (e.g., internal non-user-accessible hard disk drives, solid-state drives, etc.) (947), etc., may be connected via a system bus (948). In some computer systems, the system bus (948) may be accessed as one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (948) or connected via a peripheral bus (949). In this example, the screen (910) may be connected to the graphics adapter (950). Peripheral bus architectures include external controller interfaces (PCI), universal serial buses (USB), etc.

[0177] The CPU (941), GPU (942), FPGA (943), and accelerator (944) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (945) or RAM (946). Transient data can also be stored in RAM (946), while permanent data can be stored, for example, in internal mass storage (947). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (941), GPUs (942), mass storage (947), ROM (945), RAM (946), etc.

[0178] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this application, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0179] By way of example and not limitation, a computer system having an architecture (900), particularly a core (940), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the core (940) that is non-volatile, such as internal mass storage (947) or ROM (945). Software implementing various embodiments of this application can be stored in such a device and executed by the core (940). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (940), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (946) and modifying such data structures according to software-defined processes. Alternatively or as an alternative, the computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (944)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This application includes any suitable combination of hardware and software.

[0180] While this application has described several exemplary embodiments, various modifications, arrangements, and equivalent substitutions of the embodiments are all within the scope of this application. Therefore, it should be understood that those skilled in the art can design various systems and methods that, although not explicitly shown or described herein, embody the principles of this application and are thus within its spirit and scope.

Claims

1. A video decoding method, characterized in that, The method includes: Receive encoded information of the current block from the encoded video stream, the encoded information including context-coded syntax elements associated with the current block, the context-coded syntax elements indicating the decoding parameters of the current block; Based on the decoding parameters of two or more spatially adjacent blocks of the current block, an additional context increment is determined from four candidate additional context increments associated with the first context-encoded binary number of the context-encoded syntax element. The context-based adaptive binary arithmetic coding (CABAC) context model associated with the context-encoded syntax element is determined according to the determined additional context increment. Based on the CABAC context model, the context-encoded syntax elements are decoded; and Reconstruct the current block based on the decoded syntax elements; Determining an additional context increment further includes: Based on (i) the number of spatially adjacent blocks of the current block being equal to or greater than a threshold, and (ii) the merged sub-block flag associated with each of the spatially adjacent blocks being false, or the inter-frame affine flag associated with each of the spatially adjacent blocks being false, or the local illumination compensation flag associated with each of the spatially adjacent blocks being false, the additional context increment is determined as the fourth additional context increment among the four candidate additional context increments.

2. The method according to claim 1, characterized in that, The two or more spatially adjacent blocks of the current block include the upper right adjacent block, the upper top adjacent block, the upper left adjacent block, the left side adjacent block, and the lower left adjacent block.

3. The method according to claim 1, characterized in that, The two or more spatially adjacent blocks of the current block include a column of adjacent blocks to the left of the current block and a row of adjacent blocks to the top of the current block.

4. The method according to claim 1, characterized in that, The threshold is equal to 4.

5. The method according to claim 1, characterized in that, Based on (i) the number of spatially adjacent blocks of the current block being equal to 4, and (ii) the merged sub-block flag associated with each of the spatially adjacent blocks being false, or the inter-frame affine flag associated with each of the spatially adjacent blocks being false, or the local illumination compensation flag associated with each of the spatially adjacent blocks being false, the additional context increment is determined as the fourth additional context increment among the four candidate additional context increments.

6. The method according to claim 5, characterized in that, Further includes: In response to (i) the number of spatially adjacent blocks of the current block being neither equal to nor greater than the threshold, or (ii) the merged sub-block flag associated with each of the spatially adjacent blocks being true, or the inter-frame affine flag associated with each of the spatially adjacent blocks being true, or the local illumination compensation flag associated with each of the spatially adjacent blocks being true, The additional context increment is selected from the first, second, or third candidate additional context increments of the four candidate additional context increments based on one of the first decoding parameters of the left adjacent block or the second decoding parameters of the upper adjacent block.

7. The method according to claim 6, characterized in that, The selection further includes: The additional context increment is selected based on (i) the first decoding parameter of the left adjacent block in response to the availability of the left adjacent block; or (ii) the second decoding parameter of the upper adjacent block in response to the availability of the upper adjacent block.

8. The method according to claim 1, characterized in that, The syntax element is one of the following: inter-frame affine flag, merged sub-block flag, local illumination compensation flag, non-inter-frame flag, coding unit skip flag, prediction mode flag, or inter-frame matrix-based intra-frame prediction (MIP) flag.

9. A video encoding method, characterized in that, The method includes: Based on the encoding parameters of two or more spatially adjacent blocks of the current block, determine one of the four candidate additional context increments associated with the first binary number of the syntax element associated with the current block, and determine the context-based adaptive binary arithmetic coding (CABAC) context model of the current block according to the determined additional context increment. Based on the determined CABAC context model, the syntax elements associated with the current block are context-encoded, and the context-encoded syntax elements indicate the encoding parameters of the current block; Based on the determined CABAC context model, the encoded information of the current block is generated, wherein the encoded information includes the context-encoded syntax elements associated with the current block; The current block is encoded based on the context-encoded syntax elements; The additional context increment, which determines one of the four candidate additional context increments associated with the first binary number of the syntax element associated with the current block, further includes: Based on (i) the number of spatially adjacent blocks of the current block being equal to or greater than a threshold, and (ii) the merged sub-block flag associated with each of the spatially adjacent blocks being false, or the inter-frame affine flag associated with each of the spatially adjacent blocks being false, or the local illumination compensation flag associated with each of the spatially adjacent blocks being false, the additional context increment is determined as the fourth additional context increment among the four candidate additional context increments.

10. An electronic device, characterized in that, The method includes one or more processors and one or more memories, wherein the one or more memories store at least one program instruction, which is loaded and executed by the one or more processors to implement the method as described in any one of claims 1-9.

11. A non-volatile computer-readable storage medium for storing instructions, characterized in that, When executed by at least one processor, the instructions cause the at least one processor to perform the method as described in any one of claims 1-9.

12. A method for storing a bit stream, characterized in that, The video encoding method of claim 9 is used to generate a bitstream and store the bitstream.

13. A method for transmitting a bit stream, characterized in that, The video encoding method of claim 9 is used to generate a bitstream and transmit the bitstream.

14. A computer-readable storage medium storing computer instructions and a bit stream thereon, characterized in that, When the computer instructions are executed by the processor, the video encoding method of claim 9 is implemented to generate the bitstream.