Arithmetic decoding with selective adaptivity for video decoding

Abstract

Techniques are described for a hybrid coder configured to selectively use adaptive or non-adaptive coding techniques. A video coder (e.g., a video encoder or a video decoder) may code (e.g., encode or decode) first video data (e.g., syntax elements or values) based on an adaptive context model (e.g., one or more adaptive context models) to code a first block, and code second video data based on a non-adaptive context model (e.g., one or more non-adaptive context models) to code a second block.

Description

[0001] This application claims priority to U.S. application No. 16 / 912,315, filed on June 25, 2020, which claims priority to U.S. Provisional Patent Application No. 62 / 868,736, filed on June 28, 2019, each of which is incorporated by reference in its entirety. Technical Field

[0002] The present disclosure relates to video encoding (encode) and video decoding (decode). Background Art

[0003] Digital video capabilities can be incorporated into a wide variety of devices, including digital televisions, digital direct broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, e-book readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite wireless telephones, so-called "smart phones", video teleconferencing devices, video streaming devices, and the like. Digital video devices implement video coding techniques, such as those described in the standards specified by MPEG-2, MPEG-4, ITU-T H.263, ITU-T H.264 / MPEG-4 Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Video Coding (HEVC), and extensions of such standards. By implementing such video coding techniques, video devices can more efficiently transmit, receive, encode, decode, and / or store digital video information.

[0004] Video coding techniques include spatial (intra-picture) prediction and / or temporal (inter-picture) prediction to reduce or eliminate redundancy inherent in video sequences. For block-based video coding, a video slice (e.g., a video picture or a portion of a video picture) can be divided into video blocks, which may also be referred to as coding tree units (CTUs), coding units (CUs), and / or coding nodes. Video blocks in an intra-coded (I) slice of a picture are encoded using spatial prediction relative to reference samples in neighboring blocks in the same picture. Video blocks in an inter-coded (P or B) slice of a picture may use spatial prediction relative to reference samples in neighboring blocks in the same picture, or temporal prediction relative to reference samples in other reference pictures. Pictures may be referred to as frames, and reference pictures may be referred to as reference frames. Summary of the Invention

[0005] In general, this disclosure describes an example of a hybrid decoder configured to selectively use adaptive or non-adaptive coding techniques. For example, a video decoder (e.g., a video encoder or a video decoder) may decode (e.g., encode or decode) first video data (e.g., syntax elements or values) based on an adaptive context model (e.g., one or more adaptive context models) to decode a first block, and decode second video data based on a non-adaptive context model (e.g., one or more non-adaptive context models) to decode a second block. As an example, the video decoder may decode a first syntax element using context-based arithmetic coding with an adaptive context model, and decode a second syntax element using context-based arithmetic coding with a non-adaptive context model. The first syntax element and the second syntax element may be part of a bitstream.

[0006] Entropy decoding with an adaptive context model (e.g., context-based arithmetic decoding) (also referred to as adaptive arithmetic decoding) may refer to an example in which a context model (e.g., probability table) for a context is updated while processing a bitstream. Entropy decoding with a non-adaptive context model (e.g., context-based arithmetic decoding) (also referred to as non-adaptive arithmetic decoding) may refer to an example in which a context model (e.g., probability table) for a context is not updated while processing a bitstream. Context may refer to the type of data being encoded or decoded (e.g., motion vector differences, transform coefficients, etc.). In both adaptive and non-adaptive arithmetic decoding, the probability value is typically a non-50% value and is selected based on the context (e.g., the type of information being decoded).

[0007] Selectively using adaptive or non-adaptive decoding techniques can allow a relatively fast but size-constrained memory to store a subset of adapted context models, while another memory stores non-adapted context models. In this way, the video decoder can take advantage of the decoding gains of entropy decoding without having to store all context models in a fast but size-constrained memory.

[0008] In one example, the present disclosure describes a method for decoding video data, the method comprising: decoding first video data based on an adaptive context model to decode a first block; and decoding second video data based on a non-adaptive context model to decode a second block.

[0009] In one example, the present disclosure describes an apparatus for decoding video data, the apparatus comprising a plurality of memories configured to store a context model and an integrated circuit configured to: decode first video data based on an adaptive context model to decode a first block; and decode second video data based on a non-adaptive context model to decode a second block.

[0010] In one example, the present disclosure describes a computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to: decode first video data based on an adaptive context model to decode a first block; and decode second video data based on a non-adaptive context model to decode a second block.

[0011] In one example, the present disclosure describes an apparatus for decoding video data, the apparatus comprising: a unit for decoding first video data based on an adaptive context model to decode a first block; and a unit for decoding second video data based on a non-adaptive context model to decode a second block.

[0012] The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram illustrating an example video encoding and decoding system that may perform the techniques of this disclosure.

[0014] Figure 2A and Figure 2B is a conceptual diagram illustrating an example quadtree binary tree (QTBT) structure and a corresponding coding tree unit (CTU).

[0015] Figure 3 is a block diagram illustrating an example video encoder that may perform the techniques of this disclosure.

[0016] Figure 4 is a block diagram illustrating an example video decoder that may perform the techniques of this disclosure.

[0017] Figure 5A is a conceptual diagram illustrating an example of an adaptive arithmetic encoder.

[0018] Figure 5B is a conceptual diagram illustrating an example of a non-adaptive arithmetic encoder.

[0019] Figure 6A is a conceptual diagram illustrating an example of an adaptive arithmetic decoder.

[0020] Figure 6B is a conceptual diagram illustrating an example of a non-adaptive arithmetic decoder.

[0021] Figure 7 is a conceptual diagram illustrating an example of a combination of adaptive and non-adaptive arithmetic encoders.

[0022] Figure 8 is a conceptual diagram illustrating an example of a combination of adaptive and non-adaptive arithmetic decoders.

[0023] Figure 9 is a flow chart illustrating an example method of decoding video data. DETAILED DESCRIPTION

[0024] In video coding, image data (e.g., syntax elements or values) define information about a current block being decoded. For example, a syntax element may represent motion vector information for the current block, or include flags for defining coefficient values ​​in a transform block for a residual block for the current block, to name a few examples. In some examples, a video encoder binarizes image data (e.g., the value of a syntax element) into a string of binary symbols, where each binary symbol has a value of 1 or 0, and then entropy encodes the binarized values ​​(e.g., the string of binary symbols) to generate bits of a bitstream. A video decoder entropy decodes the bits of the bitstream to generate the binarized values ​​(e.g., the string of binary symbols), and debinarizes the binarized values ​​to determine the value of the syntax element. Binarization of the syntax elements is not required in all examples.

[0025] One example approach to entropy encoding or decoding syntax elements is to use context-based coding (also referred to as adaptive arithmetic coding) using an adaptive context model. In context-based coding, a memory stores a context model (e.g., a probability table) that indicates the probability of a binary symbol. For example, in context-based coding, the context model may indicate a probability value for a binary symbol value of 0 or 1. For ease of description, the examples are described with respect to binary arithmetic coding, but the example techniques are not limited thereto. The arithmetic decoding unit performs arithmetic decoding on a binary symbol-by-binary symbol basis using the probability values ​​and the actual binary symbol values.

[0026] The context is used to identify a context model stored in memory. In some examples, the context, along with the values ​​of neighboring blocks, can together identify the context model stored in memory. The context can refer to the type of information being decoded (e.g., the type of image data). For example, the context can form an index or address that identifies the context model to be used on a binary symbol-by-binary symbol basis.

[0027] When decoding using the adaptive context model, the adaptation unit uses the actual value of the binary symbol and the context model selected from the memory to adapt (e.g., update) the context model. Therefore, when decoding using the adaptive context model, the context model is read from the memory, updated, and written back to the memory, so that the next time the context model is used, the updated context model is used to determine the probability value of the binary symbol.

[0028] As the development of faster hardware leads to improved throughput through the decoding unit, more and more syntax elements are decoded using decoding with adaptive context models. As the number of syntax elements decoded using decoding with adaptive context models increases, the number of stored context models also increases.

[0029] However, there may be size limitations on the memory that stores the adaptive context model (i.e., a limit on how much data can be stored). For example, for an adaptive context model, the memory that stores the context model needs to be read from and written to. To ensure high throughput, the reads and writes need to be completed relatively quickly (i.e., the memory is a fast memory). The amount of power required for the memory may be based on how fast the memory is written to and read from and the size of the memory (i.e., how much data the memory can store). In some cases, the amount of power required to power the memory for storing all desired updateable context models may be greater than the available power, which limits the size of the memory. However, there may be compression advantages using context-based decoding compared to non-context-based decoding (e.g., bypass decoding).

[0030] This disclosure describes examples of selectively using context-based decoding using adaptive context models or using context-based decoding using non-adaptive context models. In one or more examples, rather than storing all context models in a memory that requires writing and reading so that the context models can be adapted, a subset of the context models is stored in the memory. Other context models can be stored in a memory that can be read but not written. Therefore, these other context models may not be adaptive.

[0031] Although compression may not be as high when using a non-adaptive context model compared to an adaptive context model, image data (e.g., syntax elements or values) can be sufficiently compressed using a non-adaptive context model. Thus, even with a non-adaptive context model, at least some of the benefits of context-based coding can be obtained. For example, the amount of compression using a non-adaptive context model can still be higher than the amount of compression from coding techniques that do not rely on a context model (such as bypass coding). However, because the memory storing the non-adaptive context model is read-only memory (e.g., read but not write), the memory storing the non-adaptive context model can be made sufficiently large without the power penalty associated with the memory storing the adaptive context model.

[0032] Figure 1 is a block diagram illustrating an example video encoding and decoding system 100 that can perform the techniques of the present disclosure. The techniques of the present disclosure are generally directed to decoding (encoding and / or decoding) video data. Generally, video data includes any data used to process video. Thus, video data can include original, unencoded video, encoded video, decoded (e.g., reconstructed) video, and video metadata (e.g., signaling data).

[0033] As in Figure 1 As shown in , in this example, system 100 includes a source device 102 that provides encoded video data to be decoded and displayed by a destination device 116. In particular, source device 102 provides the video data to destination device 116 via a computer-readable medium 110. Source device 102 and destination device 116 can include any of a wide variety of devices, including desktop computers, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets (such as smartphones), televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, source device 102 and destination device 116 can be equipped for wireless communication and, therefore, can be referred to as wireless communication devices.

[0034] exist Figure 1In the example of , source device 102 includes video source 104, memory 106, video encoder 200 and output interface 108. Destination device 116 includes input interface 122, video decoder 300, memory 120 and display device 118. According to the present disclosure, the video encoder 200 of source device 102 and the video decoder 300 of destination device 116 can be configured to apply technology for selectively using adaptive or non-adaptive decoding technology. Therefore, source device 102 represents an example of a video encoding device, while destination device 116 represents an example of a video decoding device. In other examples, the source device and destination device may include other components or arrangements. For example, source device 102 may receive video data from an external video source (such as an external camera). Similarly, destination device 116 may be connected to an external display device instead of including an integrated display device.

[0035] like Figure 1 The system 100 shown in FIG. 1 is merely an example. In general, any digital video encoding and / or decoding device can implement techniques for selectively using adaptive or non-adaptive decoding techniques. Source device 102 and destination device 116 are merely examples of decoding devices, wherein source device 102 generates decoded video data for transmission to destination device 116. This disclosure refers to a "decoding" device as a device that performs decoding (encoding and / or decoding) of data. Thus, video encoder 200 and video decoder 300 represent examples of decoding devices, and in particular, represent a video encoder and a video decoder, respectively. In some examples, devices 102 and 116 can operate in a substantially symmetrical manner, such that each of devices 102 and 116 includes video encoding and decoding components. Thus, system 100 can support one-way or two-way video transmission between video devices 102 and 116, for example, for video streaming, video playback, video broadcasting, or video telephony.

[0036] Typically, video source 104 represents a source of video data (i.e., original, unencoded video data) and provides a continuous sequence of pictures (also referred to as "frames") of the video data to video encoder 200, which encodes the data for the pictures. Video source 104 of source device 102 may include a video capture device (such as a camera), a video archive containing previously captured original video, and / or a video feed interface for receiving video from a video content provider. As another alternative, video source 104 may generate computer graphics-based data as source video, or a combination of real-time video, archived video, and computer-generated video. In each case, video encoder 200 encodes captured, pre-captured, or computer-generated video data. Video encoder 200 may rearrange the images from the order in which they were received (sometimes referred to as "display order") into a decoding order for decoding. Video encoder 200 may generate a bitstream comprising encoded video data. Source device 102 may then output the encoded video data onto computer-readable medium 110 via output interface 108 to be received and / or retrieved by, for example, input interface 122 of destination device 116 .

[0037] Memory 106 of source device 102 and memory 120 of destination device 116 represent general purpose memory. In some examples, memories 106, 120 can store raw video data, e.g., raw video from video source 104 and raw decoded video data from video decoder 300. Additionally or alternatively, memories 106, 120 can store software instructions executable by, for example, video encoder 200 and video decoder 300, respectively. Although shown separately from video encoder 200 and video decoder 300 in this example, it should be understood that video encoder 200 and video decoder 300 can also include internal memory for functionally similar or equivalent purposes. Furthermore, memories 106, 120 can store encoded video data, e.g., output from video encoder 200 and input to video decoder 300. In some examples, portions of memories 106, 120 can be designated as one or more video buffers, e.g., to store raw, decoded, and / or encoded video data.

[0038] The computer-readable medium 110 can represent any type of medium or device capable of transmitting encoded video data from the source device 102 to the destination device 116. In one example, the computer-readable medium 110 represents a communication medium that enables the source device 102 to send the encoded video data directly to the destination device 116 in real time, for example, via a radio frequency network or a computer-based network. According to a communication standard such as a wireless communication protocol, the output interface 108 can modulate the transmission signal including the encoded video data, and the input interface 122 can demodulate the received transmission signal. The communication medium can include any wireless or wired communication medium, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide area network, or a global network (such as the Internet). The communication medium can include routers, switches, base stations, or any other device that can be used to facilitate communication from the source device 102 to the destination device 116.

[0039] In some examples, source device 102 may output the encoded data from output interface 108 to storage device 112. Similarly, destination device 116 may access the encoded data from storage device 112 via input interface 122. Storage device 112 may include any of a variety of distributed or locally accessed data storage media, such as a hard drive, Blu-ray disc, DVD, CD-ROM, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data.

[0040] In some examples, source device 102 may output the encoded video data to a file server 114 or another intermediate storage device, which may store the encoded video generated by source device 102. Destination device 116 may access the stored video data from file server 114 via streaming or downloading. File server 114 may be any type of server device capable of storing and transmitting the encoded video data to destination device 116. File server 114 may represent a web server (e.g., for a website), a file transfer protocol (FTP) server, a content delivery network device, or a network attached storage (NAS) device. Destination device 116 may access the encoded video data from file server 114 via any standard data connection, including an internet connection. This may include a wireless channel (e.g., a Wi-Fi connection), a wired connection (e.g., a digital subscriber line (DSL), a cable modem, etc.), or a combination thereof, suitable for accessing the encoded video data stored on file server 114. File server 114 and input interface 122 may be configured to operate according to a streaming protocol, a download transfer protocol, or a combination thereof.

[0041] The output interface 108 and the input interface 122 may represent wireless transmitters / receivers, modems, wired network components (e.g., Ethernet cards), wireless communication components that operate according to any of the various IEEE 802.11 standards, or other physical components. In examples where the output interface 108 and the input interface 122 include wireless components, the output interface 108 and the input interface 122 may be configured to transmit data (e.g., encoded video data) according to a cellular communication standard (e.g., 4G, 4G-LTE (Long Term Evolution), Advanced LTE, 5G, etc.). In some examples where the output interface 108 includes a wireless transmitter, the output interface 108 and the input interface 122 may be configured to transmit data (e.g., encoded video data) according to other wireless standards (e.g., IEEE 802.11 specifications, IEEE 802.15 specifications (e.g., ZigBee 5G), etc.). TM ),Bluetooth TM Standards, etc.) to transmit data (such as encoded video data). In some examples, source device 102 and / or destination device 116 may include corresponding system-on-chip (SoC) devices. For example, source device 102 may include an SoC device to perform the functions attributed to video encoder 200 and / or output interface 108, and destination device 116 may include an SoC device to perform the functions attributed to video decoder 300 and / or input interface 122.

[0042] The techniques of this disclosure can be applied to video decoding to support any of a variety of multimedia applications, such as for over-the-air television broadcasting, cable television transmission, satellite television transmission, Internet streaming video transmission (such as Dynamic Adaptive Streaming over HTTP (DASH)), digital video encoded to a data storage medium, decoding of digital video stored on a data storage medium, or other applications.

[0043] The input interface 122 of the destination device 116 receives the encoded video bitstream from the computer-readable medium 110 (e.g., the storage device 112, the file server 114, etc.). The encoded video bitstream may include signaling information defined by the video encoder 200 and used by the video decoder 300, such as syntax elements with values ​​describing the characteristics and processing of video blocks or other decoded units (e.g., slices, pictures, groups of pictures, sequences, etc.). The display device 118 displays decoded pictures of the decoded video data to a user. The display device 118 may represent any of a variety of display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or another type of display device.

[0044] Although not in Figure 1, but in some examples, the video encoder 200 and the video decoder 300 may each be integrated with an audio encoder and / or an audio decoder and may include appropriate MUX-DEMUX units or other hardware and / or software to process a multiplexed stream including both audio and video in a common data stream. If applicable, the MUX-DEMUX unit may comply with the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0045] The video encoder 200 and the video decoder 300 can each be implemented as any of a variety of suitable encoder and / or decoder circuits, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented in software, the device can store instructions for the software in an appropriate non-transitory computer-readable medium and use one or more processors to execute the instructions in the hardware to perform the technology of the present disclosure. Each of the video encoder 200 and the video decoder 300 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC) in the corresponding device. The device including the video encoder 200 and / or the video decoder 300 can include an integrated circuit, a microprocessor, and / or a wireless communication device (such as a cellular phone).

[0046] The video encoder 200 and the video decoder 300 may operate in accordance with a video coding standard such as ITU-T H.265 (also known as High Efficiency Video Coding (HEVC)) or its extensions (such as multi-view and / or scalable video coding extensions). Alternatively, the video encoder 200 and the video decoder 300 may operate in accordance with other proprietary or industry standards, such as ITU-T H.266 (also known as Versatile Video Coding (VVC)). A draft of the VVC standard is described in Bross et al., “Versatile Video Coding (Draft 5),” Joint Video Experts Group of ITU-T SG 16WP3 (JVET) and ISO / IEC JTC 1 / SC 29 / WG 11, 14th Meeting, March 19-27, 2019, Geneva, Switzerland, JVET-N1001-v9 (hereinafter referred to as “VVC Draft 5”). A more recent draft of the VVC standard is described in Bross et al., “Versatile Video Coding (Draft 9),” Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29 / WG 11, 18th call, April 15–24, 2020, JVET-R2001-v8 (hereinafter referred to as “VVC Draft 9”). However, the techniques of this disclosure are not limited to any particular coding standard.

[0047] Typically, the video encoder 200 and video decoder 300 can perform block-based decoding of a picture. The term "block" generally refers to a structure that includes data to be processed (e.g., encoded, decoded, or otherwise used in an encoding and / or decoding process). For example, a block can include a two-dimensional matrix of samples of luma and / or chroma data. Typically, the video encoder 200 and video decoder 300 can decode video data represented in a YUV (e.g., Y, Cb, Cr) format. That is, rather than decoding red, green, and blue (RGB) data for samples of a picture, the video encoder 200 and video decoder 300 can decode luma and chroma components, where the chroma components can include red hue and blue hue chroma components. In some examples, the video encoder 200 converts received RGB formatted data into a YUV representation prior to encoding, and the video decoder 300 converts the YUV representation into an RGB format. Alternatively, pre-processing and post-processing units (not shown) can perform these conversions.

[0048] The present disclosure may generally refer to the decoding of a picture (e.g., encoding and decoding), to include the process of encoding or decoding data for the picture. Similarly, the present disclosure may refer to the decoding of a block of a picture, to include the process of encoding or decoding data for the block (e.g., prediction and / or residual decoding). A coded video bitstream typically includes a series of values ​​for syntax elements that indicate decoding decisions (e.g., decoding mode) and the partitioning of the picture into blocks. Therefore, references to decoding a picture or block should generally be understood as decoding the values ​​of the syntax elements used to form the picture or block.

[0049] HEVC defines various blocks, including coding units (CUs), prediction units (PUs), and transform units (TUs). According to HEVC, a video decoder (such as the video encoder 200) divides a coding tree unit (CTU) into CUs according to a quadtree structure. That is, the video decoder divides the CTU and CU into four equal, non-overlapping squares, and each node of the quadtree has zero or four child nodes. A node without child nodes may be referred to as a "leaf node", and the CU of such a leaf node may include one or more PUs and / or one or more TUs. The video decoder may further divide the PU and TU. For example, in HEVC, the residual quadtree (RQT) represents the division of TUs. In HEVC, PU represents inter-frame prediction data, and TU represents residual data. The intra-predicted CU includes intra-frame prediction information, such as an intra-frame mode indication.

[0050] As another example, the video encoder 200 and the video decoder 300 can be configured to operate according to VVC. According to VVC, a video decoder (such as the video encoder 200) divides a picture into multiple coding tree units (CTUs). The video encoder 200 can divide the CTU according to a tree structure (such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure). The QTBT structure removes the concept of multiple partition types, such as the separation between CU, PU and TU in HEVC. The QTBT structure includes two levels: a first level divided according to quadtree partitioning, and a second level divided according to binary tree partitioning. The root node of the QTBT structure corresponds to the CTU. The leaf nodes of the binary tree correspond to the coding units (CUs).

[0051] In the MTT partitioning structure, blocks can be partitioned using quadtree (QT) partitioning, binary tree (BT) partitioning, and one or more types of ternary tree (TT) partitioning. Ternary tree partitioning is a partitioning method that splits a block into three sub-blocks. In some examples, ternary tree partitioning divides a block into three sub-blocks without dividing the original block through the center. The partitioning types in MTT (e.g., QT, BT, and TT) can be symmetric or asymmetric.

[0052] In some examples, the video encoder 200 and the video decoder 300 may use a single QTBT or MTT structure to represent each of the luma and chroma components, while in other examples, the video encoder 200 and the video decoder 300 may use two or more QTBT or MTT structures, such as one QTBT / MTT structure for the luma component and another QTBT / MTT structure for the two chroma components (or two QTBT / MTT structures for corresponding chroma components).

[0053] The video encoder 200 and the video decoder 300 can be configured to use quadtree partitioning, QTBT partitioning, MTT partitioning, or other partitioning structures according to HEVC. For ease of explanation, the technology of the present disclosure is described with respect to QTBT partitioning. However, it should be understood that the technology of the present disclosure can also be applied to video decoders configured to use quadtree partitioning or other types of partitioning.

[0054] Blocks (e.g., CTUs or CUs) can be grouped in a picture in various ways. As an example, a brick can refer to a rectangular area of ​​a CTU row within a particular tile in a picture. A tile can be a rectangular area of ​​a CTU within a particular tile column and a particular tile row in a picture. A tile column refers to a CTU with a rectangular area equal to the height of the picture and a width specified by a syntax element (e.g., such as given in a picture parameter set). A tile row refers to a rectangular area of ​​a CTU with a height specified by a syntax element (e.g., such as given in a picture parameter set) and a width equal to the width of the picture.

[0055] In some examples, a tile can be divided into multiple bricks, where each brick can include one or more CTU rows within the tile. A tile that is not divided into multiple bricks can also be called a brick. However, a brick that is a true subset of a tile cannot be called a tile.

[0056] Tiles in a picture can also be arranged in slices. A slice can be an integer number of tiles of a picture that can be contained exclusively in a single Network Abstraction Layer (NAL) unit. In some examples, a slice includes several complete tiles, or only a sequence of consecutive complete tiles of a tile.

[0057] This disclosure may use "NxN" and "N by N" interchangeably to refer to the sample size of a block (such as a CU or other video block) in the horizontal and vertical dimensions, for example, 16x16 samples or 16 by 16 samples. Typically, a 16x16 CU will have 16 samples in the vertical direction (y=16) and 16 samples in the horizontal direction (x=16). Likewise, an NxN CU typically has N samples in the vertical direction and N samples in the horizontal direction, where N represents a non-negative integer value. The samples in a CU may be arranged in rows and columns. Furthermore, a CU does not necessarily have the same number of samples in the horizontal direction as in the vertical direction. For example, a CU may contain NxM samples, where M is not necessarily equal to N.

[0058] The video encoder 200 encodes video data representing prediction and / or residual information and other information for a CU. The prediction information indicates how the CU will be predicted to form a prediction block for the CU. The residual information typically represents the sample-by-sample difference between the samples of the CU and the prediction block before encoding.

[0059] To predict a CU, the video encoder 200 may typically form a prediction block for the CU through inter-frame prediction or intra-frame prediction. Inter-frame prediction typically refers to predicting a CU based on data of a previously decoded picture, while intra-frame prediction typically refers to predicting a CU based on previously decoded data of the same picture. To perform inter-frame prediction, the video encoder 200 may use one or more motion vectors to generate a prediction block. The video encoder 200 may typically perform a motion search to identify a reference block that closely matches the CU, for example, in terms of the difference between the CU and the reference block. The video encoder 200 may calculate a difference metric using the sum of absolute differences (SAD), the sum of squared differences (SSD), the mean absolute difference (MAD), the mean squared difference (MSD), or other such difference calculations to determine whether the reference block closely matches the current CU. In some examples, the video encoder 200 may use unidirectional prediction or bidirectional prediction to predict the current CU.

[0060] In some examples, VVC also provides an affine motion compensation mode, which can be considered an inter-frame prediction mode. In affine motion compensation mode, the video encoder 200 can determine two or more motion vectors representing non-translational motion (such as reduction or enlargement, rotation, perspective motion, or other irregular motion types).

[0061] To perform intra prediction, the video encoder 200 can select an intra prediction mode to generate a prediction block. In some examples, VVC provides sixty-seven intra prediction modes, including various directional modes as well as planar mode and DC mode. Typically, the video encoder 200 selects an intra prediction mode that describes neighboring samples of a current block (e.g., a block of a CU) based on which samples of the current block are predicted. Assuming that the video encoder 200 decodes CTUs and CUs in raster scan order (from left to right, from top to bottom), such samples may typically be above, above left, or to the left of the current block in the same picture as the current block.

[0062] The video encoder 200 encodes data indicating a prediction mode for the current block. For example, for inter-frame prediction mode, the video encoder 200 may encode data indicating which of various available inter-frame prediction modes to use and motion information for the corresponding mode. For unidirectional or bidirectional inter-frame prediction, for example, the video encoder 200 may use Advanced Motion Vector Prediction (AMVP) or Merge Mode to encode motion vectors. The video encoder 200 may use a similar mode to encode motion vectors for affine motion compensation mode.

[0063] After prediction (such as intra-frame prediction or inter-frame prediction of a block), the video encoder 200 can calculate residual data for the block. The residual data (such as a residual block) represents the sample-by-sample difference between the block and a prediction block for the block formed using the corresponding prediction mode. The video encoder 200 can apply one or more transforms to the residual block to produce transformed data in a transform domain rather than a sample domain. For example, the video encoder 200 can apply a discrete cosine transform (DCT), an integer transform, a wavelet transform, or a conceptually similar transform to the residual video data. In addition, the video encoder 200 can apply a secondary transform after the first transform, such as a pattern-dependent non-separable secondary transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), etc. After applying one or more transforms, the video encoder 200 produces transform coefficients.

[0064] As noted above, after performing any transforms to produce transform coefficients, the video encoder 200 may perform quantization of the transform coefficients. Quantization generally refers to a process in which transform coefficients are quantized to potentially reduce the amount of data used to represent the coefficients, thereby providing further compression. By performing the quantization process, the video encoder 200 may reduce the bit depth associated with some or all of the coefficients. For example, during quantization, the video encoder 200 may round down an n-bit value to an m-bit value, where n is greater than m. In some examples, to perform quantization, the video encoder 200 may perform a bitwise right shift on the value to be quantized.

[0065] After quantization, the video encoder 200 may scan the transform coefficients to produce a one-dimensional vector from a two-dimensional matrix including the quantized transform coefficients. The scan may be designed to place coefficients with higher energy (and therefore lower frequency) at the front of the vector and transform coefficients with lower energy (and therefore higher frequency) at the back of the vector. In some examples, the video encoder 200 may scan the quantized transform coefficients using a predefined scan order to produce a serialized vector and then entropy encode the quantized transform coefficients of the vector. In other examples, the video encoder 200 may perform adaptive scanning. After scanning the quantized transform coefficients to form a one-dimensional vector, the video encoder 200 may entropy encode the one-dimensional vector, for example, according to context-adaptive binary arithmetic coding (CABAC). The video encoder 200 may also entropy encode values ​​for syntax elements that describe metadata associated with the encoded video data for use by the video decoder 300 when decoding the video data.

[0066] To perform entropy coding (e.g., CABAC), the video encoder 200 may assign context within a context model to a symbol to be transmitted. Context may relate to, for example, whether neighboring values ​​of a symbol (e.g., a binary symbol) are zero values. Probability determination may be based on the context assigned to the symbol. CABAC is an example of context-based arithmetic coding, and the example techniques are not limited to CABAC.

[0067] The video encoder 200 may also generate syntax data (such as block-based syntax data, picture-based syntax data, and sequence-based syntax data) for the video decoder 300, for example, in a picture header, a block header, a slice header, or other syntax data (such as a sequence parameter set (SPS), a picture parameter set (PPS), or a video parameter set (VPS). The video decoder 300 may similarly decode such syntax data to determine how to decode the corresponding video data. The terms syntax data or syntax elements may be used to refer to any information encoded by the video encoder 200 for transmission into the bitstream.

[0068] As described above, in context-based decoding (e.g., entropy decoding, such as arithmetic decoding) using an adaptive context model, the context model is updated (e.g., adapted) based on the current value of the binary symbol and the current context. For example, as described in more detail below, for context-based decoding using an adaptive context model, an adaptation unit reads a current context model from a memory, where the current context model indicates a probability value that the binary symbol is a zero value or a non-zero value. Based on the current context model and the actual value of the binary symbol, the adaptation unit adapts (e.g., updates) the current context model.

[0069] To ensure sufficiently high throughput, reading, updating, and writing the context model should be relatively fast. Therefore, the memory storing the context model should be configured or designed for fast access. However, ensuring fast access may limit the size of the memory because the amount of power required to ensure fast access and a large storage size for the memory may not be available.

[0070] In one or more examples, the video encoder 200 may further include a memory, such as a read-only memory (ROM), that stores the context model. The context model stored in the ROM may not be adaptive because the context model is not updated. Based on the desired context, the video encoder 200 may access the memory storing the updated context or the memory storing the unupdated context and perform decoding on the binary symbol based on the accessed context model. Because the updated context model is not written to the ROM, the ROM can be made as large as desired without the power penalty of the memory storing the updated context model.

[0071] For example, context-based arithmetic coding using an adaptive context model can provide the highest level of compression. However, if all syntax elements that can be coded based on the context are coded using the adaptive context model, the power required to support the memory storing the adapted context model may be greater than the budgeted or desired power. To address this issue, some techniques use bypass coding (i.e., non-context-based coding) even for syntax elements that can be coded based on the context.

[0072] Using the techniques described in this disclosure, the video encoder 200 can utilize a non-adaptive context model to perform context-based coding of syntax elements. For example, for certain contexts (e.g., types of image data, such as syntax elements), utilizing a non-adaptive context model can provide a sufficient level of compression that is still greater than when utilizing bypass coding.

[0073] The selection of which syntax elements are context-based coded using an adaptive context model and which image data (e.g., syntax elements or values) are context-based coded using a non-adaptive context model can be a design choice. For example, if the video encoder 200 is configured to perform according to a video coding standard, the video coding standard can define which syntax elements are context-based coded using an adaptive context model and which syntax elements are context-based coded using a non-adaptive context model.

[0074] As another example, the video encoder 200 and the video decoder 300 can be part of a device (e.g., a mobile communication device, a television, a camera, etc.). These devices may include a video decoder that conforms to a particular standard. However, after these video decoders decode the video data, these devices may include a video encoder (e.g., video encoder 200) and a video decoder (e.g., video decoder 300), the video encoder re-encoding the decoded video data for storage for later retrieval, and the video decoder re-decoding the re-encoded video data for display. In one or more examples, the video encoder 200 and the video decoder 300 can be examples of a video encoder and a video decoder that re-encode and re-decode video data. In such an example, the designer of the television or the designer of the video encoder 200 and the video decoder 300 can determine which image data is context-based decoded using a non-adaptive context model and which image data is context-based decoded using an adaptive context model.

[0075] That is, even if the standard does not define which image data is context-based decoded using an adaptive context model and which image data is context-based decoded using a non-adaptive context model, the device may include a video encoder 200 and a video decoder 300, wherein the image data is selectively context-coded using an adaptive context model or context-based decoded using a non-adaptive context model. In the example where the standard defines image data that is context-based decoded using an adaptive context model and image data that is context-based decoded using a non-adaptive context model, the video encoder 200 and the video decoder 300 can be configured to encode or decode the image data based on an adaptive context model (e.g., one or more adaptive context models) or a non-adaptive context model (e.g., one or more non-adaptive context models) to decode the block.

[0076] For example, the video encoder 200 may encode the first video data based on an adaptive context model to encode the first block, and may encode the second video data based on a non-adaptive context model to decode the second block. The video decoder 300 may decode the first video data based on the adaptive context model to decode the first block, and may decode the second video data based on the non-adaptive context model to decode the second block. The first video data and the second video data may be syntax elements or values, such as coefficient values. Furthermore, the first block and the second block may be in the same picture or in different pictures. For example, the terms first block and second block are used to identify two different blocks and do not necessarily mean that the first block is decoded before the second block, or vice versa. In this way, the video encoder 200 and the video decoder 300 can select and switch between decoding based on an adaptive context model (e.g., one or more adaptive context models) and decoding based on a non-adaptive context model (e.g., one or more non-adaptive context models).

[0077] Accordingly, the video encoder 200 can determine whether to encode the image data using context-based coding with an adaptive context model or using context-based coding with a non-adaptive context model based on the type of the image data (e.g., the type or value of the syntax element). There may be other reasons for the video encoder 200 to determine whether to use context-based coding with an adaptive context model or context-based coding with a non-adaptive context model. As an example, if multiple consecutive syntax elements are decoded using context-based coding with an adaptive context model, then to increase throughput, the video encoder 200 can decode the next syntax element using context-based coding with a non-adaptive context model, even if the syntax element is normally decoded using an adaptive context model.

[0078] In this way, the video encoder 200 can generate a bitstream including the encoded video data, for example, syntax elements describing the partitioning of a picture into blocks (e.g., CUs) and prediction and / or residual information for the blocks. Ultimately, the video decoder 300 can receive the bitstream and decode the encoded video data.

[0079] In general, the video decoder 300 performs a process that is inverse to that performed by the video encoder 200 to decode the encoded video data of the bitstream. For example, the video decoder 300 may use CABAC to decode the values ​​of syntax elements for the bitstream in a manner substantially similar to, but inverse to, the CABAC encoding process of the video encoder 200. The syntax elements may define information about the partitioning of a picture into CTUs, and the partitioning of each CTU according to a corresponding partitioning structure (such as a QTBT structure) to define the CUs of the CTU. The syntax elements may also define prediction and residual information for a block (e.g., a CU) of video data.

[0080] The residual information may be represented by, for example, quantized transform coefficients. The video decoder 300 may inverse quantize and inverse transform the quantized transform coefficients of the block to reproduce a residual block for the block. The video decoder 300 uses the signaled prediction mode (intra-frame prediction or inter-frame prediction) and associated prediction information (e.g., motion information for inter-frame prediction) to form a prediction block for the block. The video decoder 300 may then combine the prediction block and the residual block (on a sample-by-sample basis) to reproduce the original block. The video decoder 300 may perform additional processing, such as a deblocking process to reduce visual artifacts along block boundaries.

[0081] The present disclosure generally relates to "signaling" certain information (such as syntax elements). The term "signaling" can generally refer to transmitting values ​​for syntax elements and / or other data used to decode encoded video data. That is, the video encoder 200 can signal values ​​for syntax elements in a bitstream. Generally, signaling refers to generating values ​​in a bitstream. As noted above, the source device 102 can transmit the bitstream to the destination device 116 in substantially real time or in non-real time (such as may occur when storing syntax elements to the storage device 112 for subsequent retrieval by the destination device 116).

[0082] As described above, the present disclosure describes example techniques for selectively using context-based adaptive coding techniques and context-based non-adaptive coding techniques (e.g., adaptive or non-adaptive entropy coding, such as CABAC). Adaptive coding (e.g., context-based coding using an adaptive context model) may refer to examples in which a context model (e.g., a probability table) is updated while processing a bitstream. Non-adaptive coding (e.g., context-based coding using a non-adaptive context model) may refer to examples in which a coding model (e.g., a probability table) is not updated during processing. In both adaptive and non-adaptive coding, the probability value (e.g., from the context model) is typically a non-50% value and is selected based on context (e.g., information from a previously encoded or decoded block of video data).

[0083] Non-adaptive decoding should not be confused with the bypass mode in context-adaptive binary arithmetic decoding, where a probability value of 50% is assumed, or the context is not used for decoding at all. In non-adaptive or adaptive context models, unlike the bypass mode, the probability value is often not 50% and is based on the context. Examples of adaptive and non-adaptive encoders and decoders are described in more detail below.

[0084] According to the techniques of the present disclosure, the video encoder 200 and the video decoder 300 can be configured to selectively determine whether to use context-based decoding with an adaptive context model or context-based decoding with a non-adaptive context model to encode or decode image data to decode a block (e.g., a syntax element or value). Based on the determination, the video encoder 200 and the video decoder 300 can encode or decode the image data using context-based decoding with an adaptive context model or using context-based decoding with a non-adaptive context model. In this way, a bitstream can include syntax elements that have been decoded using context-based decoding with an adaptive context model or using context-based decoding with a non-adaptive context model.

[0085] For example, the video decoder 300 may be configured to decode the first video data to decode the first block based on an adaptive context model, and to decode the second video data to decode the second block based on a non-adaptive context model. As an example, the video decoder 300 may decode a first syntax element received in a bitstream using context-based arithmetic decoding with an adaptive context model (e.g., using adaptive binary arithmetic decoding), and decode a second syntax element received in the bitstream using context-based arithmetic decoding with a non-adaptive context model (e.g., non-adaptive binary arithmetic coding).

[0086] The video encoder 200 may be configured to encode the first video data based on an adaptive context model to encode the first block, and to encode the second video data based on a non-adaptive context model to encode the second block. As an example, the video encoder 200 may be configured to encode a first syntax element received in a bitstream using context-based arithmetic coding (e.g., adaptive binary arithmetic coding) using an adaptive context model, and to encode a second syntax element received in a bitstream using context-based arithmetic coding (e.g., non-adaptive binary arithmetic coding) using a non-adaptive context model.

[0087] In the above examples, the first block and the second block may be in the same slice or picture, or in different slices or pictures. For example, the first block and the second block are terms used to identify two different blocks and do not necessarily mean that the first block is decoded before the second block, or vice versa. In addition, the first syntax element may be used to decode the first block (e.g., a motion vector difference or a significance flag for the first block), and the second syntax element may be used to decode the second block (e.g., a motion vector difference or a significance flag for the second block).

[0088] For example, the video encoder 200 and the video decoder 300 can each determine whether the first video data and / or the second video data are to be decoded (e.g., encoded or decoded) using adaptive coding (e.g., encoding or decoding) or non-adaptive coding. Techniques for determining whether to use adaptive or non-adaptive binary coding are described in more detail below. In the above example, the video encoder 200 and the video decoder 300 determine that the first video data will be decoded using context-based adaptive coding and that the second video data will be decoded using context-based non-adaptive coding.

[0089] As an example, to decode (e.g., encode or decode) first video data, the video decoder may be configured to decode the first video data using entropy decoding using an adaptive context model, and to decode second video data, the video decoder may be configured to decode the second video data using entropy decoding using a non-adaptive context model. For example, to decode the first video data using entropy decoding, the video decoder may be configured to decode the first video data using context-based decoding using an adaptive context model, and to decode the second video data using entropy decoding, the video decoder may be configured to decode the second video data using context-based decoding using a non-adaptive context model.

[0090] Figure 2A and Figure 2Bis a conceptual diagram illustrating an example quadtree binary tree (QTBT) structure 130 and a corresponding coding tree unit (CTU) 132. Solid lines represent quadtree splits, and dashed lines represent binary tree splits. In each split (i.e., non-leaf) node of the binary tree, a flag is signaled to indicate which split type (i.e., horizontal or vertical) is used, where in this example, 0 indicates a horizontal split and 1 indicates a vertical split. For a quadtree split, since the quadtree node divides the block horizontally and vertically into 4 sub-blocks of equal size, there is no need to indicate the split type. Thus, the video encoder 200 can encode and the video decoder 300 can decode syntax elements (such as split information) for the region tree level (i.e., solid lines) of the QTBT structure 130 and syntax elements (such as split information) for the prediction tree level (i.e., dashed lines) of the QTBT structure 130. The video encoder 200 may encode and the video decoder 300 may decode video data (such as prediction and transform data) for the CU represented by the terminal leaf node of the QTBT structure 130 .

[0091] generally, Figure 2B The CTU 132 may be associated with parameters defining the size of blocks corresponding to nodes at the first and second levels of the QTBT structure 130. These parameters may include a CTU size (indicating the size of the CTU 132 in samples), a minimum quadtree size (MinQTSize, indicating the minimum allowed quadtree leaf node size), a maximum binary tree size (MaxBTSize, indicating the maximum allowed binary tree root node size), a maximum binary tree depth (MaxBTDepth, indicating the maximum allowed binary tree depth), and a minimum binary tree size (MinBTSize, indicating the minimum allowed binary tree child node size).

[0092] The root node of the QTBT structure corresponding to a CTU can have four child nodes at the first level of the QTBT structure, where each child node can be divided according to a quadtree partition. That is, the nodes at the first level are leaf nodes (without child nodes) or have four child nodes. The example of the QTBT structure 130 represents such a node as including a parent node and child nodes with solid lines for branching. If the node at the first level is not larger than the maximum allowed binary tree root node size (MaxBTSize), the node can be further divided by the corresponding binary tree. The binary tree splitting of a node can be iterated until the node resulting from the split reaches the minimum allowed binary tree leaf node size (MinBTSize) or the maximum allowed binary tree depth (MaxBTDepth). The example of the QTBT structure 130 represents such a node as having a dotted line for branching. The binary tree leaf node is called a coding unit (CU), which is used for prediction (e.g., intra-frame prediction or inter-frame prediction) and transformation without any further partitioning. As discussed above, a CU can also be referred to as a "video block" or "block."

[0093] In one example of a QTBT partitioning structure, the CTU size is set to 128x128 (luminance sample and two corresponding 64x64 chroma samples), MinQTSize is set to 16x16, MaxBTSize is set to 64x64, MinBTSize (for both width and height) is set to 4, and MaxBTDepth is set to 4. First, quadtree partitioning is applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can have sizes from 16x16 (i.e., MinQTSize) to 128x128 (i.e., CTU size). If a leaf quadtree node is 128x128, it will not be further split by the binary tree because the size exceeds MaxBTSize (i.e., 64x64 in this example). Otherwise, the leaf quadtree node will be further split by the binary tree. Therefore, the quadtree leaf node is also the root node for the binary tree and makes the binary tree depth 0. When the binary tree depth reaches MaxBTDepth (4 in this example), no further splits are allowed. When a binary tree node has a width equal to MinBTSize (4 in this example), this means that no further horizontal splits are allowed. Similarly, a binary tree node with a height equal to MinBTSize means that no further vertical splits are allowed for that binary tree node. As noted above, the leaf nodes of the binary tree are called CUs and are further processed according to prediction and transformation without further partitioning.

[0094] Figure 3 is a block diagram illustrating an example video encoder 200 that may perform the techniques of this disclosure. Figure 3This is provided for purposes of explanation and should not be considered limiting of the techniques broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes video encoder 200 in the context of video coding standards such as the H.265 (HEVC) video coding standard and the developing H.266 (VVC) video coding standard. However, the techniques of this disclosure are not limited to these video coding standards and are generally applicable to video encoding and decoding.

[0095] exist Figure 3 In the example of , the video encoder 200 includes a video data memory 230, a mode selection unit 202, a residual generation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a filtering unit 216, a decoded picture buffer (DPB) 218, and an entropy coding unit 220. Any or all of the video data memory 230, the mode selection unit 202, the residual generation unit 204, the transform processing unit 206, the quantization unit 208, the inverse quantization unit 210, the inverse transform processing unit 212, the reconstruction unit 214, the filtering unit 216, the DPB 218, and the entropy coding unit 220 may be implemented in one or more processors or in processing circuitry (e.g., an integrated circuit). Furthermore, the video encoder 200 may include additional or alternative processors or processing circuitry (e.g., an integrated circuit) to perform these functions and other functions.

[0096] The video data memory 230 may store video data to be encoded by the components of the video encoder 200. The video encoder 200 may receive video data from, for example, the video source 104 ( Figure 1 ) receives video data stored in video data memory 230. DPB218 can act as a reference picture memory that stores reference video data for use by the video encoder 200 when predicting subsequent video data. Video data memory 230 and DPB 218 can be composed of any of a variety of memory devices, such as dynamic random access memory (DRAM) (including synchronous DRAM (SDRAM)), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. Video data memory 230 and DPB 218 can be provided by the same memory device or different memory devices. In various examples, video data memory 230 can be on-chip with other components of the video encoder 200, as shown, or off-chip relative to those components.

[0097] In this disclosure, references to the video data memory 230 should not be construed as limited to memory internal to the video encoder 200 (unless explicitly described as such) or memory external to the video encoder 200 (unless explicitly described as such). Rather, references to the video data memory 230 should be understood as reference memory that stores video data that the video encoder 200 receives for encoding (e.g., video data for a current block to be encoded). Figure 1 The memory 106 may also provide temporary storage for outputs from the various units of the video encoder 200 .

[0098] Show Figure 3 The various units of the video encoder 200 are described to help understand the operations performed by the video encoder 200. The units can be implemented as fixed-function circuits, programmable circuits, or a combination thereof. Fixed-function circuits refer to circuits that provide specific functions and are pre-set in the operations that can be performed. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functions in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuits can execute software instructions (for example, to receive parameters or output parameters), but the type of operations performed by the fixed-function circuits is generally immutable. In some examples, one or more of the units can be different circuit blocks (fixed-function or programmable), and in some examples, one or more units can be integrated circuits.

[0099] The video encoder 200 may include an arithmetic logic unit (ALU), an elementary function unit (EFU), a digital circuit, an analog circuit, and / or a programmable core formed by a programmable circuit. In an example where the operation of the video encoder 200 is performed using software executed by a programmable circuit, the memory 106 ( Figure 1 ) may store object code for software that the video encoder 200 receives and executes, or another memory (not shown) in the video encoder 200 may store such instructions.

[0100] The video data memory 230 is configured to store received video data. The video encoder 200 can retrieve pictures of video data from the video data memory 230 and provide the video data to the residual generation unit 204 and the mode selection unit 202. The video data in the video data memory 230 can be the original video data to be encoded.

[0101] The mode selection unit 202 includes a motion estimation unit 222, a motion compensation unit 224, and an intra prediction unit 226. The mode selection unit 202 may include additional functional units to perform video prediction according to other prediction modes. For example, the mode selection unit 202 may include a palette unit, an intra-block copy unit (which may be part of the motion estimation unit 222 and / or the motion compensation unit 224), an affine unit, a linear model (LM) unit, etc.

[0102] The mode selection unit 202 typically coordinates multiple coding passes to test combinations of coding parameters and the resulting rate-distortion values ​​for such combinations. Coding parameters may include: CTU to CU partitioning, prediction mode for a CU, transform type for a CU's residual data, quantization parameter for a CU's residual data, etc. The mode selection unit 202 may ultimately select a coding parameter combination that has a better rate-distortion value than other tested combinations.

[0103] The video encoder 200 may divide the picture retrieved from the video data memory 230 into a series of CTUs and encapsulate one or more CTUs into a slice. The mode selection unit 202 may divide the CTUs of the picture according to a tree structure (such as the QTBT structure or quadtree structure of HEVC described above). As described above, the video encoder 200 may form one or more CUs by dividing the CTUs according to the tree structure. Such CUs may also be generally referred to as "video blocks" or "blocks."

[0104] Typically, mode select unit 202 also controls its components (e.g., motion estimation unit 222, motion compensation unit 224, and intra prediction unit 226) to generate a prediction block for the current block (e.g., the current CU, or, in HEVC, the overlapping portion of a PU and TU). For inter prediction of the current block, motion estimation unit 222 may perform a motion search to identify one or more closely matching reference blocks in one or more reference pictures (e.g., one or more previously coded pictures stored in DPB 218). In particular, motion estimation unit 222 may calculate values ​​indicating how similar potential reference blocks are to the current block, such as based on sum of absolute differences (SAD), sum of squared differences (SSD), mean absolute difference (MAD), mean squared difference (MSD), and the like. Motion estimation unit 222 may typically perform these calculations using the sample-by-sample differences between the current block and the reference block being considered. Motion estimation unit 222 may identify the reference block with the smallest value resulting from these calculations, indicating the reference block that most closely matches the current block.

[0105] Motion estimation unit 222 may form one or more motion vectors (MVs) that define the position of a reference block in a reference picture relative to the position of the current block in the current picture. Motion estimation unit 222 may then provide the motion vectors to motion compensation unit 224. For example, for unidirectional inter prediction, motion estimation unit 222 may provide a single motion vector, while for bidirectional inter prediction, motion estimation unit 222 may provide two motion vectors. Motion compensation unit 224 may then use the motion vectors to generate a prediction block. For example, motion compensation unit 224 may use the motion vectors to retrieve data for the reference block. As another example, if the motion vectors have fractional sample precision, motion compensation unit 224 may interpolate the values ​​for the prediction block based on one or more interpolation filters. Furthermore, for bidirectional inter prediction, motion compensation unit 224 may retrieve data for the two reference blocks identified by the corresponding motion vectors, for example, by sample-by-sample averaging or weighted averaging, and combine the retrieved data.

[0106] As another example, for intra prediction or intra prediction decoding, the intra prediction unit 226 can generate a prediction block based on samples adjacent to the current block. For example, for directional mode, the intra prediction unit 226 can generally perform an arithmetic combination of the values ​​of the adjacent samples and pad these calculated values ​​across the current block in a defined direction to produce a prediction block. As another example, for DC mode, the intra prediction unit 226 can calculate the average of the adjacent samples of the current block and generate a prediction block to include the resulting average for each sample of the prediction block.

[0107] Mode selection unit 202 provides the prediction block to residual generation unit 204. Residual generation unit 204 receives the original, unencoded version of the current block from video data memory 230 and the prediction block from mode selection unit 202. Residual generation unit 204 calculates the sample-by-sample difference between the current block and the prediction block. The resulting sample-by-sample difference defines a residual block for the current block. In some examples, residual generation unit 204 may also determine the difference between sample values ​​in the residual block to generate the residual block using residual differential pulse coded modulation (RDPCM). In some examples, residual generation unit 204 may be formed using one or more subtractor circuits that perform binary subtraction.

[0108] In the example where the mode select unit 202 partitions the CU into PUs, each PU may be associated with a luma prediction unit and a corresponding chroma prediction unit. The video encoder 200 and the video decoder 300 may support PUs of various sizes. As indicated above, the size of a CU may refer to the size of the luma coding block of the CU, and the size of a PU may refer to the size of the luma prediction unit of the PU. Assuming a particular CU size of 2Nx2N, the video encoder 200 may support PU sizes of 2Nx2N or NxN for intra prediction, and support symmetric PU sizes of 2Nx2N, 2NxN, Nx2N, NxN, or similar sizes for inter prediction. The video encoder 200 and the video decoder 300 may also support asymmetric partitioning for PU sizes of 2NxnU, 2NxnD, nLx2N, and nRx2N for inter prediction.

[0109] In an example where the mode select unit does not further divide the CU into PUs, each CU may be associated with a luma coding block and a corresponding chroma coding block. As described above, the size of a CU may refer to the size of the luma coding block of the CU. The video encoder 200 and the video decoder 300 may support CU sizes of 2Nx2N, 2NxN, or Nx2N.

[0110] For other video coding techniques (such as intra-block copy mode coding, affine mode coding, and linear model (LM) mode coding, to name a few), mode selection unit 202 generates a prediction block for the current block being encoded via the corresponding unit associated with the coding technique. In some examples (such as palette mode coding), mode selection unit 202 may not generate a prediction block and instead generate syntax elements indicating how to reconstruct the block based on the selected palette. In such a mode, mode selection unit 202 may provide these syntax elements to entropy coding unit 220 for encoding.

[0111] As described above, the residual generation unit 204 receives video data for a current block and a corresponding prediction block. The residual generation unit 204 then generates a residual block for the current block. To generate the residual block, the residual generation unit 204 calculates the sample-by-sample difference between the prediction block and the current block.

[0112] The transform processing unit 206 applies one or more transforms to the residual block to generate a block of transform coefficients (referred to herein as a "transform coefficient block"). The transform processing unit 206 may apply various transforms to the residual block to form the transform coefficient block. For example, the transform processing unit 206 may apply a discrete cosine transform (DCT), a directional transform, a Karhunen-Loeve transform (KLT), or a conceptually similar transform to the residual block. In some examples, the transform processing unit 206 may perform multiple transforms (such as a primary transform and a secondary transform (e.g., a rotation transform)) on the residual block. In some examples, the transform processing unit 206 does not apply a transform to the residual block.

[0113] The quantization unit 208 may quantize the transform coefficients in the transform coefficient block to produce a quantized transform coefficient block. The quantization unit 208 may quantize the transform coefficients of the transform coefficient block according to a quantization parameter (QP) value associated with the current block. The video encoder 200 (e.g., via the mode selection unit 202) may adjust the degree of quantization applied to the transform coefficient block associated with the current block by adjusting the QP value associated with the CU. Quantization may introduce a loss of information, and therefore, the quantized transform coefficients may have a lower precision than the original transform coefficients produced by the transform processing unit 206.

[0114] The inverse quantization unit 210 and the inverse transform processing unit 212 may apply inverse quantization and inverse transform, respectively, to the quantized transform coefficient block to reconstruct a residual block based on the transform coefficient block. The reconstruction unit 214 may generate a reconstructed block corresponding to the current block (although possibly with some degree of distortion) based on the reconstructed residual block and the prediction block generated by the mode selection unit 202. For example, the reconstruction unit 214 may add samples of the reconstructed residual block to corresponding samples from the prediction block generated by the mode selection unit 202 to generate a reconstructed block.

[0115] Filtering unit 216 may perform one or more filtering operations on the reconstructed block. For example, filtering unit 216 may perform a deblocking operation to reduce blocking artifacts along the edges of the CU. In some examples, the operations of filtering unit 216 may be skipped.

[0116] The video encoder 200 stores the reconstructed blocks in the DPB 218. For example, in examples where the operation of the filtering unit 216 does not need to be performed, the reconstruction unit 214 can store the reconstructed blocks in the DPB 218. In examples where the operation of the filtering unit 216 needs to be performed, the filtering unit 216 can store the filtered and reconstructed blocks in the DPB 218. The motion estimation unit 222 and the motion compensation unit 224 can retrieve a reference picture formed by the reconstructed (and possibly filtered) blocks from the DPB 218 to perform inter-frame prediction on blocks of subsequently encoded pictures. In addition, the intra-frame prediction unit 226 can use the reconstructed blocks of the current picture in the DPB 218 to perform intra-frame prediction on other blocks in the current picture.

[0117] In general, entropy coding unit 220 may entropy encode syntax elements received from other functional components of video encoder 200. For example, entropy coding unit 220 may entropy encode quantized transform coefficient blocks from quantization unit 208. As another example, entropy coding unit 220 may entropy encode prediction syntax elements (e.g., motion information for inter-frame prediction or intra-frame mode information for intra-frame prediction) from mode selection unit 202. Entropy coding unit 220 may perform one or more entropy encoding operations on the syntax elements (which is another example of video data) to generate entropy-encoded data. For example, entropy coding unit 220 may perform a context-adaptive variable length coding (CAVLC) operation, a CABAC operation, a variable-to-variable (V2V) length coding operation, a syntax-based context-adaptive binary arithmetic coding (SBAC) operation, a probability interval partitioning entropy (PIPE) coding operation, an exponential Grunt coding operation, or another type of entropy coding operation on the data. In some examples, entropy coding unit 220 may operate in a bypass mode, in which no entropy encoding is performed on the syntax elements.

[0118] The video encoder 200 may output a bitstream that includes entropy-encoded syntax elements required for reconstructing blocks of a slice or picture. In particular, the entropy encoding unit 220 may output a bitstream.

[0119] The operations described above are described with respect to blocks. Such descriptions should be understood as operations for luma coding blocks and / or chroma coding blocks. As described above, in some examples, the luma coding blocks and chroma coding blocks are the luma and chroma components of a CU. In some examples, the luma coding blocks and chroma coding blocks are the luma and chroma components of a PU.

[0120] In some examples, operations performed for luma coding blocks do not need to be repeated for chroma coding blocks. As an example, operations for identifying a motion vector (MV) and reference picture for a luma coding block do not need to be repeated in order to identify the MV and reference picture for a chroma block. Specifically, the MV for a luma coding block can be scaled to determine the MV for a chroma block, and the reference picture can be the same. As another example, the intra prediction process can be the same for luma coding blocks and chroma coding blocks.

[0121] According to the techniques described in this disclosure, the entropy encoding unit 220 may be configured to perform context-based encoding (eg, context-based arithmetic coding) using an adaptive context model or a non-adaptive context model. Figure 7 An example of the entropy coding unit 220 is shown in more detail, and an example manner in which the entropy coding unit 220 can select between context-based coding using an adaptive context model or a non-adaptive context model is shown. If context-based decoding is not available (e.g., due to the random probability of a binary symbol being zero or one), the entropy coding unit 220 can be configured to perform bypass coding.

[0122] The video encoder 200 represents an example of a device configured to encode video data, including a memory configured to store the video data and one or more processing units implemented as an integrated circuit (e.g., entropy encoding unit 220) and configured to encode first video data based on an adaptive context model to encode a first block, and to encode second video data based on a non-adaptive context model to encode a second block. As an example, to encode the first video data, the integrated circuit may encode the first video data using entropy decoding using an adaptive context model, and to encode the second video data using entropy decoding using a non-adaptive context model. For example, to encode the first video data using entropy decoding, the integrated circuit may perform context-based arithmetic coding (e.g., CABAC) on the first video data using an adaptive context model, and to encode the second video data using entropy decoding, the integrated circuit may perform context-based arithmetic coding (e.g., CABAC) on the second video data using a non-adaptive context model.

[0123] For example, the video encoder 200 (e.g., via the entropy coding unit 220) can determine whether adaptive coding or non-adaptive coding will be used to encode the first video data, and determine that the first video data will be encoded using adaptive coding. The video encoder 200 (e.g., via the entropy coding unit 220) can determine whether adaptive coding or non-adaptive coding will be used to encode the second video data, and determine that the second video data will be encoded using non-adaptive coding.

[0124] There are various ways in which the video encoder 200 can determine whether to use adaptive or non-adaptive encoding for encoding. As a few examples, described in more detail below, the video encoder 200 can determine whether to use adaptive or non-adaptive encoding to encode the first video data based on at least one of the following: the context used to encode the first video data, the frequency of use of the context model used to encode the first video data, the stability of the context model used to encode the first video data, and a comparison of the context used to encode the first video data with a threshold. The same techniques can be used to determine whether to use adaptive or non-adaptive encoding to encode the second video data.

[0125] To encode first video data using context-based decoding using an adaptive context model, the video encoder 200 (e.g., via the entropy coding unit 220) can be configured to adapt (e.g., update) one or more context models based on previous input binary symbols to generate one or more adapted context models, and store the one or more adapted context models in a first memory (e.g., a fast memory, such as a random access memory including a register or an L1 cache). One example of adaptation is changing information indicating a probability of 0 or 1 based on the previous input binary symbol. Then, when encoding the video data, the video encoder 200 can read the one or more adapted context models from the first memory and perform context-based encoding on the first video data using the one or more adapted context models read from the first memory. In some examples, the adaptation of the context models and the storage of the one or more adapted context models can occur before decoding the first video data, so that the adapted one or more context models are already stored in the memory.

[0126] In order to encode the second video data using context-based decoding using a non-adaptive context model, the video encoder 200 (e.g., via the entropy encoding unit 220) can be configured to read one or more context models from a second memory (e.g., a ROM) and perform context-based encoding on the second video data using the one or more context models read from the second memory. For the non-adaptive context model, writing or updating may not be required, which allows the second memory to be a read-only memory (ROM). Therefore, the ROM can provide fast read access but not write access, which allows the ROM to be large enough to support many context models without power penalty (e.g., without consuming a large amount of power).

[0127] Figure 4 is a block diagram illustrating an example video decoder 300 that may perform the techniques of this disclosure. Figure 4 This is provided for purposes of explanation and is not intended to limit the techniques broadly illustrated and described in this disclosure. For purposes of explanation, this disclosure describes a video decoder 300 based on techniques of VVC and HEVC. However, the techniques of this disclosure may be performed by video coding devices configured according to other video coding standards.

[0128] exist Figure 4 In the example of FIG, the video decoder 300 includes a coded picture buffer (CPB) memory 320, an entropy decoding unit 302, a prediction processing unit 304, an inverse quantization unit 306, an inverse transform processing unit 308, a reconstruction unit 310, a filtering unit 312, and a decoded picture buffer (DPB) 314. Any or all of the CPB memory 320, the entropy decoding unit 302, the prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, the filtering unit 312, and the DPB 314 may be implemented in one or more processors or in processing circuitry (e.g., an integrated circuit). Furthermore, the video decoder 300 may include additional or alternative processors or processing circuitry (e.g., an integrated circuit) to perform these and other functions.

[0129] The prediction processing unit 304 includes a motion compensation unit 316 and an intra prediction unit 318. The prediction processing unit 304 may include additional units for performing prediction according to other prediction modes. As an example, the prediction processing unit 304 may include a palette unit, an intra-block copy unit (which may form part of the motion compensation unit 316), an affine unit, a linear model (LM) unit, and the like. In other examples, the video decoder 300 may include more, fewer, or different functional components.

[0130] The CPB memory 320 may store video data (such as an encoded video bitstream) to be decoded by components of the video decoder 300. The video data stored in the CPB memory 320 may be, for example, a video encoded bitstream received from the computer readable medium 110 ( Figure 1 ). CPB memory 320 may include a CPB that stores coded video data (e.g., syntax elements) from a coded video bitstream. Furthermore, CPB memory 320 may store video data other than syntax elements for decoded pictures, such as temporary data representing outputs from various units of video decoder 300. DPB 314 typically stores decoded pictures, which video decoder 300 may output and / or use as reference video data when decoding subsequent data or pictures in the coded video bitstream. CPB memory 320 and DPB 314 may be formed from any of a variety of memory devices, such as DRAM (including SDRAM), MRAM, RRAM, or other types of memory devices. CPB memory 320 and DPB 314 may be provided by the same memory device or different memory devices. In various examples, CPB memory 320 may be on-chip with other components of video decoder 300, or off-chip relative to those components.

[0131] Additionally or alternatively, in some examples, video decoder 300 may retrieve the video from memory 120 ( Figure 1 ) to retrieve the decoded video data from the CPB memory 320. That is, the memory 120 may store data, as discussed above with reference to the CPB memory 320. Likewise, when some or all of the functionality of the video decoder 300 is implemented in software to be executed by processing circuitry (e.g., an integrated circuit) of the video decoder 300, the memory 120 may store instructions to be executed by the video decoder 300.

[0132] Shown in Figure 4 The various units shown in FIG. 3 are provided to aid in understanding the operations performed by the video decoder 300. The units may be implemented as fixed function circuits, programmable circuits, or a combination thereof. Figure 3, fixed-function circuitry refers to circuits that provide specific functions and are pre-set in the operations that can be performed. Programmable circuitry refers to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations that can be performed. For example, a programmable circuit can execute software or firmware that causes the programmable circuit to operate in a manner defined by the instructions of the software or firmware. Fixed-function circuitry can execute software instructions (e.g., to receive parameters or output parameters), but the type of operations performed by the fixed-function circuitry is generally immutable. In some examples, one or more of the units may be different circuit blocks (fixed-function or programmable), and in some examples, one or more units may be integrated circuits.

[0133] The video decoder 300 may include an ALU, an EFU, digital circuits, analog circuits, and / or a programmable core formed by programmable circuits. In an example where the operation of the video decoder 300 is performed by software executed on the programmable circuits, on-chip or off-chip memory may store instructions (e.g., object code) of the software that the video decoder 300 receives and executes.

[0134] The entropy decoding unit 302 may receive the encoded video data from the CPB and perform entropy decoding on the video data to reproduce syntax elements. The prediction processing unit 304, the inverse quantization unit 306, the inverse transform processing unit 308, the reconstruction unit 310, and the filtering unit 312 may generate decoded video data based on the syntax elements extracted from the bitstream.

[0135] Typically, the video decoder 300 reconstructs a picture on a block-by-block basis. The video decoder 300 may perform a reconstruction operation on each block individually (wherein the block currently being reconstructed (ie, decoded) may be referred to as the "current block").

[0136] The entropy decoding unit 302 may entropy decode syntax elements that define the quantized transform coefficients of the quantized transform coefficient block and transform information, such as a quantization parameter (QP) and / or a transform mode indication. The inverse quantization unit 306 may use the QP associated with the quantized transform coefficient block to determine a degree of quantization and, similarly, determine a degree of inverse quantization for the inverse quantization unit 306 to apply. The inverse quantization unit 306 may, for example, perform a bitwise left shift operation to inverse quantize the quantized transform coefficients. The inverse quantization unit 306 may thus form a transform coefficient block comprising the transform coefficients.

[0137] After the inverse quantization unit 306 forms the transform coefficient block, the inverse transform processing unit 308 may apply one or more inverse transforms to the transform coefficient block to generate a residual block associated with the current block. For example, the inverse transform processing unit 308 may apply an inverse DCT, an inverse integer transform, an inverse Karhunen-Loeve transform (KLT), an inverse rotational transform, an inverse directional transform, or another inverse transform to the coefficient block.

[0138] In addition, prediction processing unit 304 generates a prediction block based on the prediction information syntax element entropy decoded by entropy decoding unit 302. For example, if the prediction information syntax element indicates that the current block is inter-predicted, motion compensation unit 316 may generate a prediction block. In this case, the prediction information syntax element may indicate a reference picture in DPB 314 from which the reference block is retrieved, and a motion vector that identifies the position of the reference block in the reference picture relative to the position of the current block in the current picture. Motion compensation unit 316 may generally generate a prediction block in the same manner as described with respect to motion compensation unit 224 ( Figure 3 ) to perform the inter-frame prediction process in a manner substantially similar to that described in the accompanying drawings.

[0139] As another example, if the prediction information syntax element indicates that the current block is intra-predicted, the intra-prediction unit 318 may generate a prediction block according to the intra-prediction mode indicated by the prediction information syntax element. Again, the intra-prediction unit 318 may generally generate a prediction block in the same manner as described with respect to the intra-prediction unit 226 ( Figure 3 The intra prediction process is performed in a manner substantially similar to that described in the preceding claims. The intra prediction unit 318 may retrieve data of neighboring samples of the current block from the DPB 314.

[0140] The reconstruction unit 310 may reconstruct the current block using the prediction block and the residual block. For example, the reconstruction unit 310 may add samples of the residual block to corresponding samples of the prediction block to reconstruct the current block.

[0141] The filtering unit 312 may perform one or more filtering operations on the reconstructed block. For example, the filtering unit 312 may perform a deblocking operation to reduce blocking artifacts along the edges of the reconstructed block. The operations of the filtering unit 312 may not necessarily be performed in all examples.

[0142] The video decoder 300 may store the reconstructed blocks in the DPB 314. As discussed above, the DPB 314 may provide reference information to the prediction processing unit 304, such as samples of the current picture for intra prediction and samples of previously decoded pictures for subsequent motion compensation. In addition, the video decoder 300 may output the decoded pictures from the DPB 314 for display on a display device such as a video processor. Figure 1 18).

[0143] According to the techniques described in this disclosure, entropy decoding unit 302 may be configured to perform context-based decoding using either an adaptive context model or a non-adaptive context model. Figure 8 An example of the entropy decoding unit 302 is shown in more detail, and an example manner in which the entropy decoding unit 302 can select between context-based decoding using an adaptive context model or a non-adaptive context model is shown. If context-based decoding is not available (e.g., due to the random probability of a binary symbol being zero or one), the entropy decoding unit 302 can be configured to perform bypass decoding.

[0144] The video decoder 300 represents an example of a video decoding device that includes a memory configured to store video data and one or more processing units implemented as an integrated circuit (e.g., such as an entropy decoding unit 302) and configured to decode first video data based on an adaptive context model to decode a first block, and decode second video data based on a non-adaptive context model to decode a second block. As an example, to decode the first video data, the integrated circuit may decode the first video data using entropy decoding using an adaptive context model, and to decode the second video data, the integrated circuit may decode the second video data using entropy decoding using a non-adaptive context model. For example, to decode the first video data using entropy decoding, the integrated circuit may perform context-based decoding on the first video data using an adaptive context model, and to decode the second video data using entropy decoding, the integrated circuit may perform context-based decoding on the second video data using a non-adaptive context model.

[0145] For example, the video decoder 300 (e.g., via the entropy decoding unit 302) can determine whether adaptive decoding or non-adaptive decoding will be used to decode the first video data, and determine that the first video data will be decoded using adaptive decoding. The video encoder 300 (e.g., via the entropy decoding unit 302) can determine whether adaptive decoding or non-adaptive decoding will be used to decode the second video data, and determine that the second video data will be decoded using non-adaptive decoding.

[0146] There are various ways in which the video encoder 300 can determine whether to use adaptive decoding or non-adaptive decoding for decoding. As a few non-limiting examples, described in more detail below, the video decoder 300 can determine whether to use adaptive decoding or non-adaptive decoding to decode the first video data based on at least one of the following: a context used to decode the first video data, a frequency of use of a context model used to decode the first video data, a stability of the context model used to decode the first video data, and a comparison of the context used to decode the first video data with a threshold. The same techniques can be used to determine whether to use adaptive decoding or non-adaptive decoding to decode the second video data.

[0147] To decode first video data using context-based decoding using an adaptive context model, the video decoder 300 (e.g., via the entropy coding unit 302) can be configured to adapt (e.g., update) one or more context models based on previously output binary symbols to generate one or more adapted context models, and store the one or more adapted context models in a first memory (e.g., a fast memory, such as a random access memory including a register or an L1 cache). An example of adaptation is changing information indicating a probability of 0 or 1 based on previously output binary symbols. Then, when decoding the video data, the video encoder 300 can read the one or more adapted context models from the first memory and perform context-based decoding on the first video data using the one or more adapted context models read from the first memory. In some examples, the adaptation of the context models and the storage of the one or more adapted context models can occur before decoding the first video data, such that the adapted one or more context models are already stored in the memory.

[0148] To decode the second video data using context-based decoding using a non-adaptive context model, the video decoder 300 (e.g., via the entropy decoding unit 302) can be configured to read one or more context models from a second memory (e.g., a ROM) and perform context-based decoding on the second video data using the one or more context models read from the second memory. For the non-adaptive context model, no writing or updating may be required, which allows the second memory to be a read-only memory (ROM). Therefore, the ROM can provide fast read access but not write access, which allows the ROM to be large enough to support many context models without a power penalty (e.g., without consuming a large amount of power).

[0149] The following describes additional information related to the development of syntax element coding. As described in M. Wien, High Efficiency Video Coding: Coding Tools and Specification, Springer-Verlag, Berlin, 2015, the HEVC video compression standard uses a form of adaptive arithmetic coding that requires 8 bits of memory (RAM) to store the state (e.g., context model) of each of its 202 coding contexts assigned to different syntax elements. Adaptive arithmetic coding for the upcoming VVC standard, as described in the following documents, increases the memory requirement to 28 bits per context and employs 424 different context models: A. Said, M. Karczewicz, H. Egilmez, L. Zhang, and X. Zhao, “EE2: Arithmetic coding with context-dependent double-window adaptation response,” Joint Video Experts Group (JVET) of ITU-T SG 16WP3 and ISO / IEC JTC 1 / SC 29 / WG 11, Macau, China, input document JVET-H0061, October 2017 (hereinafter referred to as “JVET-H0061”); and A. Said, J. Dong, H. Egilmez, YH. Chao, and M. Karczewicz, “CE5: Per-context CABAC initialization with double windows (CE5: Per-context CABAC initialization with dual windows),” Joint Video Experts Group (JVET) of ITU-T SG 16WP3 and ISO / IEC JTC 1 / SC 29 / WG 11, Marrakech, input document JVET-M0412, January 2019 (hereinafter referred to as “JVET-M0412”). Since arithmetic coding is computationally demanding on components of the video decoder 300, this 7x memory increase may limit throughput and make the implementation more expensive.

[0150] To reduce the computational complexity of entropy decoding in VVC, the present disclosure utilizes the fact that only a small portion of the decoding context models requires the most powerful adaptation. The present disclosure describes a hybrid entropy decoding architecture in which the computations used for adaptation can be selectively disabled (e.g., so that the entropy encoding unit 220 or the entropy decoding unit 302 can selectively utilize context-based decoding with an adaptive context model or a non-adaptive context model). There may be one or more advantages to the example techniques described in this disclosure. Several examples of advantages include: (a) the main arithmetic decoding "engine" employed in VVC does not need to be modified, (b) each non-adaptive decoding context model requires much less memory (e.g., 4 bits), and / or (c) the memory used for the non-adaptive context model can be much cheaper ROM (e.g., stored in a memory that does not require a lot of power). The present disclosure also describes a practical implementation of the example techniques to be used in the VVC standard.

[0151] As described in more detail, Figure 5A 、 5B , 6A, 6B, Figure 7 and Figure 8 shows a method for utilizing an adaptive context model (e.g., Figure 5A and 6A ) or non-adaptive context models (e.g. Figure 5B and 6B ) is an example of context-based arithmetic decoding. Figure 7 and Figure 8 An example of context-based arithmetic coding is shown in which one can choose to utilize an adaptive context model or utilize a non-adaptive context model. Figure 5A 、 5B , 6A, 6B, Figure 7 and Figure 8 Various units that perform example techniques are shown. Figure 5A 、 5B , 6A, 6B, Figure 7 and Figure 8 The various units of the integrated circuit are described to help understand the operations performed for encoding and decoding. The units can be implemented in the integrated circuit as one or more fixed-function circuits, programmable circuits, or a combination thereof. Similar to the above, fixed-function circuits refer to circuits that provide specific functions and are pre-set in the operations they can perform. Programmable circuits refer to circuits that can be programmed to perform various tasks and provide flexible functionality in the operations they can perform.

[0152] exist Figure 5A 、 5B and Figure 7In , the input is a plurality of input binary symbols. For example, the syntax element may have been binarized into a string of binary symbols (e.g., using Exponential-Golomb binarization). Binarization is not required in all examples, and when binarization is not performed, each bit in the bit string representing the value of the syntax element can be regarded as an input binary symbol. In Figure 5A 、 5B and Figure 7 In , the output is the output bits representing the coded syntax elements, which may form part of the signaled bitstream.

[0153] exist Figure 6A 、 6B and Figure 8 In , the input is a plurality of input bits (e.g., a bitstream received from the video encoder 200) representing coded syntax elements that may form part of the signaled bitstream. Figure 6A 、 6B and Figure 8 In the example of video encoder 200 using binarization, the output binary symbol is the output binary symbol of the decoded syntax element. In the example of video encoder 200 using binarization, the output binary symbol can be debinarized to generate the value of the syntax element. However, in the case where binarization is not performed, the output binary symbol can be the output bit representing the value of the syntax element.

[0154] Figure 5A is a conceptual diagram illustrating an example of an adaptive arithmetic encoder. Figure 6A is a conceptual diagram showing an example of an adaptive arithmetic decoder. Figure 5A is considered as an example of the entropy coding unit 220, and can be Figure 6A is considered as an example of the entropy decoding unit 302. For example, Figure 5A and Figure 6A Schematic diagram showing various forms of adaptive binary entropy coding used in the HEVC video compression standard and which may be part of the VVC standard.

[0155] For each encoded and decoded binary symbol (binary symbol) (e.g., of video data such as a syntax element), a probability estimate for the binary symbol value is given by Figure 5A Adaptation unit 508 or Figure 6A The adaptation unit 608 calculates and uses it for encoding and decoding. JVET-M0412 and JVET-H0061 can provide relevant information. After encoding or decoding a binary symbol, the value of the last binary symbol is used to update the state of the adaptation unit 508 or 608.

[0156] For example, the adaptation state RAM 506 receives a context address that identifies the type of syntax element being encoded (e.g., a picture data type), and the context address forms an index into the adaptation state RAM 506 to identify one or more context models. The adaptation unit 508 reads the one or more context models from the RAM 506 and adapts the one or more context models based on the actual binary symbol value, and stores the updated context models in the RAM 506. One example of adaptation is changing information indicating a probability of 0 or 1 based on the actual binary symbol value. In this manner, decoding video data using context-based coding with an adaptive context model includes adapting the one or more context models based on previously input binary symbols (e.g., using the adaptation unit 508) to generate one or more adapted context models, and storing the one or more adapted context models in the RAM 506. Then, when decoding the video data, the arithmetic encoder unit 502 reads one or more adapted context models (which may include probability values ​​(pa)) from the RAM 506 and performs context-based encoding (e.g., using CABAC) on the syntax elements using the one or more adapted context models read from the RAM 506 (e.g., using the probability values ​​(pa)).

[0157] Similarly, for Figure 6A During decoding in the video data, the adaptation state RAM 606 receives a context address that identifies the type of video data being decoded (e.g., a syntax element type), and the context address forms an index into the adaptation state RAM 606 to identify one or more context models. An adaptation unit 608 reads the one or more context models from the RAM 606, adapts the one or more context models based on the actual binary symbol value, and stores the updated context models in the RAM 606. An example of adaptation is changing information indicating a probability of 0 or 1 based on the actual binary symbol. In this manner, decoding video data using context-based decoding with an adaptive context model includes adapting the one or more context models based on previously output binary symbols (e.g., using the adaptation unit 608) to generate one or more adapted context models, and storing the one or more adapted context models in the RAM 606. Then, when decoding the video data, the arithmetic decoder unit 602 reads one or more adapted context models (which may include probability values ​​(pa)) from RAM 606, and performs context-based decoding (e.g., using CABAC) on the video data using the one or more adapted context models read from RAM 606 (e.g., using probability values ​​(pa)).

[0158] Figure 5A A state to probability mapper unit 504 is shown, and Figure 6A State to probability mapper unit 604 is shown. Unit 504 and unit 604 are optional. For example, in some cases, instead of outputting a probability value (pA), RAM 506 or RAM 606 may output a codeword and units 504 and unit 604 may map the codeword to a probability value. The use of units 504 and 604 may provide memory gain because all probability values ​​may not be needed. For example, if decimal increments are assumed, there may be 10 probability values ​​(e.g., 0.1, 0.2, 0.3, ..., 0.9, and 1). Unit 504 and unit 604 may map the codeword output from RAM 506 or RAM 606 to one of the 10 probability values. In the example utilizing units 504 and unit 604, the output from units 504 and unit 604 may be a probability value (pA).

[0159] Figure 5B is a conceptual diagram illustrating an example of a non-adaptive arithmetic encoder. Figure 6B is a conceptual diagram showing an example of a non-adaptive arithmetic decoder. Figure 5B and Figure 6B The non-adaptive form of arithmetic decoding shown in has a lower computational complexity because it does not require computation and memory read / write operations used by an adaptation unit (e.g., adaptation unit 508 or adaptation unit 608). Figure 5B and Figure 6B The form of arithmetic decoding given in may have poor compression performance.

[0160] Similar to Figure 5A and Figure 6A ,exist Figure 5B and Figure 6B , the probability value ROM 514 or ROM 614 receives a context address for identifying one or more context models in ROM 514 or ROM 614. The context model may include a probability value (pn) that the arithmetic encoder unit 510 or arithmetic decoder unit 610 may use to perform arithmetic encoding or arithmetic decoding.

[0161] Figure 5B and Figure 6B Shown are value to probability mapper unit 512 and value to probability mapper unit 612. Like units 504 and 604, units 512 and 612 may be optional. For example, ROM 514 or ROM 614 may output a codeword, and units 512 and 612 may map the codeword to a probability value (pn).

[0162] One problem with arithmetic decoding in the past was the computational cost of the required arithmetic operations (particularly multiplications), probability estimation, and data rearrangement that needed to be performed by the arithmetic encoder unit 502 or 510 or the arithmetic decoder unit 602 or 610. With new hardware technology, these issues are now less significant. However, the cost of accessing memory is an issue that affects the operation of the video encoder 200 and the video decoder 300.

[0163] Memory access can be a problem because advanced video compression requires a large number of decoding context models, and a certain amount of memory is required for each context model. Since arithmetic decoding is a component that can affect the throughput of video encoder 200 and video decoder 300, the context model information can be beneficially stored in very fast memory, which is often much more expensive than "normal" memory. For example, RAM 506 or RAM 606 requires much more power to operate than ROM 514 or ROM 614.

[0164] One way to reduce the need for a decoding context model while maintaining the same arithmetic decoding engine is to use a "bypass" mode, which assumes that binary symbol values ​​are equally probable. However, since this can degrade compression performance, if a decoding context model is used, this generally means that the compression performance of the bypass mode has been tested and may not provide adequate compression.

[0165] This disclosure describes example techniques that potentially achieve significant reductions in memory costs with only a small (e.g., minimal or negligible) reduction in compression efficiency. Adaptations to arithmetic decoding can be added using small, fast finite state machines, where the complexity is associated with arithmetic operations and other operations such as renormalization and bit rearrangement.

[0166] However, the characteristics of non-adaptive arithmetic decoding can also be important. The amount of memory used for each context by a non-adaptive encoder or decoder is much smaller. For example, the VVC arithmetic decoding engine cannot use more than 6 bits of precision for probability values ​​because this is the number used to perform multiplications. In contrast, the adaptive context model requires 28 bits per context model to update its probability estimates, but only the 6 most significant bits of the estimated probability are used for decoding.

[0167] According to examples described in this disclosure, because probability values ​​for certain context models (e.g., from the context models) are fixed, the probability values ​​associated with the context can be read from a ROM (e.g., ROM 514 or ROM 614). This may be useful for video coding applications and custom hardware decoders because, for the memory speeds required for video, ROM is much cheaper (e.g., consumes less power) than equivalent fast RAM.

[0168] To exploit these characteristics, the present disclosure describes the use of a hybrid system that combines adaptive and non-adaptive forms of arithmetic decoding in a single system, where the probability of changing the type of decoding depends on the context. In other words, the same arithmetic decoding engine employs adaptation for certain contexts and skips adaptation for other contexts. Thus, the video encoder 200 (e.g., the entropy encoding unit 220) can be configured to perform adaptive encoding (e.g., context-based arithmetic decoding using an adaptive context model) and non-adaptive encoding (e.g., context-based arithmetic decoding using a non-adaptive context model). The video decoder 300 (e.g., the entropy decoding unit 302) can be configured to perform adaptive decoding (e.g., context-based arithmetic decoding using an adaptive context model) and non-adaptive decoding (e.g., context-based arithmetic decoding using a non-adaptive context model).

[0169] Figure 7 is a conceptual diagram illustrating an example of a combination of adaptive and non-adaptive arithmetic encoders. Figure 8 is a conceptual diagram showing an example of a combination of an adaptive and non-adaptive arithmetic decoder. For example, Figure 7 and Figure 8 A schematic diagram showing an example encoding system and a corresponding decoding system is shown. Figure 7 and Figure 8 In the example, the decision to use one type or the other is inferred directly from the context address. For example, if the address is below a certain value, the adaptive version is used, and otherwise the non-adaptive version is used. However, Figure 7 and Figure 8 The example of is only one example, and other criteria may be used in addition to or in place of the context address. Any combination of criteria for determining whether adaptive arithmetic coding or non-adaptive arithmetic coding may be used. The following describes some examples of alternative or additional selection methods that may be shared by the video encoder 200 and the video decoder 300.

[0170] Figure 7An example of an entropy coding unit 220 according to one or more examples described in this disclosure is shown. As shown, the entropy coding unit 220 includes an arithmetic encoder unit 702, a state to probability mapper unit 704, an adaptation state RAM 706, an adaptation unit 708, a value to probability mapper unit 712, a probability value ROM 714, and a mode selector unit 716. The RAM 706 and the ROM 714 are examples of multiple memories of the entropy coding unit 220. The input binary symbols of the arithmetic encoder unit 702 represent binary symbols used to represent syntax elements, and the output bits from the arithmetic encoder unit 702 are encoded bits representing the syntax elements.

[0171] In one or more examples, the mode selector unit 716 can be configured to determine whether context-based coding using an adaptive context model or a non-adaptive context model will be used to encode video data (e.g., syntax elements) represented by input binary symbols to decode the block. As an example, the mode selector unit 716 receives a context address for identifying one or more context models stored in the RAM 706 or the ROM 714. The RAM 706 can be a memory that can be read and written relatively quickly (such as a register or L1 cache) and is therefore limited in size. The ROM 714 is a memory that can be read relatively quickly but does not need to be written to. Generally, the ROM 714 requires less power than the RAM 706 and can be larger in size (e.g., store more values) than the RAM 706.

[0172] If the context model is stored in RAM 706, the arithmetic encoder unit 702 can receive the context model from RAM 706 (if the mapper unit 704 is used, it may have a codeword-to-probability mapping using the mapper unit 704) and perform arithmetic coding to generate output bits. In one or more examples, the context model read from RAM 706 may have been adapted using previous binary symbols. For example, the adaptation unit 708 may have adapted one or more context models based on previous input binary symbols to generate one or more adapted context models and store the one or more adapted context models in memory (e.g., RAM 706). The mapper unit 704 (if used) or the arithmetic encoder unit 702 can then read the one or more adapted context models from memory (e.g., RAM 706) and perform context-based encoding on the video data using the one or more adapted context models read from memory (e.g., RAM 706). For example, the arithmetic encoder unit 702 can perform encoding using probability values ​​(pa). Subsequently, the adaptation unit 708 may further update the adapted context model based on the current value of the input binary symbol, and store the result in the RAM 706 so that the further updated context model is used next time the context model is used.

[0173] If the context model is stored in ROM 714, the arithmetic encoder unit 702 can receive the context model from ROM 714 (if the mapper unit 712 is used, it may have a codeword-to-probability mapping using the mapper unit 712) and perform arithmetic coding to generate output bits. For example, the mapper unit 712 (if used) or the arithmetic encoder unit 702 can read one or more context models from a memory (e.g., ROM 714) and perform context-based encoding (e.g., based on a non-adaptive context model) on the video data using the one or more context models read from the memory (e.g., ROM 714). For example, the arithmetic encoder unit 702 can perform encoding using the probability value (pn). Since the context model stored in ROM 714 is not updated (e.g., adapted), the context model stored in ROM 714 does not change.

[0174] Figure 8An example of an entropy decoding unit 302 according to one or more examples described in this disclosure is shown. As shown, the entropy decoding unit 302 includes an arithmetic decoder unit 802, a state to probability mapper unit 804, an adaptation state RAM 806, an adaptation unit 808, a value to probability mapper unit 812, a probability value ROM 814, and a mode selector unit 816. RAM 806 and ROM 814 are examples of multiple memories of the entropy decoding unit 302. The input bits of the arithmetic decoder unit 802 represent bits used to represent the encoded video data, and the output binary symbols from the arithmetic decoder unit 802 are decoded binary symbols representing the video data.

[0175] In one or more examples, the mode selector unit 816 can be configured to determine whether context-based decoding using an adaptive context model or a non-adaptive context model will be used to encode the video data represented by the input bits. As an example, the mode selector unit 816 receives a context address for identifying one or more context models stored in the RAM 806 or the ROM 814. The RAM 806 can be a memory that can be read and written relatively quickly (such as a register or L1 cache) and is therefore limited in size. The ROM 814 is a memory that can be read relatively quickly but does not need to have data written to it. Generally, the ROM 814 requires less power than the RAM 806 and can have a larger size than the RAM 806 (e.g., store more values).

[0176] If the context model is stored in RAM 806, the arithmetic decoder unit 802 can receive the context model from RAM 806 (if the mapper unit 804 is used, it may have a codeword-to-probability mapping using the mapper unit 804) and perform arithmetic decoding to generate an output binary symbol. In one or more examples, the context model read from RAM 806 may have been adapted based on a previous binary symbol. For example, the adaptation unit 808 may have adapted one or more context models based on a previous output binary symbol to generate one or more adapted context models and store the one or more adapted context models in a memory (e.g., RAM 806). The mapper unit 804 (if used) or the arithmetic decoder unit 802 can then read the one or more adapted context models from the memory (e.g., RAM 806) and perform context-based decoding on the video data using the one or more adapted context models read from the memory (e.g., RAM 806). For example, the arithmetic decoder unit 802 can perform decoding using probability values ​​(pa). Subsequently, the adaptation unit 808 may further update the adapted context model based on the current value of the output binary symbol, and store the result in the RAM 806 so that the further updated context model is used next time the context model is used.

[0177] If the context model is stored in ROM 814, the arithmetic decoder unit 802 can receive the context model from ROM 814 (if the mapper unit 812 is used, it may have a codeword to probability mapping using the mapper unit 812) and perform arithmetic decoding to generate output binary symbols. For example, the mapper unit 812 (if used) or the arithmetic decoder unit 802 can read one or more context models from a memory (e.g., ROM 814) and perform context-based decoding (e.g., based on a non-adaptive context model) on the video data using the one or more context models read from the memory (e.g., ROM 814). For example, the arithmetic decoder unit 802 can perform decoding using the probability value (pn). Since the context model stored in ROM 814 is not updated (e.g., adapted), the context model stored in ROM 814 does not change.

[0178] In the above example, the context address is used to identify whether context-based decoding using an adaptive context model or a non-adaptive context model will be used to encode or decode the video data. However, the example technology is not limited thereto. Encoding or decoding using a non-adaptive context model is often faster than using an adaptive context model because the adaptation unit 708 or the adaptation unit 808 uses clock cycles to update the context model, which is not required in the case of a non-adaptive context model.

[0179] In some examples, mode selector unit 716 and mode selector unit 816 can track how much video data has been encoded or decoded using the adaptive context model. If the amount of video data that has been encoded or decoded using the adaptive context model becomes too high, mode selector unit 716 and mode selector unit 816 can select a non-adaptive context model for encoding or decoding until the ratio of the amount of video data encoded or decoded using the adaptive context model to the amount of video data encoded or decoded using the non-adaptive context model reaches a threshold range (e.g., falls to or below a threshold value).

[0180] As another example, the mode selector unit 716 and the mode selector unit 816 may determine that the number of times context-based decoding using an adaptive context model is used is greater than or equal to a threshold value. In response, the mode selector unit 716 and the mode selector unit 816 may select decoding using a non-adaptive context model. For example, the arithmetic encoder unit 702 or the arithmetic decoder unit 802 may encode or decode the video data using context-based decoding using a non-adaptive context model based on determining that the number of times context-based decoding using an adaptive context model is used is greater than or equal to a threshold value.

[0181] There may be various reasons why some context models are adaptive and some are non-adaptive. For example, a context model that may be selected as non-adaptive may be a context model that is used less frequently (because they do not affect compression much), or a context model with relatively stable binary symbol value probabilities. An example of a relatively stable context model is a context model that tends not to change much during decoding of a slice or picture (e.g., as determined based on training). For relatively stable context models, an initial context model may be selected that is not subsequently changed. There may also be methods for making the best selection based on training, and various training techniques may be used.

[0182] Thus, there can be various methods for determining whether to perform context-based decoding of video data using an adaptive context model or a non-adaptive context model, such as based on one or more of: the context of the video data (e.g., the type of data being decoded), the frequency of use of the context model for the video data, the stability of the context model used to decode the video data, and a comparison of the context to a threshold value.

[0183] Some examples of implementing example techniques using video encoder 200 or video decoder 300 are described below. The example techniques for implementation are provided by way of example only and should not be considered limiting.

[0184] The actual implementation of mode selector unit 716 and mode selector unit 816 can be efficiently accomplished in several ways. Since the numbers assigned to the context models are arbitrary, but by defining the order in which they decode the video data, it may be possible to sort the context models so that those that are non-adaptive come first. In this way, the context model can be identified by comparing the context number with a fixed value (e.g., a threshold value). The value (e.g., fixed value or threshold value) can be fixed by the video compression standard or encoded with the video (e.g., the video encoder 200 signals the threshold value to the video decoder 300).

[0185] If the context models are classified in a different way, a bit array (424 bits in the case of VVC) can identify the non-adaptive context model. These bits can be fixed by the video compression standard or encoded with the video (for example, the video encoder 200 signals the bits used to identify the non-adaptive context to the video decoder 300). In addition, instead of identifying the non-adaptive context model, it may be possible to use bits to identify the adaptive context model.

[0186] The video coding standard may define an initial probability value for each adaptive context model, which is based on the quantization parameter (QP) and the slice type (I, B, or P). The video coding standard may also define values ​​for non-adaptive context models. In some examples, the video encoder 200 may signal the probability values ​​for the context models, including the adaptive and non-adaptive context models, to the video decoder 300.

[0187] Using different values ​​for each slice type can be a way to define different non-adaptive context models, as it corresponds to three different tables. The dependency on QP may need to be modified. For adaptive contexts, there is a formula for continuously mapping the normalized initialization parameter to a probability value, and there may be many probability values ​​for different QP values.

[0188] This can cause problems when the probabilities for a non-adaptive context model are stored in ROM 714 and ROM 814, as a large amount of memory may be required for all the values. One example technique for addressing the large amount of required memory is to divide the range of allowed QP values ​​into several intervals and create a different ROM 714 and ROM 814 for each of these intervals, as in the following example.

[0189] condition standard hardware 0≤QP<16 Initialize Table 1 ROM-0 16≤QP<32 Initialize Table 2 ROM-1 32≤QP<48 Initialization Table 3 ROM-2 48≤QP<64 Initialization Table 4 ROM-3

[0190] In other words, information about how the current block of video data is coded may be a factor in determining which context model to use. Examples of information about how the current block of video data is coded include at least one of a quantization parameter (QP) for the current block and a prediction mode for the current block (e.g., whether the prediction mode is inter prediction or intra prediction).

[0191] In some examples, ROM 714 and ROM 814 may be multiple ROMs (e.g., ROMs 714-1 to 714-N and ROMs 814-1 to 814-N). The multiple ROMs may be partitions of the same large ROM or separate ROMs. Each of the multiple ROMs may be associated with specific information regarding how it decodes a current block of video data. For example, ROM 714-1 and ROM 814-1 may store a context model for a QP less than 16, ROM 714-2 and ROM 814-2 may store a context model for a QP less than 32 and greater than or equal to 16, ROM 714-3 and ROM 814-3 may store a context model for a QP less than 48 and greater than or equal to 32, and ROM 714-4 and ROM 814-4 may store a context model for a QP less than 64 and greater than or equal to 48.

[0192] Thus, in one or more examples, the mode selector unit 716 or the mode selector unit 816 may determine information regarding the manner in which the current block of video data is to be decoded. The mode selector unit 716 or the mode selector unit 816 may select a location in memory to read from (e.g., which of the ROMs 714-1 to 714-N or ROMs 814-1 to 814-N to read from) based on the determined information. The arithmetic encoder unit 702 or the arithmetic decoder unit 802 may read one or more context models from the selected location (or through the mapper unit 712 and the mapper unit 812, if used) and perform context-based binary decoding of the video data using the one or more context models read from the memory (e.g., from one of the ROMs 714-1 to 714-N or ROMs 814-1 to 814-N).

[0193] Another aspect is the mapping of context bit values ​​to probabilities. This mapping may not have to be linear, and a table can be used to avoid expensive calculations. The following table shows an example where two bits are used for each context and these bits are mapped to probabilities.

[0194]

[0195]

[0196] For example, this table is an example of the operations performed by the value-to-probability mapper unit 712 or the value-to-probability mapper unit 812. If the mapper unit 712 or the mapper unit 812 receives bit 00, the mapper unit 712 or the mapper unit 812 outputs a probability value of 0.02. If the mapper unit 712 or the mapper unit 812 receives bit 01, the mapper unit 712 or the mapper unit 812 outputs a probability value of 0.1. If the mapper unit 712 or the mapper unit 812 receives bit 10, the mapper unit 712 or the mapper unit 812 outputs a probability value of 0.9. If the mapper unit 712 or the mapper unit 812 receives bit 11, the mapper unit 712 or the mapper unit 812 outputs a probability value of 0.98. Mapper units 704 and 804 can operate in a similar manner.

[0197] Figure 9 is a flow chart illustrating an example method for decoding video data. The example technique is described with respect to an integrated circuit for a video decoder. The integrated circuit can be a fixed-function circuit, a programmable circuit, or a combination thereof. Examples of video decoders include video encoder 200 or video decoder 300.

[0198] There may be one or more memories configured to store the context model. For example, there may be RAM 706 and ROM 714 of video encoder 200 for storing the context model. There may be RAM 806 and ROM 814 of video decoder 300 for storing the context model. As described above, in some examples, ROM 714 may be multiple ROMs and ROM 814 may be multiple ROMs, where each ROM is associated with information indicating the manner in which the current block of video data is decoded. Examples of information indicating the manner in which the current block of video data is decoded include at least one of the following: a quantization parameter for the current block, and a prediction mode for the current block.

[0199] The integrated circuit of the video decoder may be configured to decode first video data based on an adaptive context model to decode a first block (900). An example of decoding the first video data based on the adaptive context model includes encoding the first video data based on the adaptive context model. An example of decoding the first video data based on the adaptive context model includes decoding the first video data based on the adaptive context model.

[0200] An integrated circuit of a video decoder may determine whether to decode first video data using context-based coding with an adaptive context model or using context-based coding with a non-adaptive context model based on one or more of the following: a context of the first video data, a frequency of use of a context model for the first video data, a stability of the context model used to decode the first video data, and a comparison of the context to a threshold. In this example, based on one or more of the above factors, the integrated circuit determines to decode the first video data using context-based coding with an adaptive context model.

[0201] To decode first video data in a bitstream using context-based decoding using an adaptive context model, an integrated circuit of a video decoder may be configured to: adapt one or more context models based on at least one of a previous input binary symbol or a previous output binary symbol to generate one or more adapted context models, store the one or more adapted context models in a memory, read the one or more adapted context models from the memory, and perform context-based decoding on the first video data using the one or more adapted context models read from the memory. The adapting of the context models and the storing of the one or more adapted context models may have occurred prior to decoding the first video data, such that the adapted one or more context models are already stored in the memory. The memory may be a random access memory, such as RAM 706 or RAM 806.

[0202] The integrated circuit of the video decoder may be configured to decode second video data based on one or more non-adaptive context models to decode a second block (902). An example of decoding the second video data in a bitstream using context-based decoding based on the one or more non-adaptive context models includes encoding the second video data based on the one or more non-adaptive context models. An example of decoding the second video data based on the one or more non-adaptive context models includes decoding the second video data based on the one or more non-adaptive context models.

[0203] To decode the second video data using context-based decoding using a non-adaptive context model, the integrated circuit may be configured to read one or more context models from a memory, and perform context-based decoding on the second video data using the one or more context models read from the memory. The memory may be a read-only memory (ROM), such as ROM 714 or ROM 814.

[0204] In some examples, to decode the second video data using a non-adaptive context model, the integrated circuit can be configured to: determine information about how it decoded a current block of video data, select a location to read from a memory based on the determined information, read one or more context models from the selected location, and perform context-based binary decoding on the second video data using the one or more context models read from the memory. For example, as described above, in some examples, ROM 714 and ROM 814 can be multiple ROMs, each of which is associated with information about how it decoded the current block (e.g., a quantization parameter or prediction mode for the current block). The integrated circuit can identify which ROM to read from (e.g., select a location to read from in the memory) based on the information about how it decoded the current block, and read the one or more context models from the selected location (e.g., the selected ROM).

[0205] As another example, the integrated circuit may determine that the number of times context-based decoding using the adaptive context model is used is greater than or equal to a threshold value. In such an example, to decode second video data in the bitstream using context-based decoding using the non-adaptive context model, the integrated circuit may be configured to: decode the second video data using context-based decoding using the non-adaptive context model based on determining that the number of times context-based decoding using the adaptive context model is used is greater than or equal to the threshold value.

[0206] One or more example techniques of the present disclosure are described below. The example techniques can be used individually or in combination.

[0207] Example 1. A method of decoding video data, the method comprising: decoding a first syntax element in a bitstream using adaptive binary arithmetic decoding; and decoding a second syntax element in the bitstream using non-adaptive binary arithmetic decoding.

[0208] Example 2. The method of Example 1, further comprising determining whether the first syntax element is to be decoded using adaptive binary arithmetic decoding or non-adaptive binary arithmetic decoding, and determining that the first syntax element is to be decoded using adaptive binary arithmetic decoding.

[0209] Example 3. The method of any of Examples 1 and 2, further comprising determining whether the second syntax element is to be decoded using adaptive binary arithmetic decoding or non-adaptive binary arithmetic decoding, and determining that the second syntax element is to be decoded using non-adaptive binary arithmetic decoding.

[0210] Example 4. The method of any of Examples 2 and 3, wherein determining whether adaptive binary arithmetic decoding or non-adaptive binary arithmetic decoding is to be used to decode the first syntax element comprises determining whether adaptive binary arithmetic decoding or non-adaptive binary arithmetic decoding is to be used to decode the first syntax element based on at least one of: a context address of a context for decoding the first syntax element, a usage frequency of the context for decoding the first syntax element, a stability of the context for decoding the first syntax element, and a comparison of the context for decoding the first syntax element with a threshold value.

[0211] Example 5. The method of any of Examples 2-4, wherein determining whether adaptive binary arithmetic decoding or non-adaptive binary arithmetic decoding is to be used to decode the second syntax element comprises determining whether adaptive binary arithmetic decoding or non-adaptive binary arithmetic decoding is to be used to decode the second syntax element based on at least one of: a context address of a context for decoding the second syntax element, a frequency of use of the context for decoding the second syntax element, a stability of the context for decoding the second syntax element, and a comparison of the context for decoding the second syntax element to a threshold value.

[0212] Example 6. A method of encoding video data, the method comprising: encoding a first syntax element in a bitstream using adaptive binary arithmetic coding; and encoding a second syntax element in the bitstream using non-adaptive binary arithmetic coding.

[0213] Example 7. The method of Example 6, further comprising determining whether the first syntax element is to be encoded using adaptive binary arithmetic coding or non-adaptive binary arithmetic coding, and determining whether the first syntax element is to be encoded using adaptive binary arithmetic coding.

[0214] Example 8. The method of any of Examples 6 and 7, further comprising determining whether the second syntax element is to be encoded using adaptive binary arithmetic coding or non-adaptive binary arithmetic coding, and determining that the second syntax element is to be encoded using non-adaptive binary arithmetic coding.

[0215] Example 9. The method of any of Examples 7 and 8, wherein determining whether adaptive binary arithmetic coding or non-adaptive binary arithmetic coding is to be used to encode the first syntax element comprises determining whether adaptive binary arithmetic coding or non-adaptive binary arithmetic coding is to be used to encode the first syntax element based on at least one of: a context address of a context used to encode the first syntax element, a frequency of use of the context used to encode the first syntax element, a stability of the context used to encode the first syntax element, and a comparison of the context used to encode the first syntax element with a threshold value.

[0216] Example 10. The method of any of Examples 7-9, wherein determining whether to use adaptive binary arithmetic coding or non-adaptive binary arithmetic coding to encode the second syntax element comprises determining whether to use adaptive binary arithmetic coding or non-adaptive binary arithmetic coding to encode the second syntax element based on at least one of: a context address of a context used to encode the second syntax element, a frequency of use of the context used to encode the second syntax element, a stability of the context used to encode the second syntax element, and a comparison of the context used to encode the second syntax element to a threshold value.

[0217] Example 11. A device for decoding video data, the device comprising a memory configured for the video data and a video decoder comprising at least one of fixed-function circuitry or programmable circuitry, wherein the video decoder is configured to perform the method of any of Examples 1-5.

[0218] Example 12. An apparatus for encoding video data, the apparatus comprising a memory configured for the video data and a video encoder comprising at least one of fixed-function circuitry or programmable circuitry, wherein the video encoder is configured to perform the method of any of Examples 6-10.

[0219] Example 13. The device of any of Examples 11 and 12, wherein the device further comprises one or more of: a device for displaying the decoded video data, a camera, a computer, a mobile device, a broadcast receiver device, or a set-top box.

[0220] Example 14. An apparatus for decoding video data, the apparatus comprising means for performing the method of any of Examples 1-5.

[0221] Example 15. An apparatus for encoding video data, the apparatus comprising means for performing the method of any of Examples 6-10.

[0222] Example 16. A computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform the method of any of Examples 1-5 or 6-10.

[0223] It is to be appreciated that, depending on the examples, certain actions or events of any of the techniques described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., not all described actions or events are required for the practice of the techniques). Furthermore, in some examples, actions or events may be performed concurrently rather than sequentially, such as through multithreading, interrupt handling, or multiple processors.

[0224] In one or more examples, the functions described may be implemented using hardware, software, firmware, or any combination thereof. If implemented using software, the functions may be stored on or transmitted through a computer-readable medium as one or more instructions or codes and executed by a hardware-based processing unit. Computer-readable media may include computer-readable storage media (which corresponds to tangible media such as data storage media) or communication media (including, for example, any media that facilitates the transfer of a computer program from one place to another according to a communication protocol). In this manner, computer-readable media may generally correspond to: (1) a non-transitory tangible computer-readable storage medium; or (2) a communication medium, such as a signal or carrier wave. Data storage media may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include computer-readable media.

[0225] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage devices, flash memory or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. In addition, any connection is appropriately referred to as a computer-readable medium. For example, if an instruction is transmitted from a website, server or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technologies such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waveforms, signals or other temporary media, but are instead directed to non-temporary tangible storage media. As used herein, disks and optical disks include compact discs CDs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks and blue-ray discs, wherein disks usually copy data magnetically, while optical discs use lasers to optically copy data. Combinations of the above should also be included within the scope of computer-readable media.

[0226] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the terms "integrated circuit," "processor," and "processing circuitry" as used herein may refer to any of the aforementioned structures or any other structures suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or incorporated into a combined codec. Furthermore, the techniques may be fully implemented in one or more circuits or logic elements.

[0227] The techniques of the present disclosure can be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but do not necessarily require implementation by different hardware units. Specifically, as described above, the various units can be combined in a codec hardware unit, or provided by a collection of interoperable hardware units (including one or more processors as described above) in combination with appropriate software and / or firmware.

[0228] Various examples have been described. These and other examples are within the scope of the following claims.

Claims

1. A method for decoding video data, the method comprising: performing entropy decoding on the first video data block based on one or more adaptive context models, wherein the one or more adaptive context models are read from a random access memory (RAM); determining that the number of times context-based arithmetic decoding using an adaptive context model is used is greater than or equal to a threshold value; determining a quantization parameter QP associated with a second block of video data based on determining that the number of times is greater than or equal to the threshold; selecting a location in a read-only memory (ROM) from which to read based on the QP associated with the second block of video data, wherein different locations of the ROM are associated with different QP value intervals; and The second block of video data is entropy-coded based on one or more non-adaptive context models, wherein the one or more non-adaptive context models are read from the location of the ROM.

2. The method according to claim 1, wherein Entropy decoding the first block of video data includes decoding the first block of video data using context-based arithmetic decoding that utilizes the one or more adaptive context models, and wherein entropy decoding the second block of video data includes decoding the second block of video data using context-based arithmetic decoding that utilizes the one or more non-adaptive context models.

3. The method according to claim 1, wherein The first video data block and the second video data block are in the same picture or slice.

4. The method according to claim 1, wherein The first video data block and the second video data block are in different pictures or slices.

5. The method according to claim 1, wherein Entropy decoding the first video data block based on the one or more adaptive context models includes: adapting the one or more context models based on at least one of a previous input binary symbol or a previous output binary symbol to generate one or more adapted context models; storing the one or more adapted context models in the RAM; reading the one or more adapted context models from the RAM; and Performing context-based arithmetic decoding on the first video data block by utilizing the one or more adapted context models read from the RAM.

6. The method according to claim 1, wherein Entropy decoding the second video data block based on the one or more non-adaptive context models includes: reading the one or more non-adaptive context models from the ROM; and The second video data block is subjected to context-based arithmetic decoding using the one or more non-adaptive context models read from the ROM.

7. The method according to claim 1, further comprising: determining whether to entropy code the first block of video data based on the one or more adaptive context models based on one or more of the following: a context of the first video data block; a usage frequency of the context model of the first video data block; stability of the context model used for coding the first block of video data; as well as The context is compared to a threshold value.

8. The method according to claim 1, in, Entropy decoding the first video data block includes: entropy decoding the first video data block based on the one or more adaptive context models, and The entropy decoding of the second video data block includes: entropy decoding of the second video data block based on the one or more non-adaptive context models.

9. The method according to claim 1, in, Entropy decoding the first video data block includes: entropy encoding the first video data block based on the one or more adaptive context models, and The entropy decoding of the second video data block includes: entropy encoding the second video data block based on the one or more non-adaptive context models.

10. A device for decoding video data, the device comprising: a plurality of memories configured to store the context model; as well as An integrated circuit configured as follows: performing entropy decoding on the first video data block based on one or more adaptive context models, wherein the one or more adaptive context models are read from a random access memory (RAM); determining that the number of times context-based arithmetic decoding using an adaptive context model is used is greater than or equal to a threshold value; determining a quantization parameter QP associated with a second block of video data based on determining that the number of times is greater than or equal to the threshold; selecting a location in a read-only memory (ROM) from which to read based on the QP associated with the second block of video data, wherein different locations of the ROM are associated with different QP value intervals; and The second block of video data is entropy-coded based on one or more non-adaptive context models, wherein the one or more non-adaptive context models are read from the location of the ROM.

11. The apparatus according to claim 10, wherein To entropy decode the first block of video data, the integrated circuit is configured to decode the first block of video data using context-based arithmetic coding utilizing the one or more adaptive context models, and wherein, to entropy decode the second block of video data, the integrated circuit is configured to decode the second block of video data using context-based arithmetic coding utilizing the one or more non-adaptive context models.

12. The apparatus according to claim 10, wherein The first video data block and the second video data block are in the same picture or slice.

13. The apparatus according to claim 10, wherein The first video data block and the second video data block are in different pictures or slices.

14. The apparatus according to claim 10, wherein To entropy code the first block of video data based on the one or more adaptive context models, the integrated circuit is configured to: adapting the one or more context models based on at least one of a previous input binary symbol or a previous output binary symbol to generate one or more adapted context models; storing the one or more adapted context models in the RAM; reading the one or more adapted context models from the RAM; as well as Performing context-based arithmetic decoding on the first video data block by utilizing the one or more adapted context models read from the RAM.

15. The apparatus according to claim 10, wherein To entropy decode the second video data block based on the one or more non-adaptive context models, the integrated circuit is configured to: reading one or more non-adaptive context models from the ROM; as well as The second video data block is subjected to context-based arithmetic decoding using the one or more non-adaptive context models read from the ROM.

16. The apparatus according to claim 10, wherein The integrated circuit is configured to: determining whether to entropy code the first block of video data based on the one or more adaptive context models based on one or more of the following: a context of the first video data block; a usage frequency of the context model of the first video data block; stability of the context model used for coding the first block of video data; as well as The context is compared to a threshold value.

17. The apparatus according to claim 10, in, To entropy decode the first video data block, the integrated circuit is configured to: entropy decode the first video data block based on the one or more adaptive context models, and In order to perform entropy decoding on the second video data block, the integrated circuit is configured to: perform entropy decoding on the second video data block based on the one or more non-adaptive context models.

18. The apparatus according to claim 10, in, To entropy decode the first video data block, the integrated circuit is configured to: entropy encode the first video data block based on the one or more adaptive context models, and In order to perform entropy decoding on the second video data block, the integrated circuit is configured to: perform entropy encoding on the second video data block based on the one or more non-adaptive context models.

19. A computer-readable storage medium having stored thereon instructions that, when executed, cause one or more processors to: Entropy decoding is performed on the first video data block based on one or more adaptive context models, wherein The one or more adaptive context models are read from a random access memory RAM; determining that the number of times context-based arithmetic decoding using an adaptive context model is used is greater than or equal to a threshold value; determining a quantization parameter QP associated with a second block of video data based on determining that the number of times is greater than or equal to the threshold; selecting a location in a read-only memory (ROM) from which to read based on the QP associated with the second block of video data, wherein different locations of the ROM are associated with different QP value intervals; and The second block of video data is entropy-coded based on one or more non-adaptive context models, wherein the one or more non-adaptive context models are read from the location of the ROM.

20. An apparatus for decoding video data, the apparatus comprising: means for entropy decoding a first block of video data based on one or more adaptive context models, wherein the one or more adaptive context models are read from a random access memory (RAM); means for determining that the number of times context-based arithmetic decoding using an adaptive context model is used is greater than or equal to a threshold value; means for determining a quantization parameter QP associated with a second block of video data based on said determining that said number of times is greater than or equal to said threshold value; means for selecting a location in a read-only memory ROM to read from based on the QP associated with the second block of video data, wherein different locations of the ROM are associated with different QP value intervals; and Means for entropy coding the second block of video data based on one or more non-adaptive context models, wherein the one or more non-adaptive context models are read from the location of the ROM.

Images