Device, method and medium for video processing

Abstract

A device includes a first bitstream engine and a second bitstream engine. The first bitstream engine is configured to decode a first portion of a first video frame in a plurality of video frames to generate first decoded partial data. The first bitstream engine is also configured to generate synchronization information based on completing the decoding of the first portion. The second bitstream engine is configured to initiate decoding of a second portion of a specific video frame based on the synchronization information to generate second decoded partial data. The second bitstream engine uses the first decoded partial data during decoding of the second portion of the specific video frame. The specific video frame includes a first video frame or a second video frame in a plurality of video frames.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to commonly owned U.S. non-provisional patent application No. 17 / 217,745, filed on March 30, 2021, the contents of which are expressly incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure generally relates to video processing using multiple bitstream engines. Background Art

[0004] Advances in technology have resulted in smaller and more powerful computing devices. For example, there are currently various portable personal computing devices, including wireless phones such as mobile and smart phones, tablet computers and laptop computers, which are small, lightweight and easy for users to carry. These devices can communicate voice and data packets over wireless networks. In addition, many such devices are provided with additional functions, such as digital still cameras, digital video cameras, digital video recorders and audio file players. In addition, such devices can process executable instructions, including software applications that can be used to access the Internet, such as web browser applications. Therefore, these devices can include significant computing capabilities.

[0005] Such computing devices are typically provided with functionality for receiving video frames from a camera. For example, a first device encodes a video frame to send as a bitstream to a second device, and the second device decodes the received bitstream to reconstruct the video frame for display. Emerging video applications increasingly rely on high bitrate throughput. Reducing the encoding time at the first device and the decoding time at the second device can increase the bitrate throughput of the video frame. Summary of the invention

[0006] According to one embodiment of the present disclosure, a device includes a first bitstream engine and a second bitstream engine. The first bitstream engine is configured to decode a first part of a first video frame of multiple video frames to generate a first decoded partial data. The first bitstream engine is also configured to generate synchronization information based on completing the decoding of the first part. The second bitstream engine is configured to initiate decoding of a second part of a specific video frame based on the synchronization information to generate a second decoded partial data. The second bitstream engine uses the first decoded partial data during decoding of the second part of the specific video frame. The specific video frame includes a first video frame or a second video frame of multiple video frames.

[0007] According to another embodiment of the present disclosure, a method includes receiving a bitstream representing multiple video frames at a device. The method also includes decoding a first part of a first video frame of multiple video frames at a first bitstream engine of the device to generate a first decoded partial data. The method also includes generating synchronization information based on completing the decoding of the first part at the first bitstream engine. The method also includes initiating decoding of a second part of a specific video frame at a second bitstream engine of the device to generate a second decoded partial data. The decoding of the second part is performed based on the synchronization information and the first decoded partial data. The specific video frame includes a first video frame or a second video frame of multiple video frames.

[0008] According to another embodiment of the present disclosure, a device includes an interface, a first bitstream engine, and a second bitstream engine. The interface is configured to receive multiple video frames. The first bitstream engine is configured to encode the first part of the first part of the first video frame of the multiple video frames to generate a first encoded part. The first bitstream engine is also configured to generate synchronization information based on the completion of the encoding of the first part. The second bitstream engine is configured to initiate the encoding of the second part of the second part of the specific video frame based on the synchronization information to generate a second encoded part. The second bitstream engine uses the first encoded part of the data during the encoding of the second part of the specific video frame. The specific video frame includes a first video frame or a second video frame of the multiple video frames.

[0009] According to another embodiment of the present disclosure, a method includes receiving multiple video frames at a device. The method also includes encoding a first portion of data of a first portion of a first video frame of multiple video frames at a first bitstream engine of the device to generate a first encoded portion. The method also includes generating synchronization information based on completing the encoding of the first portion at the first bitstream engine. The method also includes initiating encoding of a second portion of data of a second portion of a specific video frame at a second bitstream engine of the device to generate a second encoded portion. The encoding of the second portion is performed based on the synchronization information and the first encoded portion. The specific video frame includes a first video frame or a second video frame of the multiple video frames.

[0010] Other aspects, advantages, and features of the present disclosure will become apparent upon examination of the complete application, which includes the following sections: Brief Description of the Drawings, Detailed Description, and Claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram of specific illustrative aspects of a system operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0012] Figure 2 is a diagram of illustrative aspects of operations associated with processing video using multiple bitstream engines according to some examples of the present disclosure.

[0013] Figure 3 is a diagram of illustrative aspects of operations associated with processing video using multiple bitstream engines according to some examples of the present disclosure.

[0014] Figure 4 According to some examples of the present disclosure Figure 1 A diagram of illustrative aspects of the operation of a video encoder of an illustrative system.

[0015] Figure 5 According to some examples of the present disclosure Figure 4 A diagram of illustrative aspects of the operation of a pre-entropy processing unit of an illustrative video encoder.

[0016] Figure 6 According to some examples of the present disclosure Figure 4 A diagram of illustrative aspects of the operation of a bitstream engine of an illustrative video encoder.

[0017] Figure 7 According to some examples of the present disclosure Figure 1 A diagram of illustrative aspects of the operation of a video decoder of an illustrative system.

[0018] Figure 8 According to some examples of the present disclosure Figure 7 A diagram of illustrative aspects of the operation of a post-entropy processing unit of an illustrative video decoder.

[0019] Fig. 9 According to some examples of the present disclosure Figure 7 A diagram of illustrative aspects of the operation of a bitstream engine of an illustrative video decoder.

[0020] Fig.10 An example of an integrated circuit operable to process video using multiple bitstream engines according to some examples of the present disclosure is illustrated.

[0021] Fig.11 is a diagram of an example of a mobile device operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0022] Fig.12 is a diagram of an example of a wearable electronic device operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0023] Fig.13 is a diagram of an example of a camera operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0024] Fig.14 is a diagram of an example of a headset (such as a virtual reality or augmented reality headset) that is operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0025] Fig.15 is a diagram of a first example of a vehicle operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0026] Fig.16 is a diagram of a second example of a vehicle operable to process video using multiple bitstream engines according to some examples of the present disclosure.

[0027] Fig.17 According to some examples of the present disclosure, Figure 1 A diagram of a specific implementation of a method for processing video using multiple bitstream engines performed by a system.

[0028] Fig.18 According to some examples of the present disclosure, Figure 1 FIG. 1 is a diagram of a specific embodiment of another method for processing video using multiple bitstream engines performed by a system.

[0029] Fig.19 is a block diagram of specific illustrative aspects of a system operable to process video using multiple bitstream engines according to some examples of the present disclosure. DETAILED DESCRIPTION

[0030] Encoding video frames to generate a bitstream for transmission and decoding the bitstream to reconstruct the video frames both increase processing delays that reduce throughput. A system and method for processing video using multiple bitstream engines are disclosed. For example, multiple bitstream engines are used at a first device to encode video frames to generate a bitstream for transmission to a second device, and multiple bitstream engines are used at the second device to decode the bitstream to reconstruct the video frames. Using multiple bitstream engines for encoding, decoding, or both reduces processing delays and improves bitrate throughput.

[0031] The video frame has a specific size (e.g., 720 pixels by 576 pixels). A video frame portion may refer to any portion of a video frame of any size and any shape. For example, a video frame may include rows and columns of video frame portions (e.g., blocks). As another example, a video frame portion may include rows of video frame portions. A video frame portion has a size that is less than or equal to the size of the video frame (e.g., 240 pixels by 288 pixels or 720 pixels by 192 pixels). In some embodiments, each video frame portion of a video frame has the same size (e.g., the same size). In an alternative embodiment, a video frame portion of a video frame has a size different from another video frame portion of the video frame. In some embodiments, a video frame is divided into non-overlapping video frame portions. In other embodiments, a video frame includes partially overlapping video frame portions.

[0032] The first bitstream engine of the video encoder at the first device is used to encode the first video frame portion, and the second bitstream engine of the video encoder is used to encode the second video frame portion. In some examples, the first video frame portion and the second video frame portion are included in the same video frame. In other examples, the first video frame portion is from the first video frame, and the second video frame portion is from another video frame.

[0033] In some cases, the encoding of the second video frame portion depends on the encoding of the first video frame portion (e.g., for motion estimation). In these cases, after the first video frame portion is encoded by the first bitstream engine, the second video frame portion is encoded by the second bitstream engine. For example, the first bitstream encodes the first video frame portion to generate a first encoded video frame portion, and provides the first encoded video frame portion and synchronization information to the second bitstream engine. The second bitstream engine encodes the second video frame portion based on the first encoded video frame portion and synchronization information. In a specific example, concurrently with the second bitstream engine encoding the second video frame portion, the first bitstream engine encodes the third video frame portion. Concurrent encoding of video frames by multiple bitstream engines reduces encoding delay and improves throughput. The encoded video frame portion is sent to the second device as a bitstream.

[0034] The first bitstream engine of the video decoder at the second device is used to decode the first video frame portion, and the second bitstream engine of the video decoder at the second device is used to decode the second video frame portion. In some cases, the decoding of the second video frame portion depends on the decoding of the first video frame portion (for example, for motion estimation). In these cases, after the first video frame portion is decoded by the first bitstream engine, the second video frame portion is decoded by the second bitstream engine. For example, the first bitstream decodes the first video frame portion to generate the first decoded video frame portion, and provides the first decoded video frame portion and synchronization information to the second bitstream engine. The second bitstream engine decodes the second video frame portion based on the first decoded video frame portion and synchronization information. In a specific example, concurrently with the second bitstream engine decoding the second video frame portion, the first bitstream engine decodes the third video frame portion. Concurrent decoding of video frames by multiple bitstream engines reduces decoding delay and improves throughput.

[0035] Digital video devices implement video compression techniques (such as those described in the standards defined by MPEG-2, MPEG-4, ITU-TH.263, Part 10 of ITU-TH.264 / MPEG-4, Advanced Video Codec (AVC), ITU-TH.265, High Efficiency Video Codec (HEVC), and extensions of these standards) to more efficiently send, receive, and store digital video information.

[0036] Video compression techniques include spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or eliminate the redundancy inherent in video sequences. In a specific example, a video frame or slice can be divided into blocks. Each block can be further divided. Blocks in a frame or slice encoded with intra-frame coding (I) are encoded using spatial predictions of reference samples in adjacent blocks in the same frame or slice. Blocks in a frame or slice encoded with inter-frame coding (P or B) can use spatial predictions of reference samples in adjacent blocks in the same frame or slice, or temporal predictions of reference samples in other reference frames. Spatial or temporal predictions result in a prediction block for the block to be coded. The residual data represents the pixel difference between the original block to be coded and the prediction block.

[0037] The inter-coded block is encoded according to a motion vector pointing to a block of reference samples forming a predicted block and residual data indicating the difference between the coded block and the predicted block. The intra-coded block is encoded according to the intra-coding mode and the residual data. For further compression, the residual data can be transformed from the pixel domain to a transform domain (e.g., Fourier or frequency domain) to produce residual transform coefficients, which can then be quantized. The quantized transform coefficients, initially arranged in a two-dimensional array, can be scanned in a specific order to produce a one-dimensional vector of transform coefficients for entropy coding.

[0038] Various entropy coding processes can be used to code the residual transform coefficients, motion vector information, syntax elements and other associated information. Examples of various entropy coding and other data compression processes include context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), probability interval partitioning entropy coding (PIPE), Golomb coding, Golomb-Rice coding and exponential Golomb coding.

[0039] Specific aspects of the present disclosure are described below with reference to the accompanying drawings. In the specification, common features are represented by common reference numerals. In some of the drawings, multiple instances of a specific type of a feature are used. Although the features are physically and / or logically different, each feature uses the same reference number, and different instances are distinguished by adding letters to the reference number. When referring to features as a group or type herein (for example, when no specific one is mentioned), the reference number without a distinguishing letter is used. However, when referring to a specific feature among multiple features of the same type herein, a reference number with a distinguishing letter is used. For example, reference numerals with distinguishing letters are used. Figure 1 , a plurality of video frames are illustrated and associated with reference numbers 109A, 109B, and 109C. When referring to a specific one of these video frames, such as video frame 109A, a distinguishing letter "A" is used. However, when referring to any one of these video frames or referring to these video frames as a group, reference number 109 is used without a distinguishing letter.

[0040] As used herein, various terms are used only for the purpose of describing specific embodiments and are not intended to be limiting of the embodiments. For example, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. In addition, some features described herein are singular in some embodiments and plural in other embodiments. For illustration purposes, Figure 1 Describes a method comprising one or more video encoders ( Figure 11 and 1 , indicating that in some embodiments, the device 102 includes a single video encoder 112, while in other embodiments, the device 102 includes multiple video encoders 112. For ease of reference herein, these features are generally introduced as "one or more" features and are subsequently referred to as the singular unless an aspect involving multiple features is being described.

[0041] As used herein, the terms "comprising," "including," and "containing" may be used interchangeably with "including," "comprising," or "including." Additionally, the term "wherein" may be used interchangeably with "wherein." As used herein, "exemplary" indicates examples, implementations, and / or aspects, and should not be construed as limiting implementations or indicating preferred or preferred implementations. As used herein, ordinal terms (e.g., "first," "second," "third," etc.) used to modify elements such as structures, components, operations do not, by themselves, indicate any priority or order of the element relative to another element, but merely distinguish the element from another element having the same name (but using an ordinal term). As used herein, the term "set" refers to one or more specific elements, and the term "plurality" refers to multiple (e.g., two or more) specific elements.

[0042] As used herein, "coupling" may include "communication coupling", "electrical coupling" or "physical coupling", and may also (or alternatively) include any combination thereof. Two devices (or components) may be directly or indirectly coupled (e.g., communication coupling, electrical coupling or physical coupling) via one or more other devices, components, lines, buses, networks (e.g., wired networks, wireless networks or combinations thereof), etc., unless otherwise specified. Two electrically coupled devices (or components) may be included in the same device or different devices, and may be connected via electronic circuits, one or more connectors or inductive coupling (as illustrative non-limiting examples). In some embodiments, two devices (or components) such as those communicatively coupled in electrical communication may directly or indirectly send and receive signals (e.g., digital signals or analog signals) via one or more lines, buses, networks, etc. As used herein, "directly coupled" may include two devices coupled (e.g., communication coupling, electrical coupling or physical coupling) without an intermediate component. Unless otherwise specified, two devices (or components) that are "coupled" may be directly and / or indirectly coupled.

[0043] In the present disclosure, terms such as "determine", "calculate", "estimate", "shift", "adjust", etc. may be used to describe how to perform one or more operations. It should be noted that these terms should not be interpreted as limiting, and other techniques may be used to perform similar operations. Additionally, as referred to herein, "generate", "calculate", "estimate", "use", "select", "access", and "determine" may be used interchangeably. For example, "generating", "calculating", "estimating" or "determining" a parameter (or signal) may refer to actively generating, estimating, calculating or determining a parameter (or signal), or may refer to using, selecting or accessing an already generated parameter (or signal), such as a parameter generated by another component or device.

[0044] refer to Figure 1 , specific illustrative aspects of a system configured to process video using multiple bitstream engines are disclosed and are generally designated 100. System 100 includes a device 102 configured to be coupled to a device 104 via a network 101. For example, the device 102 includes a transmitter 116 and a modem 114 configured to enable communication with the device 104 via the network 101. The device 104 includes a receiver 126 and a modem 124 configured to enable communication with the device 102 via the network 101. In specific aspects, the network 101 includes a wired network, a wireless network, or both.

[0045] Device 102 is configured to use multiple bitstream engines to process video using one or more video encoders 112. Device 104 is configured to use multiple bitstream engines to process video using one or more video decoders 122.

[0046] The video encoder 112 and the video decoder 122 may operate according to a video codec standard or technology, such as the HEVC standard, the ITU-T H.264 standard (alternatively referred to as MPEG4, Part 10), Advanced Video Codec (AVC), MPEG-2, ITU-T H.263, the Joint Exploration Model (JEM), other codec technologies being explored by the Joint Video Exploration Team (JVET), or extensions of these standards or technologies.

[0047] Despite Figure 1 1, but in some aspects, the video encoder 112 and the video decoder 122 can each be integrated with one or more audio encoders and decoders and can include appropriate MUX-DEMUX units or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams. If applicable, in some examples, the MUX-DEMUX units can conform to the ITU H.223 multiplexer protocol or other protocols such as the User Datagram Protocol (UDP).

[0048] The video encoder 112 may implement any or all of the techniques for binary arithmetic coding (BAC) of the present disclosure during the video coding process. Likewise, the video decoder 122 may implement any or all of these techniques BAC during the video coding process. As will be discussed in more detail below, in one example of the present disclosure, the video decoder 122 may be configured to receive video data including a coded representation of a syntax element, perform inverse binary arithmetic coding on the coded representation of the syntax element to obtain the binary bits (bins) of the syntax element, inversely binarize the binary bits of the syntax element to obtain the syntax element, and decode the video data based on the syntax element. In order to perform inverse binary arithmetic coding, the video decoder 122 may be further configured to determine the probability of a specific coded representation of the coded representation, normalize the probability of the specific coded representation using a right bit shift to create a normalized probability, determine the product of the normalized probability and the range of the specific coded representation, and use the determined product to update the range of the minimum possible symbol of the specific coded representation.

[0049] Similarly, the video encoder 112 may be configured to encode the video data to generate syntax elements, binarize the syntax elements into bins, perform binary arithmetic coding on the bins of the syntax elements to generate a coded representation of the bins, and signal the coded representation in the bitstream. To perform binary arithmetic coding, the video encoder 112 may be further configured to determine a probability of a specific bin of a specific syntax element, normalize the probability of the specific bin using a right bit shift to create a normalized probability, determine a product of the normalized probability and a range of the specific bin, and update the range of the minimum possible symbol of the specific bin using the determined product.

[0050] Digital video devices implement video compression techniques, such as those performed by video encoder 112 and video decoder 122, to more efficiently encode and decode digital video information. Video compression may apply spatial (intra-frame) prediction and / or temporal (inter-frame) prediction techniques to reduce or eliminate redundancy inherent in video sequences.

[0051] For video codecs according to the High Efficiency Video Codec (HEVC or ITU-T H.265) standard, as an example, a video frame can be segmented into codec units. A codec unit (CU) generally refers to an image region used as a basic unit to which various codec tools are applied for video compression. A CU typically has a luminance (or "brightness") component, which can be represented as Y, and two color intensity (or "chrominance") components, which can be represented as Cr and Cb. Depending on the video sampling format, the size of the Cr and Cb components can be the same or different from the size of the Y component in terms of the number of samples. A CU is typically a square image region and can be considered to be similar to a so-called macroblock, for example in other video codec standards such as ITU-T H.264.

[0052] The luminance and chrominance components of pixels in a CU can be encoded and decoded into different subsampling formats. In one proposal of HEVC, the luminance and chrominance components of pixels are encoded and decoded into a 4:2:0 format. In the 4:2:0 pixel format, for each 4x2 pixel block, there are 8 luminance components (4 in each row) and 2 chrominance components (for example, there is 1 Cr chrominance component and 1 Cb chrominance component in the first row of the 4x2 block). The second row of the 4x2 block will have no chrominance information. Therefore, in the 4x2 pixel block, the chrominance components are sampled to 1 / 2 horizontal resolution and 1 / 2 vertical resolution. However, video coding technology is not limited to 4:2:0 chrominance subsampling. Other subsampling formats can be used, including 4:2:2 and 4:4:4. In the 4:2:2 pixel format, for each 4x2 pixel block, there are 8 luma components (4 in each row) and 4 chroma components (e.g., 1 Cr chroma component and 1 Cb chroma component in each of the first and second rows of the 4x2 block). Therefore, for the 4:2:2 format, the chroma components are sampled to 1 / 2 horizontal resolution and full vertical resolution. The 4:4:4 pixel format does not involve subsampling of the chroma components. That is, for a 4x2 pixel block, there are eight luma components, eight Cr components, and eight Cb components. These or other sampling formats may be used.

[0053] In order to achieve better codec efficiency, the codec unit can have a variable size depending on the video content. In addition, the codec unit can be divided into smaller blocks for prediction or transformation. Specifically, each codec unit can be further divided into prediction units and / or transformation units. The prediction unit can be considered as a so-called partition similar to other video codec standards (such as H.264). The transformation unit refers to the residual data block to which the transformation is applied by the video encoder 112 to produce the transformation coefficients.

[0054] For purposes of illustration, coding according to some aspects of the HEVC standard will be described in this application. However, the techniques described in this disclosure may be useful for other video coding processes, such as those defined according to H.264 or other standards or proprietary video coding processes.

[0055] HEVC standardization work is based on a video codec device model called the HEVC Test Model (HM). HM has several capabilities that other codec standards (e.g., ITU-T H.264 / AVC, etc.) do not have. For example, H.264 provides nine intra-frame prediction coding modes, while HM provides up to thirty-five intra-frame prediction coding modes.

[0056] According to HM, a CU may include one or more prediction units and / or one or more transform units. The syntax data within the bitstream may define a maximum codec unit (LCU) or a codec tree block (CTB), which is the largest CU in terms of the number of pixels. Generally, the purpose of a CU is similar to that of a macroblock of H.264, except that there is no size difference in a CU. Therefore, a CU may be divided into sub-CUs. Generally, a CU mentioned in the present disclosure may refer to an LCU of a block or a sub-CU of an LCU. An LCU may be divided into sub-CUs, and each sub-CU may be further divided into more sub-CUs. The syntax data of the bitstream may define the maximum number of times an LCU may be divided, which is referred to as CU depth. Accordingly, the bitstream may also define a minimum codec unit (SCU). The present disclosure also uses the terms "block", "partition" or "portion" to refer to any one of a CU, a prediction unit or a transform unit. Generally, a "portion" may refer to any subset of a video frame (e.g., an LCU, an SCU, a sub-CU or a CU).

[0057] The video encoder 112 includes a plurality of bitstream engines 130, such as a bitstream engine 130A, a bitstream engine 130B, one or more additional bitstream engines, or a combination thereof. The video encoder 112 is configured to be coupled to the camera 106 via an input interface 110. The input interface 110 is configured to receive a camera output 107 from the camera 106 and provide the camera output 107 to the video encoder 112 as video data representing a video frame (VF) 109. For example, a video data memory 118 (e.g., a buffer) of the video encoder 112 is configured to store video data representing one or more video frame portions (VFPs) 151 of the video frame 109. In a specific aspect, the video frame is divided into rows and columns of tiles, and the VFP 151 includes tiles of the video frame, as described in reference to FIG. Figure 2 In a specific aspect, a video frame is divided into horizontal lines (eg, slices), and VFP 151 includes the lines of the video frame, as described in reference Figure 3In particular aspects, VFP 151 includes a slice, a tile, a block, an entire frame, or another portion of a video frame.

[0058] The plurality of bitstream engines 130 of the video encoder 112 are configured to encode the portion data 111 of the video frame portion 151 to generate an encoded video frame portion (EVFP) 117. The video encoder 112 is configured to initiate transmission of the encoded video frame portion 117 to the device 104 as a bitstream 103.

[0059] The video decoder 122 includes a plurality of bitstream engines 140, such as a bitstream engine 140A, a bitstream engine 140B, one or more additional bitstream engines, or a combination thereof. The video decoder 122 is configured to receive a bitstream 103 representing an encoded video frame portion 117. In a specific aspect, the encoded video frame portion 117 received by the device 104 is the same as the encoded video frame portion 117 sent by the device 102. In an alternative aspect, the encoded video frame portion 117 received by the device 104 is at least partially different from the encoded video frame portion 117 sent by the device 102 due to changes during transmission (such as bit flips, packet losses, etc.). In a specific embodiment, the encoded video frame portion 117 is sent by the device 102 using a first transmission (or packetization) protocol that is different from a second transmission (or packetization) protocol used by the device 104 to receive the encoded video frame portion 117. For example, the encoded video frame portion 117 may be transcoded between being sent by the device 102 and being received by the device 104.

[0060] The plurality of bitstream engines 140 of the video decoder 122 are configured to decode the encoded video frame portion 117 to generate decoded partial data (DPD) 121. The video decoder 122 is configured to generate a reconstructed video frame (RVF) 129 based on the decoded partial data 121. In a specific aspect, the reconstructed video frame 129 generated by the video decoder 122 is the same as the video frame 109 encoded by the video encoder 112. In an alternative aspect, the reconstructed video frame 129 is at least partially different from the video frame 109 due to changes during encoding, transmission, decoding, or a combination thereof (such as bit flipping, packet loss, processing loss, etc.). In a specific embodiment, the video decoder 122 is configured to provide the reconstructed video frame 129 to the display device 108 via the output interface 120. For example, the video decoder 122 is configured to store the reconstructed video frame 129 in the reference frame memory 128 and output the reconstructed video frame 129 stored in the reference frame memory 128 to the display device 108 for playback. Display device 108 is configured to display reconstructed video frame 129 .

[0061] In some implementations, device 102, device 104, or both correspond to or are included in various types of devices. In the illustrative example, video encoder 112, video decoder 122, or both are integrated into a Fig.11 The mobile phone or tablet computer device described, as referenced Fig.12 The wearable electronic device described in reference Fig.13 The camera device described or as referenced Fig.14 In another illustrative example, the video encoder 112, the video decoder 122, or both are integrated into a vehicle, such as a vehicle described in the accompanying drawings. Fig.15 and Fig.16 Further described.

[0062] In the same or alternative embodiments, the camera 106, the display 108, or both correspond to or are included in various types of devices. In the illustrative example, the camera 106, the display 108, or both are integrated into the device as shown in FIG. Fig.11 The mobile phone or tablet computer device described, as referenced Fig.12 The wearable electronic device described in reference Fig.13 The camera device described or as referenced Fig.14 In another illustrative example, the camera 106, the display 108, or both are integrated into a vehicle, such as the one described in the accompanying drawings. Fig.15 and Fig.16 Further described.

[0063] During operation, the input interface 110 receives a camera output 107 representing a video frame 109 from the camera 106. For example, the video frame 109 captures an image of the user 190 during a video call (e.g., a conference call) with the user 192 of the device 104. The video frame 109 includes a sequence of video frames, such as video frame 109A, video frame 109B, video frame 109C, one or more additional video frames, or a combination thereof. The video encoder 112 stores data representing a video frame portion 151 of the video frame 109 in the video data memory 118. For example, the video encoder 112 stores the video frame portion 151A of the video frame 109A in the video data memory 118. In a specific aspect, the video encoder 112 also stores the video frame portion 151B, the video frame portion 151C, or both in the video data memory 118. In a specific example, two or more of video frame portion 151A, video frame portion 151B, and video frame portion 151C are different portions of the same video frame (e.g., video frame 109A), as shown in reference Figure 2In another example, two or more of the video frame portion 151A, the video frame portion 151B, and the video frame portion 151C are different video frame portions (e.g., two or more of the video frame 109A, the video frame 109B, and the video frame 109C), as described in reference Figure 3 Further described.

[0064] The video encoder 112 generates partial data 111A based on the video frame portion 151A, as shown in FIG. Figure 5 Similarly, the video encoder 112 generates the partial data 111B based on the video frame portion 151B, as shown in FIG. Figure 5 Further described.

[0065] The video encoder 112 dispatches portions of data to the plurality of bitstream engines 130 based on dependencies between the VFPs 151. In a specific example, the VFPs 151 correspond to tiles and have dependencies at the horizontal edges of the tiles, as shown in FIG. Figure 2 In this example, the video encoder 112 distributes the partial data of the alternative tiles of the same video frame to each of the plurality of bitstream engines 130, as shown in FIG. Figure 2 In a specific example, VFP 151 corresponds to a row (eg, a slice) and has dependencies across frames, as described in reference Figure 3 In this example, the video encoder 112 distributes partial data of lines of different frames to each of the plurality of bitstream engines 130, as shown in FIG. Figure 3 In the same or alternative embodiment(s), the video encoder 112 determines dependencies of the VFP 151 based on dependency information indicated by configuration settings, user input, default data, or a combination thereof, performs dispatching portions of data to multiple bitstream engines 130, or both.

[0066] In a specific aspect, the video encoder 112 dispatches the partial data 111A corresponding to the video frame portion 151A to the bitstream engine 130A, and dispatches the partial data 111B corresponding to the video frame portion 151B to the bitstream engine 130B. The bitstream engine 130A encodes the partial data 111A to generate the encoded video frame portion 117A, as shown in FIG. Figure 4 The bitstream engine 130A also generates synchronization information 115A based on completing the encoding of the video frame portion 151A (eg, the portion of the data 111A of the video frame portion 151A), as described in reference to FIG. Figure 4 and 6In particular embodiments, the synchronization information 115A is based on (e.g., depends on or indicates) a prediction mode (e.g., an intra-prediction mode or an inter-prediction mode) associated with encoding the video frame portion 151A, coded block information, coefficient information, motion vector information, reference index, reference mode, or a combination thereof.

[0067] The bitstream engine 130B initiates encoding of the portion of data 111B based on the synchronization information 115A to generate an encoded video frame portion 117B, as shown in FIG. Figure 4 and 6 As further described. The bitstream engine 130B uses the encoded video frame portion 117A during encoding of the video frame portion 151B (eg, the portion of data 111B of the video frame portion 151B), as described in reference to Figure 4 and 6 Further described.

[0068] After the bitstream engine 130A encodes the portion of data 111A, the bitstream engine 130B encodes the portion of data 111B. In a specific aspect, concurrently with the bitstream engine 130B encoding the portion of data 111B of the video frame portion 151B, the bitstream engine 130A encodes the portion of data of another video frame portion (e.g., the video frame portion 151C), as shown in FIG. Figure 2-3 For example, bitstream engine 130A encodes a portion of video frame portion 151C to generate encoded video frame portion 117C. Concurrent encoding of video frame portion 151C with encoding of video frame portion 151B reduces encoding time and increases throughput of video frames 109. Video encoder 112 provides encoded video frame portion 117 to modem 114 for transmission as bitstream 103 via transmitter 116.

[0069] Receiver 126 receives bitstream 103 representing video frame 109 from device 102 via network 101. Receiver 126 provides bitstream 103 as encoded video frame portion 117 to video decoder 122 via modem 124. Bitstream engine 140A decodes encoded video frame portion 117A (e.g., corresponding to video frame portion 151A) to generate decoded portion data 121A, as shown in FIG. Figure 7 and 9 The bitstream engine 140A also generates synchronization information 125A based on completing the decoding of the encoded video frame portion 117A, as shown in FIG. Figure 7 and 9In particular embodiments, the synchronization information 125A is based on (e.g., depends on or indicates) a prediction mode (e.g., an intra-prediction mode or an inter-prediction mode) associated with decoding of the encoded video frame portion 117A, encoded and decoded block information, coefficient information, motion vector information, reference indexes, reference modes, or a combination thereof.

[0070] The bitstream engine 140B initiates decoding of the encoded video frame portion 117B (eg, corresponding to the video frame portion 151B) based on the synchronization information 125A to generate decoded partial data 121B, as shown in FIG. Figure 7 and 9 As further described. The bitstream engine 140B uses the decoded portion data 121A during decoding of the encoded video frame portion 117B, as described in reference Figure 7 and 9 In a specific aspect, the decoded partial data 121A and the decoded partial data 121B are respectively the same as the partial data 111A and the partial data 111B. In an alternative aspect, the decoded partial data 121A and the decoded partial data 121B are respectively at least partially different from the partial data 111A and the partial data 111B due to transmission loss, processing loss, bit flipping, etc.

[0071] After bitstream engine 140A decodes coded video frame portion 117A, bitstream engine 140B decodes coded video frame portion 117B. In a specific aspect, bitstream engine 140A decodes another coded video frame portion (e.g., coded video frame portion 117C) concurrently with bitstream engine 140B decoding coded video frame portion 117B. Concurrent decoding of coded video frame portion 117C with decoding of coded video frame portion 117B reduces decoding time and improves throughput of coded video frame portion 117.

[0072] The video decoder 122 generates a reconstructed video frame 129 from the decoded partial data 121, as shown in FIG. Figure 8 Further described. In a specific aspect, the reconstructed video frame 129A, the reconstructed video frame 129B, and the reconstructed video frame 129C are respectively the same as the video frame 109A, the video frame 109B, and the video frame 109C. In an alternative aspect, the reconstructed video frame 129A, the reconstructed video frame 129B, and the reconstructed video frame 129C are respectively different from the video frame 109A, the video frame 109B, and the video frame 109C at least in part due to transmission loss, processing loss, bit flipping, etc. The video decoder 122 provides data representing the reconstructed video frame 129 to the display device 108 via the output interface 120.

[0073] The system 100 thus improves the throughput of video frames 109 by using multiple bitstream engines for encoding, multiple bitstream engines for decoding, or a combination thereof. Sharing synchronization information across bitstream engines enables video frame portions with dependencies to be processed (e.g., encoded or decoded) by multiple bitstream engines.

[0074] Although video frames 109 are illustrated as being based on camera output 107 received from camera 106, in other implementations, video frames 109 are received from one or more storage devices, servers, other devices, or a combination thereof. In some examples, video frames 109 are generated by an application (e.g., a gaming application, a graphics application, or a combination thereof) of device 102. Although reconstructed video frames 129 are illustrated as being provided to display device 108 by video decoder 122, in some implementations, video decoder 122 provides reconstructed video frames 129 to one or more memories and / or other devices.

[0075] In some examples, video encoder 112 and video decoder 122 are integrated into a single device. In a specific example, each of device 102 and device 104 includes a copy of video encoder 112 and a copy of video decoder 122. For example, video encoder 112 of device 104 generates an encoded video frame portion corresponding to a video frame representing a video of user 192, and video decoder 122 generates a reconstructed video frame from the encoded video frame portion.

[0076] As an illustrative example, a bitstream engine 130 included in a single encoder (e.g., video encoder 112) and a bitstream engine 140 included in a single decoder (e.g., video decoder 122) are provided. In other examples, the bitstream engine 130 can be distributed across multiple encoders, multiple devices, or a combination thereof. Similarly, in some examples, the bitstream engine 140 can be distributed across multiple decoders, multiple devices, or a combination thereof.

[0077] Figure 22 is a diagram of illustrative aspects of operations associated with processing video based on horizontal edge dependencies using multiple bitstream engines. In example 200, a video frame 109A is divided into a plurality of rows 240 (e.g., rows 240A, 240B) and a plurality of columns 244 (e.g., columns 244A, 244B, 244C) of a video frame portion 151 (e.g., tiles) (e.g., video frame portions 151A, 151B, 151C, 151D, 151E, 151F). For illustration, the video frame 109A includes row 240A (row 0) and row 240B (row 1) of the video frame portion 151. The video frame 109A includes columns 244A, 244B, and 244C of the video frame portion 151. A video frame 109A including an even count of rows and an odd count of columns is provided as an illustrative example. Video frame 109A may include any row count greater than one and any column count greater than one.

[0078] In example 200, each video frame portion 151 of row 240B has a dependency on a corresponding video frame portion 151 of row 240A. For example, video frame portion 151B, video frame portion 151D, and video frame portion 151F of row 240B have dependencies on video frame portion 151A, video frame portion 151C, and video frame portion 151E, respectively.

[0079] Figure 250 is available in Figure 1 An example of operations associated with processing video using multiple bitstream engines performed at multiple bitstream engines 130, video encoder 112, device 102, system 100, or a combination thereof. The video encoder 112 dispatches alternative video frame portions 151 of a video frame 109A to multiple bitstream engines 130. For example, the video encoder 112 dispatches even-numbered video frame portions to the bitstream engine 130A and odd-numbered video frame portions to the bitstream engine 130B. To illustrate, the video encoder 112 dispatches video frame portion 151A (tile 0), video frame portion 151E (tile 2), and video frame portion 151D (tile 4) to the bitstream engine 130A, and dispatches video frame portion 151C (tile 1), video frame portion 151B (tile 3), and video frame portion 151F (tile 5) to the bitstream engine 130B.

[0080] In particular embodiments, each bitstream engine 130 has access to dependency information indicating dependencies between various video frame portions. In particular embodiments, video encoder 112 provides video frame dependency information to each bitstream engine 130 indicating whether a video frame portion has a dependency on another video frame portion.

[0081] In response to determining that video frame portion 151A (tile 0) has no dependencies on any other video frame portion, bitstream engine 130A generates synchronization information 115A and encoded video frame portion 117A based on portion data 111A of video frame portion 151A, as shown in reference Figure 1 as described. In a specific embodiment, the synchronization information 115A is based on (e.g., depends on or indicates) a prediction mode (e.g., an inter-prediction mode or an intra-prediction mode), encoded block information, coefficient information, or a combination thereof associated with encoding of the video frame portion 151A (tile 0). The bitstream engine 130A stores the synchronization information 115A and the encoded video frame portion 117A in a line buffer 202A of the video encoder 112 in response to determining that the video frame portion 151B has a dependency on the video frame portion 151A. In a specific embodiment, the bitstream engine 130A provides a notification to the bitstream engine 130B that information associated with the video frame portion 151A is available in the line buffer 202A, that the video frame portion 151B has a dependency on the video frame portion 151A, or both.

[0082] In response to determining that video frame portion 151C (tile 1) has no dependencies on any other video frame portion, bitstream engine 130B generates synchronization information 115C and encoded video frame portion 117C based on the partial data of video frame portion 151C, as shown in reference Figure 1 In response to determining that video frame portion 151D has a dependency on video frame portion 151C, bitstream engine 130B stores synchronization information 115C and encoded video frame portion 117C in line buffer 202B of video encoder 112. In a particular embodiment, bitstream engine 130B provides notification to bitstream engine 130A. The notification is that information associated with video frame portion 151C is available in line buffer 202B, video frame portion 151D has a dependency on video frame portion 151C, or both.

[0083] In a specific aspect, bitstream engine 130A encodes video frame portion 151A to generate encoded video frame portion 117A concurrently with bitstream engine 130B encoding video frame portion 151C to generate encoded video frame portion 117C. Since each of video frame portion 151A and video frame portion 151C has no dependency on any video frame portion, video frame portion 151A and video frame portion 151C can be encoded concurrently.

[0084] After encoding video frame portion 151A (tile 0), bitstream engine 130A identifies video frame portion 151E (tile 2) as the next assigned video frame portion. In response to determining that video frame portion 151E (tile 2) has no dependency on any other video frame portion, bitstream engine 130A generates synchronization information 115E and encoded video frame portion 117E based on the partial data of video frame portion 151E, as shown in reference to FIG. Figure 1 As described. The bitstream engine 130A determines, in response to determining that the video frame portion 151F has a dependency on the video frame portion 151E, that the synchronization information 115E and the encoded video frame portion 117E are to be stored in the line buffer 202A. The bitstream engine 130A stores the synchronization information 115E and the encoded video frame portion 117E in the line buffer 202A in response to determining that the line buffer 202A has available space. Alternatively, the bitstream engine 130A waits until there is available space in the line buffer 202A to store the synchronization information 115E and the encoded video frame portion 117E in the line buffer 202A in response to determining that the line buffer 202A is full. In a specific embodiment, the line buffer 202A is configured to store the synchronization information and the encoded video frame portion of a single video frame portion at a time. In this embodiment, after bitstream engine 130B retrieves synchronization information 115A and encoded video frame portion 117A from line buffer 202A, bitstream engine 130A stores synchronization information 115E and encoded video frame portion 117E in line buffer 202A. In particular embodiments, bitstream engine 130A provides notification to bitstream engine 130B that information associated with video frame portion 151E is available in line buffer 202A, that video frame portion 151F has a dependency on video frame portion 151E, or both.

[0085] The bitstream engine 130B identifies video frame portion 151B (Tile 3) as the next assigned video frame portion after encoding video frame portion 151C (Tile 1). The bitstream engine 130B determines whether information associated with video frame portion 151A (e.g., synchronization information 115A and encoded video frame portion 117A) is available in line buffer 202A in response to determining that video frame portion 151B (Tile 3) has a dependency on video frame portion 151A (Tile 1). In a specific aspect, the bitstream engine 130B initiates encoding of video frame portion 151B in response to determining that line buffer 202A has available data (e.g., synchronization information 115A and encoded video frame portion 117A), receiving a notification from the bitstream engine 130A indicating that information associated with video frame portion 151A is available in line buffer 202A, or both. The bitstream engine 130B retrieves the synchronization information 115A and the encoded video frame portion 117A from the line buffer 202A, and encodes the video frame portion 151B to generate the encoded video frame portion 117B. For example, the bitstream engine 130B encodes the video frame portion 151B (tile 3) based at least in part on the synchronization information 115A (e.g., prediction mode, encoded block information, coefficient information, or a combination thereof) associated with the encoding of the video frame portion 151A (tile 0). The bitstream engine 130B determines that the encoded video frame portion 117B will not be stored in the line buffer 202B in response to determining that none of the video frame portions 151 have a dependency on the video frame portion 151B.

[0086] In a specific aspect, concurrently with the bitstream engine 130B encoding the video frame portion 151B to generate the encoded video frame portion 117B, the bitstream engine 130A encodes the video frame portion 151E to generate the encoded video frame portion 117E. After the availability of the synchronization information 115A and the encoded video frame portion 117A in the line buffer 202A and after the availability of the bitstream engine 130B (e.g., after encoding the video frame portion 151C), the video frame portion 151E and the video frame portion 151B can be encoded concurrently.

[0087] The bitstream engine 130A identifies video frame portion 151D (tile 4) as the next assigned video frame portion after encoding video frame portion 151E (tile 2) and storing synchronization information 115E and encoded video frame portion 117E in line buffer 202A. The bitstream engine 130A retrieves synchronization information 115C and encoded video frame portion 117C from line buffer 202B in response to determining that video frame portion 151D (tile 4) has a dependency on video frame portion 151C (tile 1) and that synchronization information 115C and encoded video frame portion 117C are available in line buffer 202B. The bitstream engine 130A encodes video frame portion 151D to generate an encoded video frame portion. In response to determining that none of the video frame portions 151 have a dependency on video frame portion 151D, bitstream engine 130A determines that the encoded video frame portion corresponding to video frame portion 151D is not to be stored in line buffer 202A.

[0088] The bitstream engine 130B, after encoding the video frame portion 151B (tile 3), identifies the video frame portion 151F (tile 5) as the next assigned video frame portion. The bitstream engine 130B retrieves the synchronization information 115E and the encoded video frame portion 117E from the line buffer 202A in response to determining that the video frame portion 151F (tile 5) has a dependency on the video frame portion 151E (tile 2) and that the synchronization information 115E and the encoded video frame portion 117E of the video frame portion 151E are available in the line buffer 202A. The bitstream engine 130B encodes the video frame portion 151F to generate the encoded video frame portion. The bitstream engine 130B determines that the encoded video frame portion corresponding to the video frame portion 151F will not be stored in the line buffer 202B in response to determining that none of the video frame portions 151 have a dependency on the video frame portion 151F.

[0089] In a specific aspect, concurrently with the bitstream engine 130B encoding the video frame portion 151F by the bitstream engine 130A, the video frame portion 151D is encoded. After the availability of the synchronization information 115E and the encoded video frame portion 117E in the line buffer 202A and after the availability of the synchronization information 115C and the encoded video frame portion 117E in the line buffer 202B, the video frame portion 151D and the video frame portion 151F can be encoded concurrently. As an illustrative example, the concurrent encoding of some video frame portions that are not dependent on each other is provided. In other examples, some video frame portions that are not dependent on each other can be encoded non-concurrently (e.g., sequentially). For illustration, the video frame portion 151D can be encoded by the bitstream engine 130A before or after the bitstream engine 130B encodes the video frame portion 151F.

[0090] Figure 270 can be Figure 1 Examples of operations associated with processing video using multiple bitstream engines performed at multiple bitstream engines 140, video decoder 122, device 104, system 100, or a combination thereof.

[0091] Similar to the video encoder 112 dispatching the video frame portions 151 to the multiple bitstream engines 130, the video decoder 122 dispatches the encoded video frame portions 117 to the multiple bitstream engines 140. In a specific aspect, the video decoder 122 dispatches alternative encoded video frame portions 117 to the multiple bitstream engines 140. For example, the video decoder 122 dispatches the encoded video frame portion 117A (corresponding to tile 0), the encoded video frame portion 117E (corresponding to tile 2), and the encoded video frame portion 117D (corresponding to tile 4) to the bitstream engine 140A, and dispatches the encoded video frame portion 117C (corresponding to tile 1), the encoded video frame portion 117B (corresponding to tile 3), and the encoded video frame portion 117F (corresponding to tile 5) to the bitstream engine 140B.

[0092] Similar to the plurality of bitstream engines 130, the plurality of bitstream engines 140 stores and retrieves data from the line buffer based on dependencies. In particular embodiments, each bitstream engine 140 has access to dependency information indicating dependencies between various encoded video frame portions. In particular embodiments, the video decoder 122 provides encoded video frame dependency information to each bitstream engine 140 indicating whether an encoded video frame portion has a dependency on another encoded video frame portion.

[0093] In response to determining that the encoded video frame portion 117A (corresponding to tile 0) has no dependencies on any other encoded video frame portion, the bitstream engine 140A generates synchronization information 125A and decoded partial data 121A based on the encoded video frame portion 117A, as shown in FIG. Figure 1 as described. In a specific embodiment, the synchronization information 125A is based on (e.g., depends on or indicates) a prediction mode (e.g., an inter-prediction mode or an intra-prediction mode), encoded and decoded block information, coefficient information, or a combination thereof associated with decoding of the encoded video frame portion 117A (tile 0). The bitstream engine 140A stores the synchronization information 125A and the decoded partial data 121A in a line buffer 204A of the video decoder 122 in response to determining that the encoded video frame portion 117B has a dependency on the encoded video frame portion 117A. In a specific embodiment, the bitstream engine 140A provides a notification to the bitstream engine 140B that information associated with the encoded video frame portion 117A is available in the line buffer 204A, that the encoded video frame portion 117B has a dependency on the encoded video frame portion 117A, or both.

[0094] In response to determining that the encoded video frame portion 117C (corresponding to tile 1) has no dependencies on any other video frame portion, the bitstream engine 140B generates synchronization information 125C and decoded partial data 121C based on the encoded video frame portion 117C, as shown in FIG. Figure 1 In response to determining that the encoded video frame portion 117D has a dependency on the encoded video frame portion 117C, the bitstream engine 140B stores the synchronization information 125C and the decoded partial data 121C in the line buffer 204B of the video decoder 122. In particular embodiments, the bitstream engine 140B provides a notification to the bitstream engine 140A that the information associated with the encoded video frame portion 117C is available in the line buffer 204B, that the encoded video frame portion 117D has a dependency on the encoded video frame portion 117C, or both.

[0095] In a specific aspect, bitstream engine 140A decodes coded video frame portion 117A to generate decoded partial data 121A concurrently with bitstream engine 140B decoding coded video frame portion 117C to generate decoded partial data 121C. Coded video frame portion 117A and coded video frame portion 117C may be decoded concurrently because each of coded video frame portion 117A and coded video frame portion 117C has no dependency on any other coded video frame portion.

[0096] The bitstream engine 140A, after decoding the coded video frame portion 117A (corresponding to tile 0), identifies the coded video frame portion 117E (corresponding to tile 2) as the next allocated coded video frame portion. In response to determining that the coded video frame portion 117E (corresponding to tile 2) has no dependency on any other coded video frame portion, the bitstream engine 140A generates synchronization information 125E and decoded partial data 121E based on the coded video frame portion 117E, as shown in reference Figure 1 As described. The bitstream engine 140A determines that the synchronization information 125E and the decoded partial data 121E are to be stored in the line buffer 204A in response to determining that the encoded video frame portion 117F has a dependency on the encoded video frame portion 117E. The bitstream engine 140A stores the synchronization information 125E and the decoded partial data 121E in the line buffer 204A in response to determining that the line buffer 204A has available space. Alternatively, the bitstream engine 140A waits until there is available space in the line buffer 204A in response to determining that the line buffer 204A is full to store the synchronization information 125E and the decoded partial data 121E in the line buffer 204A. In a specific embodiment, the line buffer 204A is configured to store the synchronization information and the decoded partial data for a single video frame portion at a time. In this embodiment, after bitstream engine 140B retrieves synchronization information 125A and decoded partial data 121A from line buffer 204A, bitstream engine 140A stores synchronization information 125E and decoded partial data 121E in line buffer 204A. In particular embodiments, bitstream engine 140A provides notification to bitstream engine 140B that information associated with coded video frame portion 117E is available in line buffer 204B, that coded video frame portion 117F has a dependency on coded video frame portion 117E, or both.

[0097] The bitstream engine 140B, after decoding the coded video frame portion 117C (corresponding to tile 1), identifies the coded video frame portion 117B (corresponding to tile 3) as the next assigned coded video frame portion. In response to determining that the coded video frame portion 117B (corresponding to tile 3) has a dependency on the coded video frame portion 117A (corresponding to tile 1), the bitstream engine 140B determines whether the synchronization information 125A and the decoded partial data 121A of the coded video frame portion 117A are available in the line buffer 204A. In a specific aspect, the bitstream engine 140B initiates decoding of the encoded video frame portion 117B in response to determining that the line buffer 204A has available data (e.g., synchronization information 125A and decoded partial data 121A), receiving a notification from the bitstream engine 140A indicating that information associated with the encoded video frame portion 117A (e.g., synchronization information 125A and decoded partial data 121A) is available in the line buffer 204A, or both. The bitstream engine 140B retrieves the synchronization information 125A and the decoded partial data 121A from the line buffer 204A and decodes the encoded video frame portion 117B to generate the decoded partial data 121B. For example, the bitstream engine 130B decodes the encoded video frame portion 117B (tile 3) based at least in part on the synchronization information 125A (e.g., prediction mode, encoded and decoded block information, coefficient information, or a combination thereof) associated with the decoding of the video frame portion 117A (tile 0). The bitstream engine 140B determines that the decoded partial data 121B is not to be stored in the line buffer 204B in response to determining that none of the video frame portions 151 have a dependency on the encoded video frame portion 117B.

[0098] In a specific aspect, concurrently with the bitstream engine 140B decoding the coded video frame portion 117B to generate the decoded partial data 121B, the bitstream engine 140A decodes the coded video frame portion 117E to generate the decoded partial data 121E. After the availability of the synchronization information 125A and the decoded partial data 121A in the line buffer 204A and after the availability of the bitstream engine 140B (e.g., after decoding the coded video frame portion 117C), the coded video frame portion 117E and the coded video frame portion 117B can be decoded concurrently.

[0099] The bitstream engine 140A identifies the coded video frame portion 117D (corresponding to tile 4) as the next assigned coded video frame portion after decoding the coded video frame portion 117E (corresponding to tile 2) and storing the synchronization information 125E and the decoded partial data 121E in the line buffer 204A. The bitstream engine 140A retrieves the synchronization information 125C and the decoded partial data 121C from the line buffer 204B in response to determining that the coded video frame portion 117D (corresponding to tile 4) has a dependency on the coded video frame portion 117C (corresponding to tile 1) and the synchronization information 125C and the decoded partial data 121C are available in the line buffer 204B. The bitstream engine 140A decodes the coded video frame portion 117D to generate the decoded partial data. In response to determining that none of the encoded video frame portions 117 have a dependency on the encoded video frame portion 117D, the bitstream engine 140A determines that the decoded portion data corresponding to the encoded video frame portion 117D is not to be stored in the line buffer 204A.

[0100] The bitstream engine 140B, after decoding the coded video frame portion 117B (corresponding to tile 3), identifies the coded video frame portion 117F (corresponding to tile 5) as the next assigned coded video frame portion. In response to determining that the coded video frame portion 117F (corresponding to tile 5) has a dependency on the coded video frame portion 117E (corresponding to tile 2) and that the synchronization information 125E and the decoded partial data 121E of the coded video frame portion 117E are available in the line buffer 204A, the bitstream engine 140B retrieves the synchronization information 125E and the decoded partial data 121E from the line buffer 204A. The bitstream engine 140B decodes the coded video frame portion 117F to generate the decoded partial data. In response to determining that none of the encoded video frame portions 117 have a dependency on the encoded video frame portion 117F, the bitstream engine 140B determines that the decoded portion data corresponding to the encoded video frame portion 117F is not to be stored in the line buffer 204B.

[0101] In a specific aspect, concurrently with the bitstream engine 140B decoding the encoded video portion 117F, the bitstream engine 140A decodes the encoded video frame portion 117D. After the availability of the synchronization information 125E and the decoded partial data 121E in the line buffer 204A and after the availability of the synchronization information 125C and the decoded partial data 121E in the line buffer 204B, the encoded video frame portion 117D and the encoded video frame portion 117F can be decoded concurrently. As an illustrative example, concurrent decoding of some encoded video frame portions that have no dependency on each other is provided. In other examples, some encoded video frame portions that have no dependency on each other can be decoded non-concurrently (e.g., sequentially). For illustration, the encoded video frame portion 117D can be decoded by the bitstream engine 140A before or after the bitstream engine 140B decodes the encoded video frame portion 117F.

[0102] The use of multiple bitstream engines enables concurrent processing of video data, thereby reducing processing time and increasing throughput. Exchanging synchronization information between bitstream engines enables dependencies between video data to be taken into account during concurrent processing.

[0103] Figure 3 is a diagram of illustrative aspects of operations associated with processing video using multiple bitstream engines based on inter-frame dependencies. In example 300, each of video frames 109A-C is divided into rows (e.g., slices) corresponding to video frame portions 151. For illustration, video frame 109A includes video frame portion 151A (row 0), video frame portion 151C (row 1), video frame portion 151E (row 2), one or more additional rows, or a combination thereof. Video frame 109B includes video frame portion 151B (row 0), video frame portion 151D (row 1), video frame portion 151F (row 2), one or more additional rows, or a combination thereof. In a specific aspect, video frame portion 151A (e.g., row, block, tile, etc.) is from a first position (e.g., a first shape at a first coordinate) of video frame 109A, and video frame portion 151B (e.g., row, block, tile, etc.) is from a first position (e.g., a first shape at a first coordinate) of video frame 109B.

[0104] In example 300, each video frame portion 151 (e.g., row(s)) of video frame 109B has a dependency on a corresponding one of video frame portions 151 (e.g., row(s)) of video frame 109A. For example, video frame portion 151B, video frame portion 151D, and video frame portion 151F of video frame 109B have dependencies on video frame portion 151A, video frame portion 151C, and video frame portion 151E, respectively.

[0105] Figure 350 is available in Figure 1 Examples of operations associated with processing video using multiple bitstream engines performed at multiple bitstream engines 130, video encoder 112, device 102, system 100, or a combination thereof.

[0106] The video encoder 112 dispatches the video frame portion 151 of the video frame 109A to the bitstream engine 130A, and dispatches the video frame portion 151 of the video frame 109B to the bitstream engine 130B. For example, the video encoder 112 dispatches the video frame portion 151A (row 0 of the video frame 109A), the video frame portion 151C (row 1 of the video frame 109A), and the video frame portion 151E (row 2 of the video frame 109A) to the bitstream engine 130A, and dispatches the video frame portion 151B (row 0 of the video frame 109B), the video frame portion 151D (row 1 of the video frame 109B), and the video frame portion 151F (row 2 of the video frame 109B) to the bitstream engine 130B.

[0107] The bitstream engine 130A generates synchronization information and encoded video frame portions for each video frame portion of the video frame 109A, and stores the synchronization information and the encoded video frame portions in the line buffer 202A. The bitstream engine 130B retrieves the synchronization information and the encoded video frame portions from the line buffer 202A to generate encoded video frame portions for the corresponding video frame portions of the video frame 109B. For example, the bitstream engine 130A generates synchronization information 115A and encoded video frame portion 117A based on the portion data 111A of the video frame portion 151A (line 0 of the video frame 109A), as shown in FIG. Figure 1 In particular embodiments, synchronization information 115A is based on (e.g., depends on or indicates) a co-located motion vector, a reference index, a reference pattern, or a combination thereof associated with encoding of video frame portion 151A (line 0 of video frame 109A). Bitstream engine 130A stores synchronization information 115A and encoded video frame portion 117A in line buffer 202A of video encoder 112.

[0108] In response to determining that video frame portion 151B (row 0 of video frame 109B) has a dependency on video frame portion 151A (row 0 of video frame 109A) and that synchronization information 115A and encoded video frame portion 117A of video frame portion 151A are available in line buffer 202A, bitstream engine 130B retrieves synchronization information 115A and encoded video frame portion 117A from line buffer 202A. Bitstream engine 130B encodes video frame portion 151B to generate encoded video frame portion 117B. For example, bitstream engine 130B encodes video frame portion 151B (row 0 of video frame 109B) based at least in part on synchronization information 115A (e.g., co-located motion vectors, reference indexes, reference patterns, or a combination thereof) associated with encoding video frame portion 151A (row 0 of video frame 109A).

[0109] In a specific aspect, concurrently with the bitstream engine 130B encoding the video frame portion 151B to generate the encoded video frame portion 117B, the bitstream engine 130A encodes the video frame portion 151C (line 1 of the video frame 109A) to generate the encoded video frame portion 117C. After the availability of the synchronization information 115A and the encoded video frame portion 117A in the line buffer 202A, the video frame portion 151C (line 1 of the video frame 109A) and the video frame portion 151B (line 0 of the video frame 109B) can be encoded concurrently.

[0110] Figure 370 can be Figure 1 Examples of operations associated with processing video using multiple bitstream engines performed at multiple bitstream engines 140, video decoder 122, device 104, system 100, or a combination thereof.

[0111] Similar to how the video encoder 112 dispatches the video frame portions 151 to the multiple bitstream engines 130, the video decoder 122 dispatches the encoded video frame portions 117 to the multiple bitstream engines 140. The video decoder 122 dispatches the encoded video frame portions 117 of the video frame 109A to the bitstream engine 140A, and dispatches the encoded video frame portions 117 of the video frame 109B to the bitstream engine 140B. For example, the video decoder 122 dispatches the encoded video frame portion 117A (corresponding to line 0 of the video frame 109A), the encoded video frame portion 117C (corresponding to line 1 of the video frame 109A), and the encoded video frame portion 117E (corresponding to line 2 of the video frame 109A) to the bitstream engine 140A, and dispatches the encoded video frame portion 117B (corresponding to line 0 of the video frame 109B), the encoded video frame portion 117D (corresponding to line 1 of the video frame 109B), and the encoded video frame portion 117F (corresponding to line 2 of the video frame 109B) to the bitstream engine 140B.

[0112] The bitstream engine 140A generates synchronization information and decoded partial data for each encoded video frame portion of the video frame 109A, and stores the synchronization information and decoded partial data in the line buffer 204A. The bitstream engine 140B retrieves the synchronization information and decoded partial data from the line buffer 204A to generate decoded partial data for the corresponding encoded video frame portion of the video frame 109B. For example, the bitstream engine 140A generates the synchronization information 125A and the decoded partial data 121A based on the partial data 111A of the encoded video frame portion 117A (corresponding to line 0 of the video frame 109A), as shown in FIG. Figure 1 In particular embodiments, the synchronization information 125A is based on (e.g., depends on or indicates) a co-located motion vector, a reference index, a reference pattern, or a combination thereof associated with decoding of the encoded video frame portion 117A (corresponding to line 0 of the video frame 109A). The bitstream engine 140A stores the synchronization information 125A and the decoded portion data 121A in the line buffer 204A of the video encoder 112.

[0113] In response to determining that the encoded video frame portion 117B (corresponding to line 0 of the video frame 109B) has a dependency on the encoded video frame portion 117A (corresponding to line 0 of the video frame 109A) and that the synchronization information 125A and the decoded partial data 121A of the encoded video frame portion 117A are available in the line buffer 204A, the bitstream engine 140B retrieves the synchronization information 125A and the decoded partial data 121A from the line buffer 204A. The bitstream engine 140B decodes the encoded video frame portion 117B to generate the decoded partial data 121B. For example, the bitstream engine 130B decodes the encoded video frame portion 117B (corresponding to line 0 of the video frame 109B) based at least in part on the synchronization information 125A (e.g., a co-located motion vector, a reference index, a reference pattern, or a combination thereof) associated with the decoding of the video frame portion 117A (corresponding to line 0 of the video frame 109A).

[0114] In a specific aspect, concurrently with the bitstream engine 140B decoding the encoded video frame portion 117B to generate the decoded partial data 121B, the bitstream engine 140A decodes the encoded video frame portion 117C (line 1 of the video frame 109A) to generate the decoded partial data 121C. Following the availability of the synchronization information 125A and the decoded partial data 121A in the line buffer 204A, the encoded video frame portion 117C (line 1 of the video frame 109A) and the encoded video frame portion 117B (line 0 of the video frame 109B) can be decoded concurrently.

[0115] The use of multiple bitstream engines enables concurrent processing of video data, thereby reducing processing time and increasing throughput. Exchanging synchronization information between bitstream engines enables dependencies between video data to be taken into account during concurrent processing.

[0116] Reference Figure 4 , a diagram illustrating an example of an illustrated video encoder 112 is shown. The video encoder 112 includes a video data memory 118 coupled to an entropy encoder 456 via a pre-entropy processing unit (PU) 480. The entropy encoder 456 includes a plurality of bitstream engines 130, such as a bitstream engine 130A and a bitstream engine 130B. The video encoder 112 also includes a memory 470 coupled to the plurality of bitstream engines 130. In some examples, the video encoder 112 includes the video data memory 118 and the memory 470. In other examples, the video data memory 118, the memory 470, or both are external to the video encoder 112.

[0117] The video encoder 112 may be configured to utilize techniques for BAC, as described herein. For purposes of illustration, the video encoder 112 will be described in the context of the HEVC codec, but the present disclosure is not limited to other codec standards or methods. For example, the video encoder 112 may operate according to other video codec standards, including H.266. The video encoder 112 may perform intra-frame and inter-frame coding and decoding of CUs within a video frame. Intra-frame coding and decoding relies on spatial prediction to reduce or eliminate spatial redundancy in video data within a given video frame. Inter-frame coding and decoding relies on temporal prediction to reduce or eliminate temporal redundancy between a current frame of a video sequence and a previously coded frame. Intra-frame mode (I mode) may refer to any of several spatially based video compression modes. Inter-frame modes (such as unidirectional prediction (P mode) or bidirectional prediction (B mode)) may refer to any of several temporally based video compression modes.

[0118] The video data memory 118 may be configured to store video data to be encoded by components of the video encoder 112. For example, the video data stored in the video data memory 118 may be received from a video source (e.g., Figure 1 During the encoding process, the video encoder 112 receives a video frame 109 to be encoded and decoded. The video frame 109 can be divided into a plurality of video frame portions 151.

[0119] The pre-entropy processing unit 480 retrieves the video frame portion 151 from the video data memory 118 and generates the partial data 111, as shown in FIG. Figure 5 For example, the pre-entropy processing unit 480 retrieves the video frame portion 151A from the video data memory 118 and generates the partial data 111A (e.g., the quantized transform coefficients 433A and the prediction syntax 435A). As another example, the pre-entropy processing unit 480 retrieves the video frame portion 151B from the video data memory 118 and generates the partial data 111B (e.g., the quantized transform coefficients 433B and the prediction syntax 435B).

[0120] In a specific aspect, the pre-entropy processing unit 480 sequentially retrieves the video frame portion corresponding to the same video frame (e.g., video frame 109A) before retrieving the video frame portion of the next video frame (e.g., video frame 109B). For example, the video encoder 112 encodes the video frame portion 151 of the video frame 109A before encoding the video frame portion 151 of the video frame 109B, as shown in FIG. Figure 2 In this example, the same video frame (eg, video frame 109A) includes video frame portion 151A and video frame portion 151B.

[0121] In a specific aspect, the pre-entropy processing unit 480 retrieves video frame portions corresponding to different video frames. For example, concurrently with encoding the video frame portion 151 of the video frame 109B, the video encoder 112 encodes the video frame portion 151 of the video frame 109A, as shown in FIG. Figure 3 In this example, video frame 109A includes video frame portion 151A, and video frame 109B includes video frame portion 151B.

[0122] In response to determining that the video frame portion 151A is assigned to the bitstream engine 130A, the pre-entropy processing unit 480 provides the partial data 111A corresponding to the video frame portion 151A (e.g., the quantized transform coefficients 433A and the prediction syntax 435A) to the bitstream engine 130A. In response to determining that the video frame portion 151B is assigned to the bitstream engine 130B, the pre-entropy processing unit 480 provides the partial data 111B corresponding to the video frame portion 151B (e.g., the quantized transform coefficients 433B and the prediction syntax 435B) to the bitstream engine 130B.

[0123] The entropy encoder 456 may apply entropy coding (such as CAVLC, BAC, CABAC, PIPE, Golomb coding, Golomb-Rice coding, exponential Golomb coding, syntax-based context adaptive binary arithmetic coding (SBAC), or another entropy coding method) to entropy coding of quantized transform coefficients, prediction syntax, or a combination thereof. To perform CABAC, the entropy encoder 456 may select a context model to apply to a certain context to encode the symbols to be transmitted. The context may involve, for example, whether adjacent values ​​are non-zero. The entropy encoder 456 may also entropy code grammar elements, such as a signal representing a selected transform.

[0124] Each of the plurality of bitstream engines 130 includes an arithmetic encoder 442 coupled to a probability table 440 and a counter table 444, as shown in FIG. Figure 6 As further described. For example, the bitstream engine 130A includes an arithmetic encoder 442A coupled to the probability table 440A and the counter table 444A. As another example, the bitstream engine 130B includes an arithmetic encoder 442B coupled to the probability table 440B and the counter table 444B.

[0125] The memory 470 is configured to store probability data 441 associated with one or more previously encoded video frames. Each of the plurality of bitstream engines 130 is configured to retrieve the probability data 441 from the memory 470 before encoding any video frame portion (e.g., tile) of a subsequent video frame. Each of the plurality of bitstream engines 130 is configured to copy the probability data 441 from the memory 470 to the probability table 440. For example, the bitstream engine 130A and the bitstream engine 130B copy the probability data 441 from the memory 470 to the probability table 440A and the probability table 440B, respectively. Storing the probability data 441 in the probability table 440 is provided as an illustrative example. In other examples, the bitstream engine can store and retrieve the probability data from various types of memory (e.g., registers) or data structures other than tables.

[0126] In a specific embodiment, the bitstream engine 130A and the bitstream engine 130B encode different video frame portions of the same video frame, as shown in FIG. Figure 2 For example, prior to encoding video frame 109B, bitstream engine 130A and bitstream engine 130B retrieve probability data 441 associated with a previous encoding of video frame 109A.

[0127] In a specific embodiment, the bitstream engine 130A and the bitstream engine 130B encode video frame portions of different video frames, as shown in FIG. Figure 3 For example, before encoding video frame 109C and another video frame, bitstream engine 130A and bitstream engine 130B retrieve probability data 441 associated with the previous encoding of video frame 109A and video frame 109B.

[0128] The arithmetic encoder 442A generates the encoded video frame portion 117A by encoding the portion data 111A based on the probability data 441, as shown in FIG. Figure 6 Arithmetic encoder 442A updates counter value 445A of counter table 444A based on encoded video frame portion 117A, as described in reference to Figure 6 Further described.

[0129] The arithmetic encoder 442B generates the encoded video frame portion 117B by encoding the portion data 111B based at least in part on the probability data 441, as shown in FIG. Figure 6As further described. In a specific aspect, the bitstream engine 130A provides the encoded video frame portion 117A, the synchronization information 115A, or both to the bitstream engine 130B. For example, the bitstream engine 130A stores the encoded video frame portion 117A, the synchronization information 115A, or both in the line buffer 202A, and the bitstream engine 130B retrieves the encoded video frame portion 117A, the synchronization information 115A, or both from the line buffer 202A. In a specific embodiment, the synchronization information 115A is based on (e.g., depends on or indicates) a prediction mode (e.g., an intra-frame prediction mode or an inter-frame prediction mode) associated with encoding the video frame portion 151A, encoded and decoded block information, coefficient information, motion vector information, reference index, reference mode, or a combination thereof.

[0130] The bitstream engine 130B generates the encoded video frame portion 117B based on the encoded video frame portion 117A, the synchronization information 115A, the probability data 441, or a combination thereof, as shown in FIG. Figure 6 The arithmetic encoder 442B updates the counter value 445B of the counter table 444B based on the encoded video frame portion 117B, as described in reference to Figure 6 In some examples, the bitstream engine 130A provides the synchronization information 115A, the encoded video frame portion 117A, or both to the bitstream engine 130B. In other examples, the bitstream engine 130B provides the synchronization information, the encoded video frame portion 117B, or both to the bitstream engine 130A, as described in reference to Figure 2 as described.

[0131] In a particular embodiment, one of the plurality of bitstream engines 130 retrieves counter values ​​from the counter tables of the other bitstream engines to generate a merged counter value 447, and updates the probability data 441 in the memory 470 based on the merged counter value 447, as described in more detail below. In a particular embodiment, one of the plurality of bitstream engines 130 retrieves the counter value in response to determining that the last video frame portion of the video frame to be encoded by the bitstream engine has been encoded by the bitstream engine.

[0132] In a specific example, the bitstream engine 130B provides the counter value 445B of the counter table 444B to the bitstream engine 130A. The bitstream engine 130A determines a merged counter value 447 based on the counter value 445A and the counter value 445B (e.g., merged counter value 447=counter value 445A+counter value 445B), and stores the merged counter value 447 as the counter value 445A in the counter table 444A. The bitstream engine 130A updates the probability data 441 based on the merged counter value 447. In a specific aspect, the bitstream engine 130A generates a merged counter value 447 for each syntax element by merging the counter value 445A for the syntax element from the bitstream engine 130A with the counter value 445B for the syntax element from the bitstream engine 130B. In a specific embodiment, the bitstream engine 130A uses the merged counter value of the syntax element to perform a context adaptation process for each syntax element, and updates the probability data 441 during the context adaptation. The bitstream engine 130A stores the updated probability data 441 in the probability table 440A, the memory 470, or both. In another example, the bitstream engine 130A provides the counter value 445A of the counter table 444A to the bitstream engine 130B. The bitstream engine 130B determines the merged counter value 447 based on the counter value 445A and the counter value 445B (e.g., the merged counter value 447=counter value 445A+counter value 445B), and stores the merged counter value 447 as the counter value 445B in the counter table 444B. The bitstream engine 130B determines the updated value of the probability data 441 based on the merged counter value 447. The bitstream engine 130B stores the updated value of the probability data 441 in the probability table 440B, the memory 470, or both.

[0133] After entropy encoding by entropy encoder 456, the resulting encoded video (eg, encoded video frame portion 117) may be transmitted to another device (such as video decoder 122), or archived for subsequent transmission or retrieval.

[0134] Reference Figure 5 , a diagram showing an example of a pre-entropy processing unit 480 is shown. The pre-entropy processing unit 480 includes a mode selection unit 540 coupled to a reference frame memory 564, a prediction unit 570, and a residual generation unit 550. The prediction unit 570 includes an inter-prediction processing unit 548, an intra-prediction processing unit 546, or both. The inter-prediction processing unit 548 includes a motion estimation unit 542, a motion compensation unit 544, or both.

[0135] The residual generation unit 550 is coupled to a quantization unit (QU) 554 via a transform processing unit 552. The transform processing unit 552 applies an actual transform (e.g., a Fourier transform) or a combination of transforms to the residual data block, and in order not to be confused with a block of transform coefficients, it may also be referred to as a transform unit of a CU. For reconstruction of the video frame portion, the pre-entropy processing unit 480 also includes an inverse quantization unit 558 coupled to an adder 562 via an inverse transform processing unit 560. A deblocking filter ( Figure 5 The deblocking filter 562 is used to filter the video frame portion boundary to remove blockiness artifacts from the reconstructed video. If necessary, the deblocking filter will usually filter the output of the adder 562. The adder 562 is coupled to the residual generation unit 550 and the reference frame memory 564.

[0136] The reference frame memory 564 may be a reference picture block memory (decoded picture block buffer) that stores reference video data used for encoding video data by the video encoder 112 (eg, in intra-frame or inter-frame coding mode). Figure 1 Video data memory 118, Figure 4 The memory 470 and the reference frame memory 564 may be formed by any of a variety of memory devices, such as dynamic random access memory (DRAM), including synchronous DRAM (SDRAM), magnetoresistive RAM (MRAM), resistive RAM (RRAM), or other types of memory devices. The video data memory 118, the memory 470, and the reference frame memory 564 may be provided by the same memory device or by separate memory devices. In various examples, the video data memory 118, the memory 470, the reference frame memory 564, or a combination thereof may be on the core block with other components of the video encoder 112, or off the core block relative to these components.

[0137] The inter-prediction processing unit 548 is configured to perform inter-prediction coding of the received video frame portion 151 relative to one or more video frame portions in one or more reference frames to provide temporal compression. The intra-prediction processing unit 546 is configured to perform intra-prediction coding of the received video frame portion 151 relative to one or more adjacent video frame portions in the same frame or slice as the video frame portion 151 to be coded to provide spatial compression.

[0138] The mode selection unit 540 is configured to select one of the coding modes (intra or inter), for example, based on the error (i.e., distortion) results of each mode, and provide the resulting intra-frame or inter-frame predicted video frame portion (e.g., prediction unit) to the residual generation unit 550 (e.g., adder) to generate residual data 551, and to the adder 562 to generate a reconstructed video frame portion 563 for the reference frame. For example, the mode selection unit 540 sends an activation signal to the inter-frame prediction processing unit 548 to generate a predicted video frame portion 545, or sends an activation signal to the intra-frame prediction processing unit 546 to generate a predicted video frame portion 547. The prediction unit 570 provides the predicted video frame portion 545 or the predicted video frame portion 547 as a predicted video frame portion 571 to the residual generation unit 550 and the adder 562. The adder 562 combines the predicted video frame portion 571 with the inverse quantized, inverse transformed data from the inverse transform processing unit 560 for the video frame portion 151 to reconstruct the encoded video frame portion, as described in more detail below. Some video frames may be designated as I-frames, where all video frame portions in an I-frame are encoded in an intra-prediction mode. In some cases, the intra-prediction processing unit 546 may perform intra-prediction encoding of video frame portions in P-frames or B-frames, for example, when the motion search performed by the motion estimation unit 542 does not result in a sufficient prediction for the video frame portion.

[0139] The motion estimation unit 542 and the motion compensation unit 544 may be highly integrated, but are illustrated separately for conceptual purposes. Motion estimation (or motion search) is the process of generating motion vectors, which estimate the motion of a portion of a video frame. For example, a motion vector may indicate the displacement of a prediction unit in a current frame (e.g., a predicted video frame portion) relative to a reference sample of a reference frame (e.g., a reference video frame portion 565). The motion estimation unit 542 calculates the motion vector of a prediction unit of an inter-coded frame by comparing the prediction unit of the inter-coded frame with the reference sample of the reference frame stored in the reference frame memory 564. The reference sample may be a portion of a video frame that is found to closely match a portion of a CU including the prediction unit being coded in terms of pixel differences, which may be determined by sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. The reference sample may appear anywhere within a reference frame or reference slice, and may not necessarily appear at a video frame portion (e.g., codec unit) boundary of a reference frame or slice. In some examples, reference samples may appear at fractional pixel locations.

[0140] The motion estimation unit 542 sends the calculated motion vector and the prediction syntax 435 to the entropy encoder 456 and the motion compensation unit 544. The reference frame portion (e.g., the reference video frame portion 565) identified by the motion vector can be referred to as a reference sample. The motion compensation unit 544 can calculate the prediction value of the prediction unit (e.g., the predicted video frame portion 545) of the current CU by, for example, retrieving the reference sample identified by the motion vector of the prediction unit.

[0141] The intra-frame prediction processing unit 546 can perform intra-frame prediction on the received video frame portion (e.g., video frame portion 151) as an alternative to the inter-frame prediction performed by the inter-frame prediction processing unit 548. Assuming that the encoding order of the video frame portions is from left to right and from top to bottom, the intra-frame prediction processing unit 546 can predict the video frame portion (e.g., predicted video frame portion 547) relative to an adjacent previously encoded video frame portion (e.g., a video frame portion above, to the right, to the left, or to the left of the current video frame portion (e.g., video frame portion 151)).

[0142] The intra-frame prediction processing unit 546 selects an intra-frame prediction mode by, for example, calculating error values ​​for various intra-frame prediction modes and selecting the mode that produces the lowest error value. The directional prediction mode may include a function for combining the values ​​of spatially adjacent pixels and applying the combined values ​​to one or more pixel positions in the prediction unit. Once the values ​​of all pixel positions in the prediction unit have been calculated, the intra-frame prediction processing unit 546 can calculate the error value for the prediction mode based on the pixel difference between the prediction unit (e.g., the predicted video frame portion 547) and the received video frame portion to be encoded (e.g., video frame portion 151). The intra-frame prediction processing unit 546 can continue to test the intra-frame prediction mode until an intra-frame prediction mode that produces an acceptable error value is found. The intra-frame prediction processing unit 546 can then send the prediction unit (e.g., the predicted video frame portion 547) as the predicted video frame portion 571 to the residual generation unit 550.

[0143] The pre-entropy processing unit 480 forms a residual video frame portion (e.g., residual data 551) by subtracting the prediction data (e.g., predicted video frame portion 571) calculated by the inter-frame prediction processing unit 548 or the intra-frame prediction processing unit 546 from the encoded and decoded original video frame portion (e.g., video frame portion 151). The residual generation unit 550 represents one or more components that perform this subtraction operation. The residual data 551 can correspond to a two-dimensional matrix of pixel difference values, where the number of values ​​in the residual data 551 is the same as the number of pixels in the predicted video frame portion 571. The values ​​in the residual data 551 (e.g., residual video frame portion) can correspond to the differences (i.e., errors) between the values ​​of the co-located pixels in the predicted video frame portion 571 and the video frame portion 151. These differences can be based on differences in chrominance or luminance values, depending on the type of video frame portion that is encoded and decoded.

[0144] The transform processing unit 552 may form one or more transform units from the residual data 551. The transform processing unit 552 may select a transform from a plurality of transforms (e.g., a discrete cosine transform (DCT), an integer transform, a wavelet transform, a pattern-dependent non-separable quadratic transform (MDNSST), a signal-dependent transform, a Karhunen-Loeve transform (KLT), or a combination thereof). The transform may be selected based on one or more codec characteristics, such as a video frame portion size, a codec mode, etc. The transform processing unit 552 may then apply the selected transform to the transform unit, generating a video frame portion including a two-dimensional array 553 of transform coefficients. In addition, the transform processing unit 552 may signal a coded video bitstream (e.g., Figure 1 The transform partition selected in the bitstream 103).

[0145] The transform processing unit 552 may send the resulting transform coefficients 553 to the quantization unit 554. The quantization unit 554 may then quantize the transform coefficients 553 to generate quantized transform coefficients 433. A quantization matrix may specify values ​​that may be used with a quantization parameter to determine the amount of quantization to be applied to the corresponding transform coefficients. Figure 4 The entropy encoder 456 of the bitstream engine 130 may perform a scan of the quantized transform coefficients 433 in the matrix according to a scan pattern. In some examples, the entropy encoder 456 performs the scan of the quantized transform coefficients 433. In other examples, other processing units (such as the quantization unit 554) may perform the scan. Once the transform coefficients 433 are scanned from a two-dimensional array to a one-dimensional array, the entropy encoder 456 provides the quantized transform coefficients 433 to the bitstream engine 130, as shown in FIG. Figure 4 as described.

[0146] The inverse quantization unit 558 and the inverse transform processing unit 560 apply inverse quantization and inverse transform, respectively, to reconstruct the residual block into reconstructed residual data 561 in the pixel domain, for example, for subsequent use as a reference block. The motion compensation unit 544 can calculate the reference block by adding the residual block to a prediction block of a frame of the reference frame memory 564. The motion compensation unit 544 can also apply one or more interpolation filters to the reconstructed residual block (e.g., the reconstructed residual data 561) to calculate sub-integer pixel values ​​for motion estimation. The adder 562 adds the reconstructed residual block (e.g., the reconstructed residual data 561) to the predicted video frame portion 571 (e.g., the predicted video frame portion 545 or the predicted video frame portion 547) to generate a reconstructed video block (e.g., the reconstructed video frame portion 563) for storage in the reference frame memory 564. The reconstructed video block (eg, reconstructed video frame portion 563) may be used by the inter-prediction processing unit 548 as a reference block (eg, reference video frame portion 565) to inter-code a block in a subsequent video frame.

[0147] Figure 6 is a diagram of an example of a bitstream engine 130 of a video encoder 112. In a specific aspect, the bitstream engine 130 is Figure 1 In particular aspects, bitstream engine 130 is configured to perform CABAC according to the techniques of this disclosure.

[0148] The bitstream engine 130 includes a binarizer 620 coupled to an arithmetic encoder 442. In a specific aspect, the arithmetic encoder 442 includes a context modeler 622, a conventional encoding engine 624, a bypass encoding engine 626, or a combination thereof. In a specific aspect, the arithmetic encoder 442 is coupled to a RangeLPS table 628, a TransIdxLPS table 630, a probability table 440, a counter table 444, or a combination thereof. In a specific aspect, the RangeLPS table 628 indicates a range of values ​​for the least-probable symbol (LPS) for a given specific probability state. The TransIdxLPS table 630 indicates which probability state is used for the next bin (bin n+1) when the previous bin (bin n) is an LPS.

[0149] In particular aspects, the bitstream engine 130 (e.g., Figure 1The bitstream engine 130B of the embodiment is configured to perform entropy encoding based on the encoded video frame portion, the synchronization information, or both. In this aspect, the bitstream engine 130 (e.g., the bitstream engine 130B) is configured to retrieve the encoded video frame portion (e.g., the encoded video frame portion 117A), the synchronization information (e.g., the synchronization information 115A), or both from the line buffer 202A. In a specific example, the bitstream engine 130 corresponds to Figure 1 The bitstream engine 130A of FIG. 1 corresponds to the bitstream engine 130A of FIG. 1 and outputs the synchronization information 115A to the line buffer 202A. In a specific example, the bitstream engine 130 corresponds to Figure 1 The bitstream engine 130B of the embodiment of the present invention and the synchronization information 115A is retrieved from the line buffer 202A.

[0150] Syntax elements corresponding to the quantized transform coefficients 433 and syntax elements corresponding to the prediction syntax 435 are input to the entropy encoder 456. For example, syntax element 619 is input to the entropy encoder 456. For illustration, syntax element 619 corresponds to a quantized transform coefficient of the quantized transform coefficient 433 or a syntax element of the prediction syntax 435. In a specific example, the synchronization information 115A is based on (e.g., depends on or indicates) syntax elements corresponding to the quantized transform coefficient 433A and syntax elements corresponding to the prediction syntax 435A of the video frame portion 151A.

[0151] If the syntax element 619 is already a binary-valued syntax element (i.e., a syntax element having only values ​​0 and 1), the binarization step may be skipped and the binary-valued syntax element is processed by the arithmetic encoder 442. If the syntax element 619 is a non-binary-valued syntax element (e.g., a syntax element represented by multiple bits, such as a transform coefficient level), the non-binary-valued syntax element is binarized by the binarizer 620. The binarizer 620 performs a mapping of the syntax element 619 (e.g., a non-binary-valued syntax element) to a sequence of binary decisions. These binary decisions are often referred to as "bins". For example, for a transform coefficient level, the value of the level may be decomposed into consecutive bins, each bin indicating whether the absolute value of the coefficient level is greater than a certain value. For example, a bin 0 (sometimes referred to as a significance flag) indicates whether the absolute value of the transform coefficient level is greater than 0. A bin 1 indicates whether the absolute value of the transform coefficient level is greater than 1, and so on. A unique mapping may be made for each non-binary-valued syntax element. In a specific example, synchronization information 115A is based on (eg, depends on or indicates) a binary bit value corresponding to a syntax element associated with video frame portion 151A.

[0152] Each bin produced by the binarizer 620 is fed to the arithmetic encoder 442 (e.g., the binary arithmetic coding side of the entropy encoder 456). That is, for a predetermined set of non-binary valued syntax elements, each bin type (e.g., bin 0) is coded before the next bin type (e.g., bin 1). The coding can be performed in a normal mode or a bypass mode. In the bypass mode, the bypass coding engine 626 performs arithmetic coding using a fixed probability model, such as using Golomb-Rice or exponential Golomb coding. The bypass mode is typically used for more predictable syntax elements. The coding in the normal mode includes performing CABAC. The normal mode CABAC is used to code a bin, where the probability of the bin value is predictable given the previously coded bin value. The probability that a bin is an LPS is determined by the context modeler 622. The context modeler 622 outputs a bin value and a context model (e.g., a probability state σ). The context model may be an initial context model for a series of bins, or may be determined based on a previously coded value of a previously coded bin (e.g., indicated by probability data 441). In a specific aspect, the bitstream engine 130 corresponds to the bitstream engine 130B, and the context modeler 622 determines the context model based on a coded value indicated by the coded video frame portion 117A, the synchronization information 115A, or a combination thereof. As described above, the context modeler 622 may update the state based on whether the previously coded bin is an MPS or an LPS. After the context modeler 622 determines the context model and the probability state σ, the conventional encoding engine 624 performs BAC on the bin value. In a specific example, the synchronization information 115A is based on (e.g., depends on or indicates) the bin value of the syntax element associated with the video frame portion 151A, the context model, or both.

[0153] In general, BAC is a recursive interval subdivision procedure. In the H.264 / AVC video codec standard and the HEVC video codec standard, BAC is used to encode the binary bits of the syntax elements in the CABAC process. Similarly, on the decoder side, CABAC can be used to decode the encoded and decoded representation of the binary bits and retrieve the binary bit values. The syntax elements that can be encoded and decoded using the CABAC process can be any data structure representing video data or how the video data is encoded and decoded, including syntax elements representing encoding and decoding modes, motion vectors, transform coefficients representing residual values, etc. The output of the BAC codec is a binary stream (e.g., an encoded and decoded representation of the binary bits of the syntax elements) representing interval values ​​or pointers to the probability within the final encoded and decoded probability interval. In some examples, the probability interval (also referred to as an arithmetic encoding and decoding interval or interval range) is specified by a range and a lower value. The range is the size of the probability interval. Low is the lower boundary of the encoding and decoding interval. Both the video encoder 112 and the video decoder 122 are configured to determine and update the interval range. The video decoder 122 may receive the coded representation and perform an inverse BAC process to recover the bin values ​​of the syntax elements. The coded representation is the interval value that the video decoder 122 uses to determine the bin values ​​of the syntax elements based on the updated interval ranges.

[0154] The application of arithmetic coding in video coding is described in D. Marpe, H. Schwarz and T. Wiegand, IEEE Trans. Circuits and Systems for Video Technology, Vol. 13, No. 7, July 2003, “Context-Based Adaptive Binary Arithmetic Coding in the H.264 / AVC Video Compression Standard”, which is incorporated herein by reference. Each context (i.e., probability model) in CABAC is represented by a state. Each state (σ) implicitly represents a specific symbol (e.g., a binary bit of a binarized syntax element) as the probability (p) of the least-probable symbol (LPS). σ ). The symbol can be an LPS or a most probable symbol (MPS). The symbol is binary, so the MPS and LPS can be either 0 or 1. The probability of the corresponding context is estimated, and the symbol is entropy encoded and decoded (implicitly) using an arithmetic codec.

[0155] In some examples, the BAC process is performed by a state machine that changes the internal values ​​"range" and "low" depending on the context to be encoded and the value of the binary bit being encoded. Depending on the state of the context (i.e., its probability), the range is divided into rangeMPSσ (the range of the most likely symbols in state σ) and rangeLPSσ (the range of the least likely symbols in state σ). The rangeLPSσ value for a probabilistic state σ can be obtained by multiplication:

[0156] rangeLPSσ =range×p σ ,

[0157] where p σ is the probability of selecting LPS. In some examples, the multiplication to determine rangeLPSσ may be performed directly by video encoder 112 and video decoder 122. In other examples, video encoder 112 and video decoder 122 may store a calculated multiplication table. Video encoder 112 and video decoder 122 may use an index to the table (based on the index of the possible values ​​of range and pσ) to determine the value of rangeLPSσ. The probability of MPS is 1-p σ Equivalently, rangePσ is equal to range minus rangeLPSσ. The BAC iteratively updates the range depending on the state of the context bin to be decoded, the current range, and the value of the bin being decoded (ie, based on whether the bin is equal to the LPS or the MPS).

[0158] According to some exemplary techniques of the present disclosure, the conventional encoding engine 624 uses a TransIdxLPS table 630 including probability states to perform BAC. The TransIdxLPS table 630 is used to determine which probability state is used for the next bin (bin n+1) when the previous bin (bin n) is LPS. The conventional encoding engine 624 can also use the RangeLPS table 628 to determine the range value of the LPS for a given specific probability state. As described above, the conventional encoding engine 624 can determine the product of the probability and the range. The output of the conventional encoding engine 624 (e.g., the encoded and decoded representation of the bins of the syntax element 619) is sent to the bitstream buffer for eventual output to the bitstream 103 as part of the encoded video frame portion 117. For example, once all syntax elements corresponding to the quantized transform coefficients 433 and the prediction syntax 435 are processed by the bitstream engine 130, the output of the bitstream engine 130 is collected in the bitstream buffer and output from the bitstream buffer as the encoded video frame portion 117. In a specific aspect, bitstream engine 130 corresponds to bitstream engine 130A and encoded video frame portion 117 corresponds to encoded video frame portion 117A. In an alternative aspect, bitstream engine 130 corresponds to bitstream engine 130B and encoded video frame portion 117 corresponds to encoded video frame portion 117B.

[0159] In a specific aspect, a bitstream engine 130 (e.g., bitstream engine 130A) is configured to provide an encoded video frame portion, synchronization information, or both to another bitstream engine. In this aspect, a bitstream engine 130 (e.g., bitstream engine 130A) is configured to store an encoded video frame portion 117 (e.g., encoded video frame portion 117B), synchronization information (e.g., synchronization information 115A), or both to a line buffer (e.g., line buffer 202A).

[0160] In some cases, in addition to entropy coding and decoding, the entropy encoder 456 or another unit of the video encoder 112 can be configured to perform other coding and decoding functions. For example, the entropy encoder 456 can be configured to determine the coded block pattern (CBP) values ​​of the CU and the prediction unit. In addition, in some cases, the entropy encoder 456 can perform the length coding and decoding of the running coefficients. In addition, the entropy encoder 456 or other processing units can also encode and decode other data, such as the value of the quantization matrix. In a specific example, the synchronization information 115A is based on (e.g., depends on or indicates) the CBP value associated with the video frame portion 151A, the coefficient information, the value of the quantization matrix, the prediction mode (e.g., intra-frame prediction mode or inter-frame prediction mode), the coded block information, the motion vector information, the reference index, the reference mode, or a combination thereof.

[0161] Reference Figure 7 , a diagram illustrating an example of an illustrated video decoder 122 is shown. The video decoder 122 includes a video data memory 718 coupled to a post-entropy processing unit (PU) 780 via an entropy decoder 756. The entropy decoder 756 includes a plurality of bitstream engines 140, such as a bitstream engine 140A and a bitstream engine 140B. The video decoder 122 also includes a memory 770 coupled to the plurality of bitstream engines 140. In some examples, the video decoder 122 includes the video data memory 718 and the memory 770. In other examples, the video data memory 718, the memory 770, or both are external to the video decoder 122.

[0162] In some examples, video decoder 122 may perform a decoding process that is substantially reciprocal to the encoding process described with respect to video encoder 112 (see Figure 4 ). Video data memory 718 may be configured to store video data (eg, encoded video frame portions 117) to be decoded by components of video decoder 122. The video data stored in video data memory 718 may be obtained, for example, from video encoder 112.

[0163] The entropy decoder 756 or another decoding unit may be configured to use the inverse of the modified mapping described above, for example, for quantization matrix values ​​or other values ​​(such as video data), using a modified mapping of source symbols. Specifically, the entropy decoder 756 may apply a process that is generally inverse to the process used by the entropy encoder 456. The entropy decoder 756 performs an entropy decoding process on the bitstream 103 (e.g., a coded bitstream) to retrieve a one-dimensional array of transform coefficients. The entropy decoding process used depends on the entropy coding (e.g., CABAC, PIPE, or other processes described above) used by the video encoder 112. According to the techniques described in the present disclosure, for example, the entropy decoder 756 may apply a BAC process within a CABAC process, as described in the present disclosure. The entropy encoding process used by the encoder may be signaled in the bitstream 103, or may be a predetermined process.

[0164] The entropy decoder 756 retrieves the encoded video frame portion 117 from the video data memory 718. In a particular aspect, the entropy decoder 756 retrieves the next video frame (e.g., Figure 1 Before the encoded video frame portion of the video frame 109B of the video frame 109B is read, the encoded video frame portion corresponding to the same video frame (for example, Figure 1 For example, before the video decoder 122 decodes the encoded video frame portion 117 of the video frame 109B, as shown in FIG. Figure 2 As depicted, encoded video frame portion 117 corresponding to video frame 109A is decoded. In this example, encoded video frame portion 117A and encoded video frame portion 117B correspond to the same video frame (eg, video frame 109A).

[0165] In a specific aspect, the entropy decoder 756 retrieves the encoded video frame portions corresponding to different video frames. For example, the video decoder 122 concurrently decodes the encoded video frame portion 117 corresponding to the video frame 109A and decodes the encoded video frame portion 117 corresponding to the video frame 109B, as shown in FIG. Figure 3 In this example, encoded video frame portion 117A corresponds to video frame 109A, and encoded video frame portion 117B corresponds to video frame 109B.

[0166] The entropy decoder 756 provides the encoded video frame portion 117 to the plurality of bitstream engines 140. For example, the entropy decoder 756 provides the encoded video frame portion 117A to the bitstream engine 140A in response to determining that the encoded video frame portion 117A is assigned to the bitstream engine 140A. As another example, the entropy decoder 756 provides the encoded video frame portion 117B to the bitstream engine 140B in response to determining that the encoded video frame portion 117B is assigned to the bitstream engine 140B.

[0167] Each of the plurality of bitstream engines 140 includes an arithmetic decoder 742 coupled to a probability table 740 and a counter table 744, as shown in FIG. Fig. 9 As further described. For example, the bitstream engine 140A includes an arithmetic decoder 742A coupled to the probability table 740A and the counter table 744A. As another example, the bitstream engine 140B includes an arithmetic decoder 742B coupled to the probability table 740B and the counter table 744B.

[0168] The memory 770 is configured to store probability data 741 associated with one or more previously decoded video frames. Each of the plurality of bitstream engines 140 is configured to retrieve the probability data 741 from the memory 770 before decoding any encoded video frame portion corresponding to a subsequent video frame. Each of the plurality of bitstream engines 140 is configured to copy the probability data 741 from the memory 770 to the probability table 740. For example, the bitstream engine 140A and the bitstream engine 140B copy the probability data 741 from the memory 770 to the probability table 740A and the probability table 740B, respectively.

[0169] In a specific embodiment, the bitstream engine 140A and the bitstream engine 140B decode different encoded video frame portions corresponding to the same video frame, as shown in FIG. Figure 2 For example, prior to decoding video frame 109B, bitstream engine 140A and bitstream engine 140B retrieve probability data 741 associated with a previous encoding of video frame 109A.

[0170] In a specific embodiment, the bitstream engine 140A and the bitstream engine 140B decode the encoded video frame portions corresponding to different video frames, as shown in FIG. Figure 3 For example, before decoding video frame 109C and another video frame, bitstream engine 140A and bitstream engine 140B retrieve probability data 741 associated with previous decoding of video frame 109A and video frame 109B.

[0171] The arithmetic decoder 742A generates decoded partial data 121A (eg, quantized transform coefficients 733A and prediction syntax 735A) by decoding the encoded video frame portion 117A based on the probability data 741, as shown in FIG. Fig. 9 The arithmetic decoder 742A updates the counter value 745A of the counter table 744A based on the decoded partial data 121A, as shown in FIG. Fig. 9 Further described.

[0172] The arithmetic decoder 742B generates decoded partial data 121B (e.g., quantized transform coefficients 733B and prediction syntax 735B) by decoding the encoded video frame portion 117B based at least in part on the probability data 741, as shown in FIG. Fig. 9 In a specific aspect, the bitstream engine 140A provides the decoded partial data 121A, the synchronization information 125A, or both to the bitstream engine 140B. For example, the bitstream engine 140A stores the decoded partial data 121A, the synchronization information 115A, or both in Figure 2 The bitstream engine 140B retrieves the decoded portion data 121A, the synchronization information 125A, or both from the line buffer 204A. In particular embodiments, the synchronization information 125A is based on (e.g., depends on or indicates) a prediction mode (e.g., an intra-prediction mode or an inter-prediction mode) associated with decoding of the encoded video frame portion 117A, encoded and decoded block information, coefficient information, motion vector information, reference indexes, reference modes, or a combination thereof.

[0173] The bitstream engine 140B generates the decoded partial data 121B based on the decoded partial data 121A, the synchronization information 125A, the probability data 741 or a combination thereof, as shown in FIG. Fig. 9 The arithmetic decoder 742B updates the counter value 745B of the counter table 744B based on the decoded partial data 121B, as shown in FIG. Fig. 9 In some examples, the bitstream engine 140A provides the synchronization information 125A, the decoded portion of the data 121A, or both to the bitstream engine 140B. In other examples, the bitstream engine 140B provides the synchronization information, the decoded portion of the data 121B, or both to the bitstream engine 140A, as described in reference to Figure 2 as described.

[0174] In a particular embodiment, one of the plurality of bitstream engines 140 retrieves counter values ​​from the counter tables of the other bitstream engines to generate a merged counter value 747, and updates the probability data 741 in the memory 770 based on the merged counter value 747. In a particular embodiment, the bitstream engine retrieves the counter value in response to determining that the last encoded video frame portion of the video frame has been decoded.

[0175] In a specific example, the bitstream engine 140B provides the counter value 745B of the counter table 744B to the bitstream engine 140A. The bitstream engine 140A determines the merged counter value 747 based on the counter value 745A and the counter value 745B (e.g., the merged counter value 747=counter value 745A+counter value 745B), and stores the merged counter value 747 as the counter value 745A in the counter table 744A. The bitstream engine 140A updates the probability data 741 based on the merged counter value 747. In a specific aspect, the bitstream engine 140A generates a merged counter value 747 for each syntax element by merging the counter value 745A for the syntax element from the bitstream engine 140A with the counter value 745B for the syntax element from the bitstream engine 140B. In a specific embodiment, the bitstream engine 140A uses the merged counter value of the syntax element to perform a context adaptation process for each syntax element, and updates the probability data 741 during the execution of the context adaptation process. The bitstream engine 140A stores the updated probability data 741 in the probability table 740A, the memory 770, or both. In another example, the bitstream engine 140A provides the counter value 745A of the counter table 744A to the bitstream engine 140B. The bitstream engine 140B determines the merged counter value 747 based on the counter value 745A and the counter value 745B (e.g., the merged counter value 747=counter value 745A+counter value 745B), and stores the merged counter value 747 as the counter value 745B in the counter table 744B. The bitstream engine 140B determines an updated value of the probability data 741 based on the merged counter value 747 and stores the updated value of the probability data 741 in the probability table 740B, the memory 770, or both.

[0176] The video decoder 122 generates a frame prediction syntax 777, frame quantized transform coefficients 779, or a combination thereof corresponding to the video frame 109 based on the decoded portion 121 corresponding to the video frame 109. In a specific aspect, the encoded video frame portion 117A and the encoded video frame portion 117B correspond to video frame portions of the same video frame (e.g., video frame 109A). In this aspect, the video decoder 122 generates the frame prediction syntax 777, frame quantized transform coefficients 779, or a combination thereof corresponding to the video frame 109A based at least in part on the decoded portion data 121A and the decoded portion data 121B. For example, the video decoder 122 determines the frame prediction syntax 777 based on the prediction syntax 735A, the prediction syntax 735B, the prediction syntax of one or more additional video frame portions corresponding to the video frame 109A, or a combination thereof. Video decoder 122 determines frame quantized transform coefficients 779 based on quantized transform coefficients 733A, quantized transform coefficients 733B, quantized transform coefficients of one or more additional video frame portions corresponding to video frame 109A, or a combination thereof.

[0177] In a specific aspect, the encoded video frame portion 117A and the encoded video frame portion 117B correspond to video frame portions of two different video frames. For example, the encoded video frame portion 117A corresponds to the video frame portion 151A of the video frame 109A, and the encoded video frame portion 117B corresponds to the video frame portion 151B of the video frame 109B. In this aspect, the video decoder 122 generates a frame prediction syntax 777, a frame quantized transform coefficient 779, or a combination thereof corresponding to the video frame 109A based at least in part on the decoded portion data 121A. For example, the video decoder 122 determines the frame prediction syntax 777 based on the prediction syntax 735A, the prediction syntax of one or more additional video frame portions corresponding to the video frame 109A, or a combination thereof. The video decoder 122 determines the frame quantized transform coefficient 779 based on the quantized transform coefficient 733A, the quantized transform coefficient of one or more additional video frame portions corresponding to the video frame 109A, or a combination thereof.

[0178] Similarly, the video decoder 122 generates a frame prediction syntax 777, a frame quantized transform coefficient 779, or a combination thereof corresponding to the video frame 109B based at least in part on the decoded portion data 121B. For example, the video decoder 122 determines the frame prediction syntax 777 based on the prediction syntax 735A, the prediction syntax of one or more additional video frame portions corresponding to the video frame 109B, or a combination thereof. The video decoder 122 determines the frame quantized transform coefficient 779 based on the quantized transform coefficient 733B, the quantized transform coefficient of one or more additional video frame portions corresponding to the video frame 109A, or a combination thereof.

[0179] The video decoder 122 provides the frame prediction syntax 777 corresponding to the video frame 109 (e.g., the video frame 109A or the video frame 109B), the frame quantized transform coefficients 779, or a combination thereof to the post-entropy processing unit 780 to generate the reconstructed video frame 129, as shown in FIG. Figure 8 For example, when the video decoder 122 provides the frame prediction syntax 777 corresponding to the video frame 109A, the frame quantized transform coefficients 779, or a combination thereof to the post-entropy processing unit 780, the reconstructed video frame 129 corresponds to the reconstructed video frame 129A. As another example, when the video decoder 122 provides the frame prediction syntax 777 corresponding to the video frame 109B, the frame quantized transform coefficients 779, or a combination thereof to the post-entropy processing unit 780, the reconstructed video frame 129 corresponds to the reconstructed video frame 129B.

[0180] Reference Figure 8 , a diagram showing an example of a post entropy processing unit 780. The post entropy processing unit 780 includes an inverse quantization unit 876 coupled to a reconstruction unit 880 via an inverse transform processing unit 878. The post entropy processing unit 780 also includes a prediction unit 860 coupled to the reconstruction unit 880. The reconstruction unit 880 is coupled to the reference frame memory 128. The prediction unit 860 includes a motion compensation unit 872, an intra prediction processing unit 874, or both.

[0181] The reference frame memory 128 may be a reference picture block memory (decoded picture block buffer) that stores reference video data for decoding of video data by the post-entropy processing unit 780, for example in intra-frame or inter-frame coding mode. Figure 7 The memory 770, the reference frame memory 128, or a combination thereof may be formed by any of a variety of memory devices (such as DRAM (including SDRAM, MRAM, RRAM), or other types of memory devices). The video data memory 718, the memory 770, the reference frame memory 128, or a combination thereof may be provided by the same memory device or by separate memory devices. In various examples, the video data memory 718, the memory 770, the reference frame memory 128, or a combination thereof may be on the core image block with other components of the video decoder 122, or off the core image block relative to these components.

[0182] The entropy decoder 756 (or inverse quantization unit 876) may use the same Figure 4 The entropy encoder 456 (or Figure 5The frame quantized transform coefficients 779 are scanned in a scanning pattern mirrored by the scanning pattern used by the quantization unit 554 of the video decoder. In addition, although shown as separate functional units for ease of illustration, the structures and functions of the entropy decoder 756, the inverse quantization unit 876, and other units of the video decoder 122 may be highly integrated with each other.

[0183] The inverse quantization unit 876 inversely quantizes, i.e., dequantizes, the frame quantized transform coefficients 779 to generate transform coefficients 877. The inverse quantization process may include a conventional process, for example, similar to that proposed for HEVC or defined by the H.264 decoding standard. The inverse quantization process may include using the quantization parameter QP calculated by the video encoder 112 for the CU to determine the degree of quantization, and similarly, the degree of inverse quantization that should be applied. The inverse quantization unit 876 may inversely quantize the frame quantized transform coefficients 779 before or after the coefficients are converted from a one-dimensional array to a two-dimensional array.

[0184] The inverse transform processing unit 878 applies an inverse transform to the transform coefficients 877. In some examples, the inverse transform processing unit 878 may determine the inverse transform based on signaling from the video encoder 112, or by inferring the transform from one or more codec characteristics, such as block size, codec mode, etc. In some examples, the inverse transform processing unit 878 may apply a cascaded inverse transform, where the inverse transform processing unit 878 applies two or more inverse transforms to the transform coefficients 877 of the current block being decoded. Additionally, the inverse transform processing unit 878 may apply the inverse transform to produce a transform unit partition.

[0185] The intra-prediction processing unit 874 may generate prediction data for the current frame (eg, an intra-predicted video frame 875 ) based on the signaled intra-prediction mode.

[0186] Based on the retrieved motion prediction direction, reference frame index, and calculated current motion vector (eg, motion vector copied from a neighboring block according to a merge mode), the motion compensation unit 872 generates a motion compensated frame (eg, inter-predicted video frame 873) for the current frame.

[0187] The motion compensation unit 872 may generate a motion compensated video frame (e.g., an inter-predicted video frame 873), possibly performing interpolation based on an interpolation filter. The frame prediction syntax 777 may include an identifier for an interpolation filter to be used for motion estimation with sub-pixel precision. The motion compensation unit 872 may use the interpolation filter used by the video encoder 112 during encoding of the video frame to calculate interpolated values ​​of sub-integer pixels of the reference video frame 857. The motion compensation unit 872 may determine the interpolation filter used by the video encoder 112 based on the received syntax information (e.g., the frame prediction syntax 777), and use the interpolation filter to generate the inter-predicted video frame 873.

[0188] Additionally, in the HEVC example, the motion compensation unit 872 and the intra prediction processing unit 874 can use some syntax information (e.g., provided by the quadtree) to determine the size of the LCU used to encode the (one or more) frames of the encoded video sequence. The motion compensation unit 872 and the intra prediction processing unit 874 can also use syntax information (e.g., frame prediction syntax 777) to determine the partition information describing how each CU of the frame of the encoded video sequence is partitioned (and similarly, how the sub-CUs are partitioned). The syntax information (e.g., frame prediction syntax 777) can also include a mode indicating how each partition is encoded (e.g., intra-frame or inter-frame prediction, and (for intra-frame prediction) intra-frame prediction coding mode), one or more reference frames for each inter-frame encoded PU (and / or a reference list containing identifiers for reference frames), and other information for decoding the encoded video sequence. The prediction unit 860 provides the intra-frame predicted video frame 875 or the inter-frame predicted video frame 873 as the predicted video frame 861 to the reconstruction unit 880.

[0189] The reconstruction unit 880 (e.g., an adder) combines the residual data 879 with the corresponding predicted video frame 861 to form a decoded video frame (e.g., reconstructed video frame 129). For example, the reconstructed video frame 129 corresponds to the reconstructed video frame 129A or the reconstructed video frame 129B. If necessary, a deblocking filter may also be applied to filter the reconstructed video frame 129 to remove blocking artifacts. The reconstructed video frame 129 is then stored in the reference frame memory 128, which provides a reference video frame (e.g., reference video frame 857) for subsequent motion compensation and also generates a decoded video frame for display on a display device (such as a video frame). Figure 1 The decoded video is presented on a display device 108).

[0190] Fig. 9is a diagram of an example of a bitstream engine 140 of a video decoder 122. In specific examples, bitstream engine 140 is an example of bitstream engine 140A, bitstream engine 140B, or both. In specific aspects, bitstream engine 140 is configured to perform CABAC according to the techniques of this disclosure.

[0191] The bitstream engine 140 includes an arithmetic decoder 742 coupled to an inverse binarizer 930. In a specific aspect, the arithmetic decoder 742 includes a context modeler 920 coupled to the inverse binarizer 930 via a conventional decoding engine 924. The arithmetic decoder 742 also includes a bypass decoding engine 922 coupled to the inverse binarizer 930. In a specific aspect, the arithmetic decoder 742 is coupled to a probability table 740, a RangeLPS table 926, a TransIdxLPS table 928, a counter table 744, or a combination of one or more thereof.

[0192] In particular aspects, the bitstream engine 140 (e.g., Figure 1 The bitstream engine 140B of the embodiment is configured to perform entropy decoding based on the decoded partial data, the synchronization information, or both. In this aspect, the bitstream engine 140 (e.g., the bitstream engine 140B) is configured to retrieve the decoded partial data (e.g., the decoded partial data 121A), the synchronization information (e.g., the synchronization information 125A), or both from the line buffer 204A. In a specific example, the bitstream engine 140 corresponds to Figure 1 The bitstream engine 140A of FIG. 1 corresponds to the bitstream engine 140A of FIG. 1 and outputs the synchronization information 125A to the line buffer 204A. In a specific example, the bitstream engine 140 corresponds to Figure 1 The bitstream engine 140B of the embodiment of the present invention and the synchronization information 125A is retrieved from the line buffer 204A.

[0193] In a specific embodiment, the bitstream engine 140 is used with Figure 6 CABAC is performed in a manner reciprocal to that of the bitstream engine 130 described in . The coded bits of the coded video frame portion 131 from the bitstream 103 are input to the bitstream engine 140. The coded bits are fed to either the context modeler 920 or the bypass decoding engine 922, based on whether the coded bits were entropy coded using the bypass mode or the normal mode. If the coded bits were coded in the bypass mode, the bypass decoding engine 922 uses, for example, Golomb-Rice or exponential Golomb decoding to retrieve the bins of the binary-valued syntax elements or non-binary-valued syntax elements.

[0194] If the coded bits are coded in a conventional mode, the context modeler 920 may determine a probability model for the coded bits based at least in part on the probability data 741. For example, if the bitstream engine 140 corresponds to Figure 1 140B, the context modeler 920 determines the binary bit values ​​and the context model based on the probability data 741, the decoded partial data 121A, the synchronization information 125A, or a combination thereof. In a specific example, the synchronization information 125A is based on (e.g., depends on or indicates) the binary bit values, the probability model, or both associated with the decoding of the encoded video frame portion 117A. The conventional decoding engine 924 can decode the encoded bits to produce the binary bits of the non-binary valued syntax element (or if it is a binary value, the syntax element itself). After the context modeler 920 determines the context model and the probability state σ, the conventional decoding engine 924 performs BAC on the binary bit values.

[0195] According to some exemplary techniques of the present disclosure, the conventional decoding engine 924 performs BAC using a TransIdxLPS table 928 including probability states. TransIdxLPS 928 is used to determine which probability state is used for the next bin (bin n+1) when the previous bin (bin n) is LPS. The conventional decoding engine 924 may also use a RangeLPS table 926 to determine a range value for the LPS given a particular probability state. The conventional decoding engine 924 may determine the product of the probability and the range. After the bin is decoded by the conventional decoding engine 924, the inverse binarizer 930 may perform a reverse mapping to convert the bin back to a value for a non-binary valued syntax element. Fig. 9As shown in , the BAC process may be performed on all bins of the syntax element 919 (e.g., as indicated by the loop over the bins to produce a bin string prior to inverse binarization by the inverse binarizer 930). In a specific aspect, the bitstream engine 140 outputs the syntax element 919 as part of the decoded portion of data 121. In a specific embodiment, the bitstream engine 140 stores the decoded syntax elements of the encoded video frame portion 131 in a decoder buffer and outputs the decoded syntax elements (e.g., including the syntax element 919) as the decoded portion of data 121 when generating all the decoded syntax elements of the encoded video frame portion 131. In a specific aspect, the syntax element 919 corresponds to a quantized transform coefficient of the quantized transform coefficient 733 or a syntax element of the prediction syntax 735. In a specific aspect, the bitstream engine 140 (e.g., the bitstream engine 140A) is configured to store the decoded portion of data 121 (e.g., the decoded portion of data 121A), the synchronization information (e.g., the synchronization information 125A), or both in the line buffer 204 (e.g., the line buffer 204A). In a specific example, the synchronization information 125A is based on (e.g., depends on or indicates) a CBP value associated with the encoded video frame portion 117A, coefficient information, a value of a quantization matrix, a prediction mode (e.g., an intra-frame prediction mode or an inter-frame prediction mode), coded and decoded block information, motion vector information, a reference index, a reference mode, or a combination thereof.

[0196] Fig.10 An implementation 1000 of device 102, device 104, or both is depicted as an integrated circuit 1002 that includes one or more processors 1090. Integrated circuit 1002 also includes a signal input 1004 (such as one or more bus interfaces) to enable input signals 1028 to be received for processing. Integrated circuit 1002 also includes a signal output 1006 (such as a bus interface) to enable output signals 1050 to be sent. One or more processors 1090 include a video encoder 112, a video decoder 122, or both. In a specific aspect, input signal 1028 represents a video frame 109, and output signal 1050 includes a bitstream 103. In a specific aspect, input signal 1028 includes a bitstream 103, and output signal 1050 represents a reconstructed video frame 129. Integrated circuit 1002 enables implementations using multiple bitstream engines as components in a system to process video, such as Fig.11 The depicted mobile phone or tablet, Fig.12 The wearable electronic devices depicted, Fig.13 The camera depicted in Fig.14 A virtual reality headset or augmented reality headset as depicted in Fig.15 or Fig.16The means of transport depicted in .

[0197] Fig.11 An embodiment 1100 is depicted in which the device 102, the device 104, or both include a mobile device 1102 (such as a phone or tablet) as an illustrative, non-limiting example. The mobile device 1102 includes a camera 106, a display device 108 (e.g., a display screen), a memory 1186, or a combination thereof. Components of a processor 1090 including a video encoder 112, a video decoder 122, or both are integrated into the mobile device 1102 and are illustrated using dashed lines to indicate internal components that are generally not visible to a user of the mobile device 1102. In a specific example, the video encoder 112 operates to encode a camera output 107 of the camera 106 using a plurality of bitstream engines 130 to generate a bitstream 103 for transmission to another device. In another example, the video decoder 122 operates to decode a bitstream 103 received from another device using a plurality of bitstream engines 140 to generate a reconstructed video frame 129 for display at the display device 108. In a specific example, the video encoder 112 operates to encode the camera output 107 of the camera 106 using multiple bitstream engines 130 to generate the bitstream 103, and the video decoder 122 operates to decode the bitstream 103 using multiple bitstream engines 140 to generate a reconstructed video frame 129 for display at the display device 108. In a specific aspect, comparing the reconstructed video frame 129 to the camera output 107 can be used to determine the image quality loss (if any) associated with encoding the camera output 107 and decoding the bitstream 103. In a specific aspect, storing the bitstream 103 in the memory 1186 uses less storage space than storing the camera output 107 to the memory 1186. In this aspect, the video encoder 112 stores the bitstream 103 in the memory 1186 and discards (e.g., marks for deletion) the camera output 107, and the video decoder 122 retrieves the bitstream 103 from the memory 1186 to generate the reconstructed video frame 129 for display.

[0198] Fig.12An embodiment 1200 is depicted in which device 102, device 104, or both include a wearable electronic device 1202, which is illustrated as a "smart watch". A camera 106, a display device 108 (e.g., a display screen), a video encoder 112, a video decoder 122, or a combination thereof is integrated into the wearable electronic device 1202. In a specific example, the video encoder 112 operates to encode a camera output 107 of the camera 106 using a plurality of bitstream engines 130 to generate a bitstream 103 for transmission to another device. In another example, the video decoder 122 operates to decode a bitstream 103 received from another device using a plurality of bitstream engines 140 to generate a reconstructed video frame 129 for display at the display device 108.

[0199] In a specific example, the wearable electronic device 1202 includes a tactile device that provides a tactile notification (e.g., vibration) in response to the encoding or decoding of the video. For example, the tactile notification can enable a user to look at the wearable electronic device 1202 to see a display notification indicating that the bitstream 103 is ready to be sent or that the reconstructed video frame 129 is ready to be displayed. Thus, the wearable electronic device 1202 can warn a user with hearing loss or a user wearing headphones that video content is available for transmission or display.

[0200] Fig.13 An implementation 1300 is depicted in which device 102, device 104, or both comprise a portable electronic device corresponding to camera 106. Display device 108 (eg, display screen), video encoder 112, video decoder 122, or a combination thereof are integrated into camera 106.

[0201] In a specific example, the video encoder 112 operates to encode the camera output 107 using multiple bitstream engines 130 to generate a bitstream 103 for transmission to another device. In another example, the video decoder 122 operates to decode the bitstream 103 received from another device using multiple bitstream engines 140 to generate a reconstructed video frame 129 for display at the display device 108.

[0202] Fig.14An implementation 1400 is depicted in which device 102, device 104, or both comprise a portable electronic device corresponding to a virtual reality, augmented reality, or mixed reality headset 1402. Camera 106, display device 108 (e.g., a display screen), video encoder 112, video decoder 122, or a combination thereof are integrated into headset 1402. A visual interface device (e.g., including display device 108) is positioned in front of the user's eyes to enable display of augmented reality images, virtual reality images, or both to the user while wearing headset 1402. In specific aspects, augmented reality images, virtual reality images, or both comprise reconstructed video frames 129.

[0203] Fig.15 An embodiment 1500 is depicted in which device 102, device 104, or both correspond to or are integrated within a vehicle 1502, which is illustrated as a manned or unmanned aerial vehicle (e.g., a birthday package delivery drone). Camera 106, video encoder 112, video decoder 122, or a combination thereof are integrated into vehicle 1502.

[0204] In a specific example, the display device 108 is coupled to the vehicle 1502, and the video decoder 122 operates to decode the bitstream 103 received from another device using the multiple bitstream engines 130 to generate a reconstructed video frame 129 (e.g., a video birthday greeting) for display at the display device 108. In a specific example, the video encoder 112 operates to encode the camera output 107 (e.g., capturing the recipient's reaction) using the multiple bitstream engines 130 to generate a bitstream 103 for transmission to another device (e.g., a device of a user ordering a birthday package delivery).

[0205] Fig.16Another embodiment 1600 is depicted, in which device 102, device 104, or both correspond to traffic 1602, or are integrated into a vehicle 1602, which is illustrated as a car. Vehicle 1602 includes a video encoder 112, a video decoder 122, or both. Vehicle 1602 also includes a camera 106, a display device 108, or both. In some examples, video encoder 112 operates to encode camera output 107 from an internal camera (e.g., camera 106) using multiple bitstream engines 130, for example, to capture a video of a passenger of vehicle 1602, to generate a bitstream 103 for sending to another device. In some examples, video encoder 112 operates to encode camera output 107 from an external camera (e.g., camera 106) using multiple bitstream engines 130, for example, to capture a video of another vehicle or pedestrian, to generate a bitstream 103. In a specific example, the video decoder 122 operates to decode a bitstream 103 received from another device using a plurality of bitstream engines 140 to generate reconstructed video frames 129 (eg, a movie) for display at a display device 108 .

[0206] Reference Fig.17 , shows a specific implementation of a method 1700 for processing video using multiple bitstream engines. In a specific aspect, one or more operations of the method 1700 are performed by the input interface 110, the video encoder 112, the multiple bitstream engines 130, the device 102, Figure 1 The system 100 or at least one of a combination thereof is performed.

[0207] The method 1700 includes, at 1702, receiving a plurality of video frames. For example, Figure 1 The video encoder 112 receives the video frame 109 via the input interface 110, as shown in FIG. Figure 1 as described.

[0208] The method 1700 also includes, at 1704, encoding a first portion of data of a first portion of a first video frame of the plurality of video frames at a first bitstream engine to generate a first encoded portion. For example, Figure 1 The bitstream engine 130A encodes the partial data 111A of the video frame portion 151A of the video frame 109A to generate the encoded video frame portion 117A, as shown in FIG. Figure 1 as described.

[0209] The method 1700 also includes, at 1706, generating synchronization information at the first bitstream engine based on completing encoding of the first portion. For example, Figure 1 The bitstream engine 130A generates synchronization information 115A based on the completion of encoding the video frame portion 151A, as shown in FIG. Figure 1 as described.

[0210] The method 1700 also includes, at 1708, initiating encoding of a second portion of data of a second portion of the particular video frame at a second bitstream engine to generate a second encoded portion. Figure 1 The bitstream engine 130B initiates encoding of the portion of data 111B of the video frame portion 151B to generate an encoded video frame portion 117B, as shown in FIG. Figure 1 The video frame portion 151B is included in the video frame 109A or the video frame 109B, as shown in FIG. Figure 1 The encoding of the video frame portion 151B is based on the synchronization information 115A and the encoded video frame portion 117A.

[0211] Method 1700 enables encoding of video frame portions using multiple bitstream engines that take into account dependencies between video frame portions. Multiple bitstream engines can be used for concurrent encoding to increase bitrate. For example, concurrently with bitstream engine 130B encoding video frame portion 151B, bitstream engine 130A can be used to encode video frame portion 151B. Figure 1 The video frame portion 151C is encoded.

[0212] Fig.17 The method 1700 may be implemented by a field programmable gate array (FPGA) device, an application specific integrated circuit (ASIC), a processing unit such as a central processing unit (CPU), a digital signal processor (DSP), a controller, another hardware device, a firmware device, or any combination thereof. As an example, Fig.17 The method 1700 may be performed by a processor executing instructions, such as reference Fig.19 as described.

[0213] Reference Fig.18 , shows a specific implementation of method 1800 for processing video using multiple bitstream engines. In a specific aspect, one or more operations of method 1800 are performed by Figure 1 The method may be performed by at least one of the receiver 126, the modem 124, the video decoder 122, the plurality of bitstream engines 140, the device 104, the system 100, or a combination thereof.

[0214] The method 1800 includes, at 1802, receiving a bitstream representing a plurality of video frames. For example, Figure 1 The video decoder 122 receives the bitstream 103 representing the video frame 109 via the receiver 126, as shown in FIG. Figure 1 as described.

[0215] The method 1800 also includes, at 1804, decoding a first portion of a first video frame in the plurality of video frames at a first bitstream engine to generate first decoded partial data. For example, Figure 1 The bitstream engine 140A decodes the encoded video frame portion 117A of the video frame 109A to generate decoded partial data 121A, as shown in FIG. Figure 1 as described.

[0216] The method 1800 also includes, at 1806, generating synchronization information at the first bitstream engine based on completing decoding of the first portion. Figure 1 The bitstream engine 140A generates synchronization information 125A based on completing the decoding of the encoded video frame portion 117A, as shown in FIG. Figure 1 as described.

[0217] The method 1800 also includes initiating decoding of a second portion of the particular video frame at a second bitstream engine to generate second decoded partial data at 1808. For example, Figure 1 The bitstream engine 140B initiates decoding of the encoded video frame portion 117B to generate decoded partial data 121B, as shown in FIG. Figure 1 The encoded video frame portion 117B corresponds to the video frame portion 151B included in the video frame 109A or the video frame 109B, as described in reference Figure 1 The decoding of the encoded video frame portion 117B is based on the synchronization information 125A and the decoded partial data 121A.

[0218] Method 1800 enables decoding of coded video frame portions using multiple bitstream engines that take into account dependencies between coded video frame portions. Multiple bitstream engines can be used for concurrent decoding to increase bitrate. For example, concurrently with bitstream engine decoding coded video frame portion 117B, bitstream engine 140A can be used to decode coded video frame portion 117B. Figure 1 The encoded video frame portion 117C is decoded.

[0219] Fig.18 The method 1800 may be implemented by a field programmable gate array (FPGA) device, an application specific integrated circuit (ASIC), a processing unit (such as a central processing unit (CPU)), a digital signal processor (DSP), a controller, another hardware device, a firmware device, or any combination thereof. As an example, Fig.18 The method 1800 may be performed by a processor executing instructions, such as reference Fig.19 as described.

[0220] refer to Fig.19, depicts a block diagram of a specific illustrative implementation of a device, and generally designates the device as 1900. In various implementations, the device 1900 may have more than Fig.19 In an illustrative embodiment, device 1900 may correspond to device 102, device 104, or both. In an illustrative embodiment, device 1900 may execute reference Figure 1-18 One or more operations described.

[0221] In particular embodiments, device 1900 includes a processor 1906 (e.g., a central processing unit (CPU)). Device 1900 may include one or more additional processors 1910 (e.g., one or more DSPs). In particular aspects, Fig.10 The one or more processors 1090 correspond to processor 1906, processor 1910, or a combination thereof. Processor 1910 may include a speech and music coder-decoder (codec) 1908, which includes a speech coder-decoder ("vocoder") encoder 1936, a vocoder decoder 1938, or both. In a specific aspect, processor 1910 includes a video encoder 112, a video decoder 122, or both. In a specific embodiment, speech and music codec 1908 includes a video encoder 112, a video decoder 122, or both.

[0222] The device 1900 may include a memory 1986 and a CODEC 1934. The memory 1986 may include instructions 1956 that may be executed by one or more additional processors 1910 (or processor 1906) to implement the functionality described with reference to the video encoder 112, the video decoder 122, or both. The memory 1986 is configured to store data generated or used to perform one or more operations described herein. In a specific aspect, the memory 1986 includes the video data memory 118, Figure 1 The reference frame memory 128, Figure 2 The line buffer 202A, the line buffer 202B, the line buffer 204A, the line buffer 204B, the probability table 440A, the probability table 440B, the counter table 444A, the counter table 444B, Figure 4 Memory 470, Figure 5 The reference frame memory 564, the probability table 740A, the probability table 740B, the counter table 744A, the counter table 744B, Figure 7 Memory 770, Fig.11 1186 or a combination thereof. The device 1900 may include a modem 1948 coupled to an antenna 1952 via a transceiver 1950. In a specific aspect, the modem 1948 corresponds to Figure 1The modem 114, the modem 124, or both. In a specific aspect, the transceiver 1950 includes the transmitter 116, the receiver 126, or both.

[0223] Device 1900 may include display device 108 coupled to display controller 1926. Speaker 1992 and microphone 1990 may be coupled to CODEC 1934. CODEC 1934 may include digital-to-analog converter (DAC) 1902, analog-to-digital converter (ADC) 1904, or both. In a particular embodiment, CODEC 1934 may receive an analog signal from microphone 1990, convert the analog signal to a digital signal using analog-to-digital converter 1904, and provide the digital signal to voice and music codec 1908. Voice and music codec 1908 may process the digital signal. In a particular embodiment, voice and music codec 1908 may provide the digital signal to CODEC 1934. CODEC 1934 may convert the digital signal to an analog signal using digital-to-analog converter 1902, and may provide the analog signal to speaker 1992.

[0224] The video encoder 112 and the video decoder 122 can each be implemented as any of a variety of suitable encoder or decoder circuits, including fixed-function and / or programmable processing circuits, such as one or more microprocessors, DSPs, ASICs, FPGAs, discrete logic, software, hardware, firmware, or any combination thereof. When the technology is partially implemented as software, the device can store instructions for the software in a suitable non-transitory computer-readable medium and use one or more processors to execute the instructions in hardware to perform the technology of the present disclosure. Each of the video encoder 112 and the video decoder 122 can be included in one or more encoders or decoders, any of which can be integrated as part of the CODEC 1934.

[0225] In particular embodiments, device 1900 may be included in a system in package or system on a block device 1922. In particular embodiments, memory 1986, processor 1906, processor 1910, display controller 1926, CODEC 1934, and modem 1948 are included in a system in package or system on a block device 1922. In particular embodiments, input device 1930, camera 106, and power supply 1944 are coupled to system on a block device 1922. In addition, in particular embodiments, as shown in FIG. Fig.19As illustrated, the display 1928, input device 1930, speaker 1992, microphone 1990, camera 106, antenna 1952, and power supply 1944 are located external to the on-tile system device 1922. In particular embodiments, each of the display 1928, input device 1930, speaker 1992, microphone 1990, camera 106, antenna 1952, and power supply 1944 may be coupled to a component of the on-tile system device 1922, such as an interface (e.g., input interface 110 or output interface 120) or a controller.

[0226] Device 1900 may include a smart speaker, a speaker bar, a mobile communication device, a smart phone, a cellular phone, a laptop, a computer, a tablet, a personal digital assistant, a display device, a television, a game console, a music player, a radio, a digital video player, a digital video disc (DVD) player, a tuner, a camera, a navigation device, a vehicle, a headset, an augmented reality headset, a virtual reality headset, an aircraft, a home automation system, a voice-activated device, a wireless speaker and voice-activated device, a portable electronic device, an automobile, a vehicle, a computing device, a communication device, an Internet of Things (IoT) device, a virtual reality (VR) device, a base station, a mobile device, or any combination thereof.

[0227] In conjunction with the described embodiments, the apparatus includes a component for receiving a plurality of video frames. For example, the component for receiving may correspond to the input interface 110, the video encoder 112, Figure 1 The device 102, the processor 1906, the processor 1910, one or more other circuits or components configured to receive the plurality of video frames, or any combination thereof.

[0228] The apparatus further includes a component for encoding a first portion of data of a first portion of a first video frame in a plurality of video frames at a first bitstream engine to generate a first encoded portion. For example, the component for encoding may correspond to the bitstream engine 130A, the video encoder 112, Figure 1 device 102, processor 1906, processor 1910, one or more other circuits or components configured to encode at the first bitstream engine, or any combination thereof.

[0229] The apparatus further includes a component for generating synchronization information at the first bitstream engine based on completing the encoding of the first portion. For example, the component for generating may correspond to the bitstream engine 130A, the video encoder 112, Figure 1 The device 102, the processor 1906, the processor 1910, one or more other circuits or components configured to generate synchronization information at the first bitstream engine, or any combination thereof.

[0230] The apparatus further includes a component for initiating encoding of a second portion of data of a second portion of a specific video frame at a second bitstream engine to generate a second encoded portion, wherein the encoding of the second portion is based on the synchronization information and the first encoded portion, wherein the specific video frame includes a first video frame or a second video frame in the plurality of video frames. For example, the component for initiating encoding may correspond to the bitstream engine 130B, the video encoder 112, Figure 1 Device 102, processor 1906, processor 1910, one or more other circuits or components configured to initiate encoding in a second bitstream engine, or any combination thereof.

[0231] Also in conjunction with the described embodiments, the apparatus includes means for receiving a bit stream representing a plurality of video frames. For example, the means for receiving may correspond to the receiver 126, the modem 124, the video decoder 122, Figure 1 device 104, processor 1906, processor 1910, modem 1948, transceiver 1950, one or more other circuits or components configured to receive a bit stream, or any combination thereof.

[0232] The apparatus further includes a component for decoding a first portion of a first video frame in a plurality of video frames at a first bitstream engine to generate a first decoded partial data. For example, the component for decoding may correspond to the bitstream engine 140A, the video decoder 122, Figure 1 device 104, processor 1906, processor 1910, one or more other circuits or components configured to decode at the first bitstream engine, or any combination thereof.

[0233] The apparatus further includes a component for generating synchronization information at the first bitstream engine based on completing decoding of the first portion. For example, the component for generating may correspond to the bitstream engine 140A, the video decoder 122, Figure 1 device 104, processor 1906, processor 1910, one or more other circuits or components configured to generate synchronization information at the first bitstream engine, or any combination thereof.

[0234] The apparatus further includes a component for initiating decoding of a second portion of a specific video frame at a second bitstream engine to generate second decoded partial data, wherein the decoding of the second portion is based on the synchronization information and the first decoded partial data, wherein the specific video frame includes a first video frame or a second video frame in a plurality of video frames. For example, the component for initiating decoding at the second bitstream engine may correspond to the bitstream engine 140B, the video decoder 122, Figure 1device 104, processor 1906, processor 1910, one or more other circuits or components configured to initiate decoding at a second bitstream engine, or any combination thereof.

[0235] In some embodiments, a non-transitory computer-readable medium (e.g., a computer-readable storage device such as memory 1986) includes instructions (e.g., instructions 1956) that, when executed by one or more processors (e.g., one or more processors 1910 or processor 1906), cause the one or more processors to receive a plurality of video frames (e.g., video frame 109). When executed by the one or more processors, the instructions further cause the one or more processors to encode a first portion of data (e.g., portion data 111A) of a first portion (e.g., video frame portion 151A) of a first video frame (e.g., video frame 109A) of the plurality of video frames at a first bitstream engine (e.g., bitstream engine 130A) to generate a first encoded portion (e.g., encoded video frame portion 117A). When executed by the one or more processors, the instructions further cause the one or more processors to generate synchronization information (e.g., synchronization information 115A) at the first bitstream engine based on completing encoding of the first portion. When executed by one or more processors, the instructions also cause the one or more processors to initiate encoding of a second portion of data (e.g., portion data 111B) of a second portion (e.g., video frame portion 151B) of a particular video frame (e.g., video frame 109A or video frame 109B) at a second bitstream engine (e.g., bitstream engine 130B) to generate a second encoded portion (e.g., encoded video frame portion 117B), wherein encoding of the second portion is based on synchronization information and the first encoded portion, wherein the particular video frame includes a first video frame or a second video frame (e.g., video frame 109B) among multiple video frames.

[0236] In some embodiments, a non-transitory computer-readable medium (e.g., a computer-readable storage device such as memory 1986) includes instructions (e.g., instructions 1956) that, when executed by one or more processors (e.g., one or more processors 1910 or processor 1906), cause the one or more processors to receive a bitstream (e.g., bitstream 103) representing a plurality of video frames (e.g., video frame 109). When executed by the one or more processors, the instructions further cause the one or more processors to decode a first portion (e.g., encoded video frame portion 117A) of a first video frame (e.g., video frame 109A) of the plurality of video frames at a first bitstream engine (e.g., bitstream engine 140A) to generate a first decoded portion of data (e.g., decoded portion of data 121A). When executed by the one or more processors, the instructions further cause the one or more processors to generate synchronization information (e.g., synchronization information 125A) at the first bitstream engine based on completing decoding of the first portion. When executed by one or more processors, the instructions also cause the one or more processors to initiate decoding of a second portion (e.g., encoded video frame portion 117B) of a particular video frame (e.g., video frame 109A or video frame 109B) at a second bitstream engine (e.g., bitstream engine 140B) to generate second decoded partial data (e.g., decoded partial data 121B), wherein decoding of the second portion is based on synchronization information and the first decoded partial data, wherein the particular video frame includes a first video frame or a second video frame (e.g., video frame 109B) among multiple video frames.

[0237] Specific aspects of the present disclosure are described below in a first set of related clauses:

[0238] According to clause 1, a device includes: a first bitstream engine, which is configured to decode a first portion of a first video frame among multiple video frames to generate first decoded partial data, and which is configured to generate synchronization information based on completing the decoding of the first portion; and a second bitstream engine, which is configured to initiate decoding of a second portion of a specific video frame based on the synchronization information to generate second decoded partial data, wherein the second bitstream engine uses the first decoded partial data during decoding of the second portion of the specific video frame, and wherein the specific video frame includes the first video frame or the second video frame among the multiple video frames.

[0239] Clause 2 includes the apparatus of clause 1, further including an entropy decoder comprising a first bitstream engine and a second bitstream, wherein the first decoded partial data includes a first quantized transform coefficient, wherein the synchronization information includes a first prediction syntax, wherein the second decoded partial data includes a second quantized transform coefficient, wherein the second bitstream engine is configured to generate a second prediction syntax, and wherein the entropy decoder is configured to: provide a frame quantized transform coefficient to an inverse quantization unit, wherein the frame quantized transform coefficient is based on the first quantized transform coefficient and the second quantized transform coefficient; and provide a frame prediction syntax to a motion compensation unit, an intra-frame prediction processing unit, or both, wherein the frame prediction syntax includes a first prediction syntax and a second prediction syntax.

[0240] Item 3 includes the device of Item 2, and further includes: an inverse quantization unit, which is configured to inverse quantize the frame quantization transform coefficients to generate transform coefficients; an inverse transform processing unit, which is configured to apply one or more inverse transforms to the transform coefficients to generate residual data; a motion compensation unit, which is configured to apply motion compensation to the reference frame based on the frame prediction syntax to generate an inter-frame predicted frame; an intra-frame prediction processing unit, which is configured to apply intra-frame prediction to the frame prediction syntax to generate an intra-frame predicted frame; and a reconstruction unit, which is configured to generate a reconstructed frame based on the residual data and the predicted frame, the predicted frame including an inter-frame predicted frame, an intra-frame predicted frame, or both.

[0241] Clause 4 includes a device of any one of clauses 1 to 3, wherein the first bitstream engine includes a first arithmetic decoder configured to: generate first decoded partial data based on first probability data of a first probability table; and update a first counter value of a first counter table based on the first decoded partial data.

[0242] Clause 5 includes the device of clause 4, wherein the second bitstream engine includes a second arithmetic decoder configured to: generate second decoded partial data based on second probability data of a second probability table; and update a second counter value of a second counter table based on the second decoded partial data.

[0243] Item 6 includes the device of Item 5, wherein the second bitstream engine is configured to provide a second counter value to the first bitstream engine, and wherein the first bitstream engine is configured to: update a first counter value of a first counter table based on the second counter value; update first probability data of a first probability table based on the updated first counter value; and store the updated first probability data in a memory.

[0244] Clause 7 includes the apparatus of clause 5 or clause 6, wherein the second bitstream engine is configured to copy the probability data from the memory to the second probability table as the second probability data before decoding any tile of the subsequent video frame.

[0245] Item 8 includes the device of Item 5, wherein the first bitstream engine is configured to provide a first counter value to a second bitstream engine, and wherein the second bitstream engine is configured to: update a second counter value of a second counter table based on the first counter value; update second probability data of a second probability table based on the updated second counter value; and store the updated second probability data in a memory.

[0246] Clause 9 includes the apparatus of clause 5 or clause 8, wherein the first bitstream engine is configured to copy the probability data from the memory to the first probability table as the first probability data before decoding any tile of a subsequent video frame.

[0247] Clause 10 includes the apparatus of any of clauses 1 to 9, wherein the first portion includes at least a first tile of a first video frame and the second portion includes at least a second tile of the first video frame.

[0248] Clause 11 includes the device of clause 10, wherein the first video frame comprises rows and columns of tiles, wherein the first tile is in a first row of the rows and a first column of the columns, wherein the second tile is in a second row of the rows and a first column, and wherein the second row is immediately adjacent to and below the first row of the rows.

[0249] Clause 12 includes the apparatus of clause 11, wherein the first video frame comprises an even count of rows and an odd count of columns.

[0250] Clause 13 includes a device of any of clauses 1 to 12, further comprising a plurality of buffers, wherein the first bitstream engine is configured to store the first decoded partial data and synchronization information in a particular buffer of the plurality of buffers, and wherein the second bitstream engine is configured to retrieve the first decoded partial data and synchronization information from the particular buffer before performing processing on the second portion.

[0251] Clause 14 includes the apparatus of any of clauses 1 to 13, further comprising a plurality of buffers, wherein the second bitstream engine is configured to initiate decoding of the second portion based at least in part on determining that a particular buffer of the plurality of buffers has available data.

[0252] Clause 15 includes the apparatus of any of clauses 1 to 14, wherein the first portion comprises a first block located at a first position of a first video frame, and wherein the second portion comprises a second block located at a first position of a second video frame.

[0253] Clause 16 includes a device of any one of clauses 1 to 15, wherein the second bitstream engine includes a context modeler configured to generate binary bit values ​​and a context model based on probability data and synchronization information, wherein the second bitstream engine is configured to generate a second decoded partial data based on the binary bit values ​​and the context model.

[0254] Clause 17 includes the apparatus of any of clauses 1 to 16, wherein the first bitstream engine is configured to decode a third portion of the first video frame and the second bitstream engine decodes a second portion of the particular video frame.

[0255] Clause 18 includes the apparatus of any of clauses 1 to 16, wherein at least one of the first bitstream engine and the second bitstream engine is integrated into an application specific integrated circuit (ASIC).

[0256] Clause 19 includes the apparatus of any of clauses 1 to 18, further comprising a modem configured to receive a bitstream representing a plurality of video frames.

[0257] Clause 20 includes the apparatus of any one of clauses 1 to 19, further comprising a memory configured to store the first decoded partial data, the synchronization information, the second decoded partial data, or a combination thereof.

[0258] Clause 21 includes the apparatus of any of clauses 1 to 20, further comprising a display device configured to display a first reconstructed video frame corresponding to the first video frame, the first reconstructed video frame being based at least on the first decoded partial data.

[0259] Specific aspects of the present disclosure are described below in a second set of related clauses:

[0260] According to clause 22, a method includes: receiving a bitstream representing multiple video frames at a device; decoding a first portion of a first video frame among the multiple video frames at a first bitstream engine of the device to generate first decoded partial data; generating synchronization information based on completing the decoding of the first portion at the first bitstream engine; and initiating decoding of a second portion of a specific video frame at a second bitstream engine of the device to generate second decoded partial data, the decoding of the second portion being based on the synchronization information and the first decoded partial data, wherein the specific video frame includes the first video frame or the second video frame among the multiple video frames.

[0261] Clause 23 includes the method of clause 22, and further includes: at a first bitstream engine, generating first decoded partial data based on the first probability data of a first probability table; at the first bitstream engine, updating a first counter value of a first counter table based on the first decoded partial data; at a second bitstream engine, generating second decoded partial data based on the second probability data of a second probability table; at the second bitstream engine, updating a second counter value of a second counter table based on the second decoded partial data; at the first bitstream engine, receiving a second counter value from the second bitstream engine; at the first bitstream engine, updating a first counter value of the first counter table based on the second counter value; at the first bitstream engine, updating the first probability data of the first probability table based on the updated first counter value; and the first bitstream engine storing the updated first probability data in a memory.

[0262] Specific aspects of the present disclosure are described below in a third set of related clauses:

[0263] According to clause 24, a device includes: an interface configured to receive multiple video frames; a first bitstream engine configured to encode a first portion of data of a first portion of a first video frame of the multiple video frames to generate a first encoded portion, and configured to generate synchronization information based on completing the encoding of the first portion; and a second bitstream engine configured to initiate encoding of a second portion of data of a second portion of a specific video frame to generate a second encoded portion based on the synchronization information, wherein the second bitstream engine uses the first encoded portion during encoding of the second portion of the specific video frame, and wherein the specific video frame includes the first video frame or the second video frame of the multiple video frames.

[0264] Clause 25 includes the device of clause 24, wherein the first bitstream engine includes a first arithmetic encoder configured to: generate a first encoded portion based on first probability data of a first probability table; and update a first counter value of a first counter table based on the first encoded portion, and wherein the second bitstream engine includes a second arithmetic encoder configured to: generate a second encoded portion based on second probability data of a second probability table; and update a second counter value of a second counter table based on the second encoded portion.

[0265] Item 26 includes the device of Item 25, wherein the second bitstream engine is configured to provide a second counter value to the first bitstream engine, and wherein the first bitstream engine is configured to: update a first counter value of a first counter table based on the second counter value; update first probability data of a first probability table based on the updated first counter value; and store the updated first probability data in a memory.

[0266] Clause 27 includes the apparatus of clause 25 or clause 26, wherein the second bitstream engine is configured to copy the probability data from the memory to the second probability table as the second probability data before encoding any tile of the subsequent video frame.

[0267] Item 28 includes the device of Item 25, wherein the first bitstream engine is configured to provide a first counter value to a second bitstream engine, and wherein the second bitstream engine is configured to: update a second counter value of a second counter table based on the first counter value; update second probability data of a second probability table based on the updated second counter value; and store the updated second probability data in a memory.

[0268] Clause 29 includes a device of any of clauses 24 to 28, further comprising a plurality of buffers, wherein the first bitstream engine is configured to store the first encoded portion and synchronization information in a particular buffer of the plurality of buffers, and wherein the second bitstream engine is configured to retrieve the first encoded portion and synchronization information from the particular buffer before encoding the second portion.

[0269] Clause 30 includes an apparatus of any one of clauses 24 to 29, further comprising an entropy encoder comprising a first bitstream engine and a second bitstream engine, wherein the first portion of data comprises a first quantized transform coefficient, wherein the synchronization information comprises a first prediction syntax, wherein the second portion of data comprises a second quantized transform coefficient, and wherein the entropy encoder is configured to: receive the first quantized transform coefficient and the second quantized transform coefficient from a quantization unit; and receive the first prediction syntax and the second prediction syntax from a motion estimation unit.

[0270] Item 31 includes the device of Item 30, and further includes: a residual generation unit, which is configured to: generate first residual data based on the first part and the first predicted part, the first predicted part is received from the motion estimation unit; and generate second residual data based on the second part and the second predicted part, the second predicted part is received from the motion estimation unit; a transform processing unit, which is configured to: apply one or more transforms to the first residual data to generate a first transform coefficient; and apply one or more transforms to the second residual data to generate a second transform coefficient; and a quantization unit, which is configured to: quantize the first transform coefficient to generate a first quantized transform coefficient; and quantize the second transform coefficient to generate a second quantized transform coefficient.

[0271] Specific aspects of the present disclosure are described below in a fourth set of related clauses:

[0272] Item 32 includes a method comprising: at a device, receiving multiple video frames; at a first bitstream engine of the device, encoding a first portion of data of a first portion of a first video frame of the multiple video frames to generate a first encoded portion; at the first bitstream engine, generating synchronization information based on completing the encoding of the first portion; and at a second bitstream engine of the device, initiating encoding of a second portion of data of a second portion of a specific video frame to generate a second encoded portion, encoding the second portion based on the synchronization information and the first encoded portion, wherein the specific video frame includes the first video frame or the second video frame of the multiple video frames.

[0273] Clause 33 includes the method of clause 32, further comprising encoding a third portion of data of a third portion of the first video frame at the first bitstream engine while encoding the second portion of the particular video frame at the second bitstream engine.

[0274] Specific aspects of the present disclosure are described below in a fifth set of related clauses:

[0275] According to clause 34, a device includes: a component for receiving a bitstream representing multiple video frames; a component for encoding a first portion of a first video frame among the multiple video frames at a first bitstream engine to generate first decoded partial data; a component for generating synchronization information based on completing decoding of the first portion at the first bitstream engine; and a component for initiating decoding of a second portion of a specific video frame at a second bitstream engine to generate second decoded partial data, the decoding of the second portion being based on the synchronization information and the first decoded partial data, wherein the specific video frame includes the first video frame or the second video frame among the multiple video frames.

[0276] Clause 35 includes the apparatus of clause 34, wherein the components for receiving, the components for decoding, the components for generating, and the components for initiating decoding are integrated into at least one device of a smart speaker, a speaker bar, a mobile communication device, a smart phone, a cellular phone, a laptop, a computer, a tablet, a personal digital assistant, a display device, a television, a game console, a music player, a radio, a digital video player, a digital video disc (DVD) player, a tuner, a camera, a navigation device, a vehicle, a headset, an augmented reality headset, a virtual reality headset, an aircraft, a home automation system, a voice-activated device, a wireless speaker and a voice-activated device, a portable electronic device, a car, a vehicle, a computing device, a communication device, an Internet of Things (IoT) device, a virtual reality (VR) device, a base station, or a mobile device.

[0277] Specific aspects of the present disclosure are described below in a sixth set of related clauses:

[0278] According to clause 36, an apparatus includes: a component for receiving a plurality of video frames; a component for encoding a first portion of data of a first portion of a first video frame of the plurality of video frames at a first bitstream engine to generate a first encoded portion; a component for generating synchronization information based on completing the encoding of the first portion at the first bitstream engine; and a component for initiating encoding of a second portion of data of a second portion of a particular video frame at a second bitstream engine of the device to generate a second encoded portion, the encoding of the second portion being based on the synchronization information and the first encoded portion, wherein the particular video frame includes the first video frame or the second video frame of the plurality of video frames.

[0279] Clause 37 includes the apparatus of clause 36, wherein the components for receiving, the components for encoding, the components for generating, and the components for initiating encoding are integrated into at least one device of a smart speaker, a speaker bar, a mobile communication device, a smart phone, a cellular phone, a laptop, a computer, a tablet, a personal digital assistant, a display device, a television, a game console, a music player, a radio, a digital video player, a digital video disc (DVD) player, a tuner, a camera, a navigation device, a vehicle, a headset, an augmented reality headset, a virtual reality headset, an aircraft, a home automation system, a voice-activated device, a wireless speaker and a voice-activated device, a portable electronic device, a car, a vehicle, a computing device, a communication device, an Internet of Things (IoT) device, a virtual reality (VR) device, a base station, or a mobile device.

[0280] It will be further understood by those skilled in the art that the various illustrative logic blocks, configurations, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software run by a processor, or a combination of the two. Various illustrative components, blocks, configurations, modules, circuits, and steps have been generally described above according to their functions. Whether such functions are implemented as hardware or processor executable instructions depends on the specific application and the design constraints imposed on the entire system. Those skilled in the art may implement the described functions in different ways for each specific application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

[0281] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly embodied in hardware, a software module operated by a processor, or a combination of the two. The software module can reside in a random access memory (RAM), a flash memory, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a register, a hard disk, a removable disk, a compact disk read-only memory (CD-ROM), or any other form of non-temporary storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Alternatively, the storage medium can be integrated into a processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a computing device or a user terminal. Alternatively, the processor and the storage medium can reside in a computer device or a user terminal as discrete components.

[0282] The previous description of the disclosed aspects is provided to enable those skilled in the art to make or use the disclosed aspects. Various modifications to these aspects will be apparent to those skilled in the art, and the principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but should have the broadest scope consistent with the principles and novel features defined by the following claims.

Claims

1. A device for video processing, the device comprising: a first bitstream engine configured to decode bits associated with a first portion of a first video frame of a plurality of video frames to generate first decoded portion data, and configured to generate synchronization information based on completing decoding of the bits associated with the first portion; as well as a second bitstream engine configured to initiate decoding of bits associated with a second portion of a particular video frame to generate second decoded partial data based on the synchronization information, wherein the second bitstream engine uses the first decoded partial data during decoding of bits associated with the second portion of the particular video frame, and wherein the particular video frame comprises a second video frame of the plurality of video frames; Wherein, when the first bitstream engine is also configured to decode bits associated with a third portion of the first video frame, the second bitstream engine decodes the bits associated with the second portion of the particular video frame, wherein the third portion of the first video frame is different from the first portion of the first video frame.

2. The apparatus of claim 1 , further comprising an entropy decoder comprising the first bitstream engine and the second bitstream engine, wherein the first decoded partial data comprises first quantized transform coefficients, wherein the synchronization information comprises a first prediction syntax, wherein the second decoded partial data comprises second quantized transform coefficients, wherein the second bitstream engine is configured to generate a second prediction syntax, and wherein the entropy decoder is configured to: providing a frame quantized transform coefficient to an inverse quantization unit, wherein the frame quantized transform coefficient is based on the first quantized transform coefficient and the second quantized transform coefficient; and A frame prediction syntax is provided to a motion compensation unit, an intra prediction processing unit, or both, wherein the frame prediction syntax includes the first prediction syntax and the second prediction syntax.

3. The device according to claim 2, further comprising: The inverse quantization unit is configured to perform inverse quantization on the quantized transform coefficients of the frame to generate transform coefficients; an inverse transform processing unit configured to apply one or more inverse transforms to the transform coefficients to generate residual data; the motion compensation unit configured to apply motion compensation to a reference frame based on the frame prediction syntax to generate an inter-predicted frame; the intra prediction processing unit configured to apply intra prediction to the frame prediction syntax to generate an intra predicted frame; as well as A reconstruction unit is configured to generate a reconstructed frame based on the residual data and a predicted frame, wherein the predicted frame includes the inter-predicted frame, the intra-predicted frame, or both.

4. The apparatus of claim 1 , wherein the first bitstream engine comprises a first arithmetic decoder configured to: generating the first decoded partial data based on first probability data of a first probability table; and A first counter value of a first counter table is updated based on the first decoded partial data.

5. The apparatus of claim 4, wherein the second bitstream engine comprises a second arithmetic decoder, the second arithmetic decoder being configured to: generating the second decoded partial data based on second probability data of a second probability table; and A second counter value of a second counter table is updated based on the second decoded partial data.

6. The apparatus of claim 5, wherein the second bitstream engine is configured to provide the second counter value to the first bitstream engine, and wherein the first bitstream engine is configured to: updating the first counter value of the first counter table based on the second counter value; updating the first probability data of the first probability table based on the updated first counter value; as well as The updated first probability data is stored in the memory.

7. The apparatus of claim 5, wherein the second bitstream engine is configured to copy the probability data from the memory to the second probability table as the second probability data before decoding bits associated with any tile of a subsequent video frame.

8. The apparatus of claim 5, wherein the first bitstream engine is configured to provide the first counter value to the second bitstream engine, and wherein the second bitstream engine is configured to: updating the second counter value of the second counter table based on the first counter value; updating the second probability data of the second probability table based on the updated second counter value; as well as The updated second probability data is stored in the memory.

9. The apparatus of claim 5, wherein the first bitstream engine is configured to copy probability data from a memory to the first probability table as the first probability data before decoding bits associated with any tile of a subsequent video frame.

10. The apparatus of claim 1, wherein the first portion comprises at least a first tile of the first video frame, the second portion comprises at least a second tile of the first video frame, and the third portion comprises at least a third tile of the first video frame.

11. The apparatus of claim 10, wherein the first video frame comprises rows and columns of tiles, wherein the first tile is located in a first row of the rows and a first column of the columns, wherein the second tile is located in a second row of the rows and in the first column, and wherein the second row is adjacent to and below the first row of the rows.

12. The apparatus of claim 11, wherein the first video frame comprises even-numbered rows and odd-numbered columns.

13. The device according to claim 1 further includes multiple buffers, wherein the first bitstream engine is configured to store the first decoded partial data and the synchronization information in a specific buffer of the multiple buffers, and wherein the second bitstream engine is configured to retrieve the first decoded partial data and the synchronization information from the specific buffer before decoding the bits associated with the second portion.

14. The device of claim 1, further comprising a plurality of buffers, wherein the second bitstream engine is configured to initiate decoding of the bits associated with the second portion based at least in part on determining that a particular buffer of the plurality of buffers has available data.

15. The apparatus of claim 1, wherein the first portion comprises a first block located at a first position of the first video frame, wherein the second portion comprises a second block located at the first position of the second video frame, and wherein the third portion comprises a third block located at a second position of the first video frame.

16. The device of claim 1, wherein the second bitstream engine comprises a context modeler configured to generate binary bit values ​​and a context model based on probability data and the synchronization information, wherein the second bitstream engine is configured to generate the second decoded partial data based on the binary bit values ​​and the context model.

17. The device of claim 1, wherein the second bitstream engine is further configured to initiate decoding of bits associated with a fourth portion of the specific video frame based on the first bitstream engine completing decoding of bits associated with a third portion of the first video frame to generate a fourth decoded partial data.

18. The apparatus of claim 1, wherein at least one of the first bitstream engine and the second bitstream engine is integrated into an application specific integrated circuit (ASIC).

19. The apparatus of claim 1, further comprising a modem configured to receive a bitstream representing the plurality of video frames.

20. The apparatus of claim 1, further comprising a memory configured to store the first decoded partial data, the synchronization information, the second decoded partial data, or a combination thereof.

21. The device of claim 1, further comprising a display device configured to display a first reconstructed video frame corresponding to the first video frame, the first reconstructed video frame being based at least on the first decoded partial data.

22. The apparatus of claim 1, wherein the second bitstream engine is configured to decode the bits associated with the second portion of the second video frame and the first bitstream engine decodes the bits associated with the third portion of the first video frame.

23. A method for video processing, comprising: receiving, at a device, a bitstream representing a plurality of video frames; decoding, at a first bitstream engine of the device, bits associated with a first portion of a first video frame of the plurality of video frames to generate first decoded portion data; generating, at the first bitstream engine, synchronization information based on completing decoding of the bits associated with the first portion; initiating, at a second bitstream engine of the device, decoding of bits associated with a second portion of a particular video frame to generate second decoded partial data, the decoding of the bits associated with the second portion being based on the synchronization information and the first decoded partial data, wherein the particular video frame comprises a second video frame of the plurality of video frames; as well as When the second bitstream engine decodes the bits associated with the second portion of the particular video frame, decoding of bits associated with a third portion of the first video frame is initiated at the first bitstream engine, wherein the third portion of the first video frame is different from the first portion of the first video frame.

24. The method according to claim 23, further comprising: At the first bitstream engine, generating the first decoded partial data based on first probability data of a first probability table; At the first bitstream engine, updating a first counter value of a first counter table based on the first decoded partial data; At the second bitstream engine, generating the second decoded partial data based on second probability data of a second probability table; At the second bitstream engine, updating a second counter value of a second counter table based on the second decoded partial data; receiving, at the first bitstream engine, the second counter value from the second bitstream engine; At the first bitstream engine, updating the first counter value of the first counter table based on the second counter value; At the first bitstream engine, updating the first probability data of the first probability table based on the updated first counter value; as well as The first bitstream engine stores the updated first probability data in a memory.

25. A device for video processing, the device comprising: an interface configured to receive a plurality of video frames; a first bitstream engine configured to encode a first portion of data of a first portion of a first video frame of the plurality of video frames to generate bits associated with a first encoded portion, and configured to generate synchronization information based on completing encoding of the first portion; as well as a second bitstream engine configured to initiate encoding of a second portion of data of a second portion of a particular video frame to generate bits associated with a second encoded portion based on the synchronization information, wherein the second bitstream engine uses the first encoded portion during encoding of the second portion of the particular video frame, and wherein the particular video frame comprises a second video frame of the plurality of video frames; The first bitstream engine is further configured to encode bits associated with a third portion of the first video frame, and the second bitstream engine encodes bits associated with the second portion of the particular video frame, wherein the third portion of the first video frame is different from the first portion of the first video frame.

26. The device according to claim 25, The first bitstream engine comprises a first arithmetic encoder, and the first arithmetic encoder is configured to: generating the first encoded portion based on first probability data of a first probability table; as well as updating a first counter value of a first counter table based on the first encoded portion, and The second bitstream engine comprises a second arithmetic encoder, and the second arithmetic encoder is configured to: generating the second encoded portion based on second probability data of a second probability table; as well as A second counter value of a second counter table is updated based on the second encoded portion.

27. The apparatus of claim 26, wherein the second bitstream engine is configured to provide the second counter value to the first bitstream engine, and wherein the first bitstream engine is configured to: updating the first counter value of the first counter table based on the second counter value; updating the first probability data of the first probability table based on the updated first counter value; as well as The updated first probability data is stored in the memory.

28. The apparatus of claim 26, wherein the second bitstream engine is configured to copy the probability data from the memory to the second probability table as the second probability data before encoding bits associated with any tile of a subsequent video frame.

29. The apparatus of claim 26, wherein the first bitstream engine is configured to provide the first counter value to the second bitstream engine, and wherein the second bitstream engine is configured to: updating the second counter value of the second counter table based on the first counter value; updating the second probability data of the second probability table based on the updated second counter value; and The updated second probability data is stored in the memory.

30. A method for video processing, comprising: receiving a plurality of video frames at a device; encoding, at a first bitstream engine of the device, a first portion of data of a first portion of a first video frame of the plurality of video frames to generate bits associated with a first encoded portion; generating, at the first bitstream engine, synchronization information based on completing encoding of the first portion; initiating, at a second bitstream engine of the device, encoding of a second portion of data of a second portion of a particular video frame to generate bits associated with a second encoded portion, the encoding of the second portion being based on the synchronization information and the first encoded portion, wherein the particular video frame comprises a second video frame of the plurality of video frames; as well as While the second bitstream engine is encoding the second portion of the particular video frame, encoding of a third portion of the first video frame is initiated at the first bitstream engine, wherein the third portion of the first video frame is different from the first portion of the first video frame.

31. The method of claim 30, further comprising initiating encoding of a fourth portion of the particular video frame at the second bitstream engine based on the first bitstream engine completing encoding of a third portion of the first video frame.

32. A non-transitory computer readable medium having stored thereon instructions which, when executed, cause one or more processors to perform the method of any of claims 23-24 or 30-31.

Images