Parallel intra coding of sub-partitions

Abstract

A method includes determining a first set of reference pixels from processing of a first sub-partition of a current block and determining a second set of reference pixels from processing of a second sub-partition of the current block. The first sub-partition and the second sub-partition are processed in parallel. The method processes a third sub-partition of the current block based on the first set of reference pixels and processes a fourth sub-partition of the current block based on the second set of reference pixels.

Description

[0001] This application is a divisional application of Patent Application No. 202080039044.9, which is the national stage entry of the PCT application with International Application No. PCT / US2020 / 035360 and International Filing Date of May 29, 2020.

[0002] Cross - reference to related applications

[0003] This disclosure claims priority to U.S. Non - Provisional Application No. 16 / 886,627, filed on May 28, 2020, entitled "PARALLEL INTRA - CODING OF SUB - PARTITIONS", which claims priority to U.S. Provisional Application No. 62 / 854,736, filed on May 30, 2019. All of these applications are hereby incorporated by reference in their entireties for all purposes. Background of the Invention

[0004] In video coding, video content is compressed via a set of processes that include prediction, transformation, quantization, and entropy coding. In intra - prediction, the spatial redundancy between adjacent pixels in a video frame is reduced. For example, the original pixels of a video frame may not be encoded and sent to the decoder. Instead, the predicted pixels can be subtracted from the original pixels to form a residual. The residual is a smaller amount of information to be encoded and sent to the decoder than the original pixels. This reduces the bandwidth used to transmit the encoded bitstream. Brief Description of the Drawings

[0005] Regarding the following discussion and particularly regarding the drawings, it is emphasized that for purposes of illustrative discussion, the details shown represent examples and are presented to provide a description of the principles and conceptual aspects of the present disclosure. In this regard, no attempt is made to show implementation details beyond those necessary for a basic understanding of the present disclosure. The following discussion in conjunction with the drawings will make it clear to those skilled in the art how embodiments in accordance with the present disclosure may be practiced. In the various drawings and the supporting description, like or identical reference numerals may be used to identify or otherwise refer to like or identical elements. In the drawings:

[0006] Figure 1 A simplified system for performing intra - prediction according to some embodiments is depicted.

[0007] Figure 2 An example of a frame for intra - prediction according to some embodiments is depicted.

[0008] Figure 3 An example of a transcoder according to some embodiments is depicted.

[0009] Figure 4 Depicts an example of a decoder according to some embodiments.

[0010] Figure 5 Depicts an example of a sub - partition in a current block according to some embodiments.

[0011] Figure 6 Depicts an example of parallel intra - prediction processing of sub - partitions according to some embodiments.

[0012] Figure 7 Depicts a more detailed flowchart of transcoding of a block according to some embodiments.

[0013] Figure 8 Depicts a more detailed flowchart of decoding of a block according to some embodiments.

[0014] Figure 9 Depicts an example of an encoding system according to some embodiments.

[0015] Figure 10 Depicts an example of a decoding system according to some embodiments. Detailed Description

[0016] Techniques for video coding systems are described herein. In the following description, for purposes of illustration, numerous examples and specific details are set forth in order to provide a thorough understanding of some embodiments. Some embodiments defined by the claims may include some or all of these features of these examples, either alone or in combination with other features described below, and may also include modifications and equivalents of the features and concepts described herein.

[0017] In intra - prediction, a transcoder may subtract predicted pixels from the original pixels of a video frame to reduce the amount of values to be transcoded. One method of intra - prediction coding mode is to divide an intra - coded block into multiple sub - partitions and process the sub - partitions separately. Conventionally, sub - partitions are processed serially, either when transcoding one sub - partition first and then starting the transcoding process of another sub - partition, or when decoding one sub - partition first and then starting the decoding process of another sub - partition. However, some embodiments process groups of sub - partitions in parallel. This increases the speed of the coding process by at least two times compared to serial coding of sub - partitions.

[0018] System

[0019] Figure 1FIG. 0 depicts a simplified system 100 for performing intra prediction according to some embodiments. System 100 transcodes a source video asset, which can be any type of video, such as for a television program, movie, or video clip. The source video may need to be transcoded into one or more formats, such as one or more bitrates. In some embodiments, the server system 102 sends an encoded bitstream to the client 104. For example, the server system 102 can send video to the client 104 for playback.

[0020] The server system 102 includes a transcoder 106 that transcodes video into an encoded bitstream. The transcoder 106 can be a software video processor / transcoder configured on a central processing unit (CPU), a hardware-accelerated video processor / transcoder with a graphics processing unit (GPU), a field-programmable gate array (FPGA), and / or a hardware processor / transcoder implemented in an application-specific integrated circuit (ASIC). Transcoding can be a conversion from one digital format to another. Transcoding can involve decoding the source format and encoding the source video into another digital format, or converting the source content into a video with a specific resolution, frame rate, bitrate, codec, etc. Moreover, encoding can be converting analog source content into a digital format. As used, the term transcoding can include encoding.

[0021] The transcoder 106 includes a transcoder intra prediction block 108 that performs intra prediction. The intra coding mode is used on the transcoder side and the decoder side to compress video by removing the spatial redundancy of adjacent pixels in a video frame. Traditionally, intra coding can intra code a current block with a pixel size equal to NxN. In some embodiments, pixels inside the current block are intra coded using pixels from adjacent blocks of the current block, such as the top, left, and diagonally opposite blocks of the current block.

[0022] The client 104 includes a decoder 112 that decodes the encoded bitstream. During the decoding process, the decoder intra prediction block 110 performs intra prediction to decode a block using pixels from adjacent blocks of the current block, such as the top, left, and diagonally opposite blocks of the current block.

[0023] Figure 2Depicts an example of frame 200 for intra prediction according to some embodiments. Frame 200 can be an image being transcoded or decoded. Frame 200 includes a plurality of blocks, and the current block 202 (#16) is being transcoded or decoded. The top block 204 (#10) above the current block 202 and the left block 206 (#15) to the left of the current block 202 have been decoded. Pixels from the left block 206 and the top block 204 can be used to predict the content of the current block 202. Since these pixels have been decoded, these pixels can be used to predict the content of the current block. For example, adjacent pixels can be similar to some pixels of the current block and provide a good prediction of the current pixels in the current block 202. Pixels from other blocks can also be used, such as pixels from blocks #9, #11, and #21 (diagonally located), if the pixels from these blocks are available (e.g., have been transcoded and then decoded).

[0024] Transcoder

[0025] Figure 3 Depicts an example of transcoder 106 according to some embodiments. Figure 3 Includes a simplified version of the transcoder, and a more detailed version will be described below. Transcoder 106 receives the original pixels from the current block 202 being transcoded. At 302, transcoder 106 then subtracts the predicted pixels from the original pixels to produce a residual, which includes the pixels to be transcoded. For example, as part of the transcoding process, the transform and quantization block 304 performs a transform step and a quantization step (T / Q) on the residual value to further compress the data size to be transcoded. Finally, the entropy coding block 306 processes the resulting data by performing entropy coding on the data. Transcoder 106 packs the data into an encoded bitstream, and the encoded bitstream is sent by transcoder 106 to the decoder side.

[0026] To ensure that the decoder side performs the decoding process with the same behavior as the transcoder side, transcoder 106 includes an inverse quantization and inverse transform block 308 and an intra prediction block that mirrors the decoding pipeline (e.g., transcoder intra prediction block 108). This loop generates reconstructed pixels for the decoded current block 202, and these pixels are used as reference pixels for the next encoded block in the encoding order. For example, at 312, transcoder 106 adds the predicted pixels used for transcoding the current block to the decoded pixels of the residual to generate reference pixels.

[0027] The transcoder intra prediction block 108 generates predicted pixel values using reference pixel values adjacent to the next current block to be transcoded. For example, the transcoder intra prediction block 108 receives reference pixel values and generates predicted pixel values according to the intra mode used. The intra prediction mode may include the direction of prediction, and different reference pixels are used based on the direction used. That is, the horizontal intra prediction direction may use the reference pixels on the left side of the current block, and the vertical intra prediction direction may use the reference pixels on the top of the current block. The direction may also include diagonal directions. The transcoder intra prediction block 108 may generate predicted pixels from the reference pixels and the prediction direction. The transcoder intra prediction block 108 may determine the prediction direction in different ways. In some embodiments, the transcoder intra prediction block 108 may examine all possible intra prediction directions and select the best one based on rules such as rate-distortion optimization metrics. Predicted pixels may be used because some directions may require interpolation of some of the reference pixel values; for example, diagonal directions may interpolate pixel values from one or more reference pixels.

[0028] Decoder

[0029] Figure 4 A more detailed example of the decoder 112 according to some embodiments is depicted. Figure 4 Includes a simplified version of the decoder, and a more detailed version will be described below. On the decoder side, the decoder 112 receives the encoded bitstream, and the entropy decoding block 402 performs entropy decoding. Then, the inverse quantization and inverse transform block 404 performs the inverse quantization step and the inverse transform step (Q -1 / T -1 ). The output of the inverse quantization and inverse transform steps is the residual. At 408, the decoder 112 then combines the residual with the predicted pixels that were used to generate the residual at the transcoder 106 to generate the decoded pixels. The decoded pixels are a reconstructed representation of the original pixels.

[0030] The decoder intra prediction block 406 generates predicted pixels from reference pixels, such as the decoded reference pixels from the decoded current block. The decoded pixels are used as reference pixels for the next encoded block in the decoding order. Another encoded block in the encoding order may use the predicted pixels generated from the reference pixels to decode the encoded bitstream of another block.

[0031] Intra prediction

[0032] During intra prediction, the transcoder side and the decoder side may use the intra sub-partition (ISP) mode to perform intra sub-division. The intra sub-division divides the intra-coded block into multiple sub-partitions (SP), and transcodes or decodes these sub-partitions separately.

[0033] Figure 5Depicts an example of sub - partitions 502 in the current block 202 according to some embodiments. The current block 202 is divided into four sub - partitions: sub - partition #1 502 - 1, sub - partition #2 502 - 2, sub - partition #3 502 - 3, and sub - partition #4 502 - 4. Although four sub - partitions are described, other numbers of sub - partitions may be used, such as two sub - partitions, six sub - partitions, eight sub - partitions, etc. Also, the sub - partitions may be divided horizontally or vertically. In Figure 5 , the current block is divided horizontally; however, the current block may be divided vertically or in other patterns.

[0034] During the transcoding process, the transcoder 106 processes the sub - partitions 502 separately, which means the transcoder 106 first transcodes the pixels of sub - partition 502 - 1, then the pixels of the next sub - partition 502 - 2, and so on. As described above, the transcoder 106 uses reference pixels to transcode each sub - partition 502. For example, reference pixels from already transcoded blocks are used to transcode each sub - partition 502.

[0035] The transcoder 106 selects the reference pixels for each sub - partition 502 in such a way that the transcoder can transcode some sub - partitions in parallel. Conventionally, sub - partitions may be transcoded serially; for example, the transcoder first transcodes sub - partition #1 502 - 1, then uses the reference pixels from sub - partition #1 502 - 1 to transcode sub - partition #2 502 - 2, then uses the reference pixels from sub - partition #2 502 - 2 to transcode sub - partition #3 502 - 3, and finally uses the reference pixels from sub - partition #3 502 - 3 to transcode sub - partition #4 502 - 4. This forces the transcoder to transcode the sub - partitions 502 serially. Instead of transcoding the sub - partitions 502 serially, the transcoder transcodes groups of sub - partitions 502 in parallel, which speeds up the transcoding time of the current block. The transcoder 106 may include multiple transcoding processes capable of performing the transcoding of sub - partitions 502 in parallel, or may use discrete transcoders.

[0036] To transcode sub - partitions 502 in parallel, the reference pixels for some of the sub - partitions 502 are changed from the reference pixels used for transcoding the sub - partitions serially. For example, the reference pixels for sub - partition #1 502 - 1 and sub - partition #3 502 - 3 are shared. Then, the reference pixels for sub - partition #2 502 - 2 are from sub - partition #1 502 - 1, and the reference pixels for sub - partition #4 502 - 4 are from sub - partition #3 502 - 3. This allows the transcoder 106 to transcode sub - partition #1 502 - 1 and sub - partition #3 502 - 3 in parallel. Once the transcoder 106 transcodes and decodes sub - partition #1 502 - 1 and sub - partition #3 502 - 3, then the transcoder 106 can use the reference pixels from sub - partition #1 502 - 1 to transcode sub - partition #2 502 - 2 in parallel and use the reference pixels from sub - partition #3 502 - 3 to transcode sub - partition #4 502 - 4.

[0037] Three sets of reference pixels 504 - 1 to 504 - 3 are shown for four sub - partitions 502. If there are a different number of sub - partitions 502, then there can be a different number of sets of reference pixels 504. However, at least two sub - partitions 502 can share reference pixels to allow parallel processing. Note that the reference pixels shown are drawn for clarity. In some embodiments, the reference pixels can include one or more pixel rows closest to or adjacent to the current block 202. That is, the reference pixels 504 - 1 can include reference pixels from the pixel row closest to the current block 202 - rather than a pixel row farther away compared to the reference pixels 504 - 2 and 504 - 4. The reference pixels are separated to clearly show each reference pixel. However, the reference pixels in the overlapping part on the left side of the block can include the same pixels. For example, the pixels at 506 can be the same, and the pixels at 508 can be the same. Also, the reference pixels used are not limited to the reference pixels shown. Additional reference pixels can be used, such as more reference pixels from the top block or the left - hand block, or reference pixels from blocks other than the top block and the left - hand block (e.g., a block positioned diagonally opposite to the current block).

[0038] When transcoding sub - partition #1 502 - 1 and sub - partition #3 502 - 3, the transcoder 106 uses the reference pixels 504 - 1. The reference pixels 504 - 1 can include reference pixels from the block at the top of the current block 202 and reference pixels from the block to the left of the current block 202. In some embodiments, when transcoding sub - partition #3 502 - 3, the transcoder 106 can preferentially use the intra - frame direction that uses the reference pixels from the block to the left of the current block 202 rather than the reference pixels from the top of the current block 202. The reference pixels to the left of sub - partition #3 502 - 3 are closer and can provide a more accurate prediction.

[0039] Moreover, the reference pixel 504-2 may include reference pixels from sub-partition #2 502-2 and the blocks to the left of the current block 202. The reference pixels for sub-partition #2 502-2 may be the same as the conventional process.

[0040] The reference pixel 504-3 may include reference pixels from sub-partition #3 502-3 and the blocks to the left of the current block 202. The reference pixels for sub-partition #4 502-2 may be the same as the conventional process.

[0041] Note that if the current block 202 is vertically split, the reference pixels used may be slightly different. For example, the current block 202 may be rotated 90 degrees to illustrate the vertical split. However, the concept is the same as sharing the reference pixels of sub-partition #1 502-1 and sub-partition #3 502-3. Then, the reference pixels for sub-partition #2 502-2 are from sub-partition #1 502-1, and the reference pixels for sub-partition #4 502-4 are from sub-partition #3 502-3.

[0042] The decoder 112 may use the sub-partitions in a similar manner during the decoding process, which will be described in more detail below.

[0043] Figure 6 An example of parallel intra prediction processing of sub-partitions 502 according to some embodiments is depicted. The following processing pipeline may be used during the transcoding process or the decoding process. During the transcoding process, the pipeline generates the predicted pixels of a sub-partition, and then after transcoding the sub-partition, generates the decoded pixels of the sub-partition to be used as reference pixels for another sub-partition. If the process is performed serially, the above process of generating the predicted pixels of a sub-partition and then constructing the decoded pixels of the sub-partition is performed serially for each sub-partition. For example, first, the transcoder 106 generates the predicted pixels of sub-partition #1 502-1, generates the residual and transcodes the residual, and then the transcoder 106 constructs the decoded pixels of sub-partition #1 502-1. The transcoder 106 uses these decoded pixels as reference pixels to generate the predicted pixels of sub-partition #2 502-2. After intra encoding sub-partition #2 502-2, the transcoder 106 constructs the decoded pixels of sub-partition #2 502-2. The transcoder 106 uses these decoded pixels to generate the predicted pixels of sub-partition #3 502-3, and the process continues serially until the transcoder 106 transcodes the current block. Therefore, when processing the sub-partitions 502 serially, the transcoder 106 must conventionally complete transcoding a sub-partition and decoding the pixels of the sub-partition before continuing to transcode another sub-partition.

[0044] As described above, transcoder 106 does not need to wait for a previous sub-partition to be transcoded and then decoded before starting the transcoding process for another sub-partition. In Figure 6 when starting the transcoding of the current block, the reference pixels of sub-partition #1 502-1 and sub-partition #3 502-3 are the same, and transcoder 106 starts the transcoding of sub-partition #1 502-1 at 602 and the transcoding of sub-partition #3 502-3 at 610 in parallel. For example, transcoding process #1 600-1 transcodes sub-partition #1 502-1, and transcoding process #2 600-2 transcodes sub-partition #3 502-3.

[0045] At 602, transcoding process #1 600-1 generates predicted pixels for sub-partition #1 502-1 using reference pixels from adjacent blocks, such as the block at the top of the current block 202 and the block to the left of the current block 202. Then, transcoding process #1 600-1 performs the transcoding process for sub-partition #1 502-1. After transcoding the pixels, at 604, transcoding process #1 600-1 then reconstructs (e.g., decodes) the pixels of sub-partition #1 502-1. In parallel, at 610, transcoding process #2 600-2 generates predicted pixels for sub-partition #3 502-3 using the same reference pixels as those used for sub-partition #1 502-1. Then, transcoding process #2 600-2 performs the transcoding process for sub-partition #3 502-3. After transcoding the pixels, at 612, transcoding process #2 600-2 then reconstructs (e.g., decodes) the pixels of sub-partition #3 502-3.

[0046] Sub-partition #2 502-2 uses reference pixels from sub-partition #1 502-1 and from adjacent blocks such as the block to the left. After reconstructing the decoded pixels of sub-partition #1 502-1, at 606, transcoding process #1 600-1 can generate predicted pixels from the reference pixels of sub-partition #2 502-2 and the block to the left. Transcoding process #1 600-1 performs the transcoding process for sub-partition #2 502-2. After transcoding the pixels, at 608, transcoding process #1 600-1 then reconstructs (e.g., decodes) the pixels of sub-partition #2 502-2.

[0047] Moreover, sub - partition #4 502 - 4 uses reference pixels from sub - partition #3 502 - 3. After reconstructing the transcoded pixels of sub - partition #3 502 - 3, at 614, transcoding process #2 600 - 2 can generate predicted pixels from reference pixels from sub - partition #4 502 - 4 and adjacent blocks such as the block on the left. Transcoding process #2 600 - 2 performs the transcoding process for sub - partition #4 502 - 4. After transcoding the pixels, at 616, transcoding process #2 600 - 2 then reconstructs (e.g., decodes) the pixels of sub - partition #4 502 - 4.

[0048] After reconstructing the transcoded pixels of sub - partition #2 502 - 2 and reconstructing the transcoded pixels of sub - partition #4 502 - 4, the transcoding process ends. Thus, due to the parallel processing of two sets of sub - partitions 502, transcoder 106 can perform the transcoding process twice as fast as serial execution. Although two sets are described, the transcoder can divide the current block 202 into different numbers of groups. For example, if eight sub - partitions 502 are used, two groups can be used; however, the transcoder can also divide the sub - partitions 502 into four groups to perform the process four times as fast as serial and twice as fast as using two groups.

[0049] During the decoding process, the sub - partitions 502 can also be decoded in parallel. For example, when decoder 112 starts decoding the current block 202, the reference pixels of sub - partition #1 502 - 1 and sub - partition #3 502 - 3 are the same, and decoder 112 starts decoding sub - partition #1 502 - 1 and sub - partition #3 502 - 3 in parallel. Decoder 112 can also use two decoding processes, decoding process #1 601 - 1 and decoding process #2 600 - 2, similar to transcoder 106.

[0050] Decoding process #1 601 - 1 generates predicted pixels for sub - partition #1 502 - 1 using pixels from adjacent blocks - such as the block at the top of the current block 202 and the block on the left of the current block 202. Then, decoding process #1 601 - 1 performs the decoding process for sub - partition #1 502 - 1 to generate residuals. After decoding the pixels, decoding process #1 601 - 1 then uses the predicted pixels and the residuals to reconstruct the pixels of sub - partition #1 502 - 1. In parallel, decoding process #2 600 - 2 uses the same reference pixels as those used for sub - partition #1 502 - 1 to generate predicted pixels for sub - partition #3 502 - 3. Then, decoding process #2 600 - 2 performs the decoding process for sub - partition #3 502 - 3. After decoding the pixels, decoding process #2 600 - 2 then uses the predicted pixels and the residuals to reconstruct the reference pixels of sub - partition #3 502 - 3.

[0051] Sub - partition #2 502 - 2 uses reference pixels from sub - partition #1 502 - 1 and adjacent blocks such as the left block. After decoding the pixels of sub - partition #1 502 - 1, the decoding process #1 601 - 1 can generate predicted pixels from the decoded sub - partition #1 502 - 1 and reference pixels of adjacent blocks such as the left block. The decoding process #1 601 - 1 performs the decoding process of sub - partition #2 502 - 2 to generate residuals. After decoding the pixels, the decoding process #1 601 - 1 then uses the predicted pixels and residuals to reconstruct the reference pixels of sub - partition #2 502 - 2.

[0052] Sub - partition #3 502 - 3 uses the same reference pixels as sub - partition #1 502 - 1, and the decoding process can be performed in parallel with sub - partition #1 502 - 1. The decoding process #2 600 - 2 performs the decoding process of sub - partition #3 502 - 3 to generate residuals. After decoding the pixels, the decoding process #2 600 - 2 then uses the predicted pixels and residuals to reconstruct the reference pixels of sub - partition #3 502 - 3.

[0053] Sub - partition #4 502 - 4 uses reference pixels from sub - partition #3 502 - 3 and adjacent blocks such as the left block. After reconstructing the reference pixels of sub - partition #3 502 - 3, the decoding process #2 600 - 2 can generate predicted pixels from the reference pixels of sub - partition #3 502 - 3 and the left block. Then, the decoding process #2 600 - 2 performs the decoding process of sub - partition #4 502 - 4 to generate residuals. After decoding the pixels, the decoding process #2 600 - 2 then uses the predicted pixels and residuals to reconstruct the reference pixels of sub - partition #4 502 - 4.

[0054] Figure 7 A more detailed flowchart 700 of transcoding of blocks according to some embodiments is depicted. The above process discusses the generation of predicted pixels. The following flowchart describes the transcoding process in more detail.

[0055] At 702, transcoding process #1 600-1 receives the original pixels of sub-partition #1 502-1 and the predicted pixels of sub-partition #1 502-1. At 704, transcoding process #1 600-1 generates the residual of sub-partition #1 from the original pixels and the predicted pixels. At 706, transcoding process #1 600-1 transcodes the residual of sub-partition #1 502-1 by performing transformation, quantization, and entropy coding. At 708, transcoding process #1 600-1 decodes the residual to generate the decoded pixels of sub-partition #1 502-1. At 710, transcoding process #1 600-1 combines the decoded pixels with the predicted pixels to generate the reference pixels of sub-partition #1 502-1. At 712, the transcoding process for sub-partition #2 502-2 is performed using steps similar to those described in 702 to 710. However, sub-partition #2 uses the reference pixels from sub-partition #1.

[0056] The following operations are performed in parallel. Note that the corresponding steps may not be executed exactly simultaneously. That is, the steps of 702 and 712 do not need to start and end at the same time. Instead, the steps of 702 to 712 are executed in parallel with the steps of 714 to 724, where each step can proceed at its own pace. At 714, transcoding process #2 600-2 receives the original pixels of sub-partition #3 502-3 and the predicted pixels of sub-partition #3 502-3. At 716, transcoding process #2 600-2 generates the residual of sub-partition #1 from the original pixels and the predicted pixels. At 718, transcoding process #2 600-2 transcodes the residual of sub-partition #3 502-3 by performing transformation, quantization, and entropy coding. At 720, transcoding process #2 600-2 decodes the residual to generate the decoded pixels of sub-partition #3 502-3. At 722, transcoding process #2 600-2 combines the decoded pixels with the predicted pixels to generate the reference pixels of sub-partition #3 502-3. At 724, the transcoding process for sub-partition #4 502-4 is performed using steps similar to those described in 714 to 724. However, sub-partition #4 uses the reference pixels from sub-partition #2.

[0057] Figure 8 Depicts a more detailed flowchart 800 of the decoding of a block according to some embodiments. At 802, decoding process #1 601-1 generates predicted pixels from the reference pixels of the previous block. The previous block can be the block above and / or to the left of the current block 202. Moreover, the previous block can be a sub-partition of the current block 202 or another block.

[0058] At 804, decoding process #1 601-1 receives the encoded pixels of sub-partition #1. At 806, decoding process #1 601-1 generates the residuals of sub-partition #1 from the encoded pixels. For example, decoding process #1 601-1 may perform entropy decoding and inverse quantization and transformation. At 808, decoding process #1 601-1 generates the decoded pixels from the predicted pixels and the residuals. For example, decoding process #1 601-1 may combine the decoded pixels with the predicted pixels.

[0059] At 810, decoding process #1 601-1 generates the predicted pixels of sub-partition #1 from the reference pixels. At 812, decoding process #1 601-1 performs the prediction process for sub-partition #2. This process may be similar to steps 802-810. However, sub-partition #2 uses the reference pixels from sub-partition #1.

[0060] The following operations are performed in parallel. Note that the corresponding steps may not be executed exactly simultaneously. That is, the steps of 802 and 812 do not need to start and end at the same time. Instead, the steps of 802 to 812 are executed in parallel with the steps of 814 to 824, where each step can proceed at its own pace. At 814, decoding process #2 601-2 generates the predicted pixels from the reference pixels of the previous block. At 816, decoding process #2 601-2 receives the encoded pixels of sub-partition #3. At 818, decoding process #2 601-2 generates the residuals of sub-partition #3 from the encoded pixels. At 820, decoding process #2 601-2 generates the decoded pixels from the predicted pixels and the residuals. At 822, decoding process #2 601-2 generates the predicted pixels of sub-partition #3 from the reference pixels. At 824, decoding process #2 601-2 performs the prediction process for sub-partition #4. This process may be similar to steps 814-822. However, sub-partition #4 uses the reference pixels from sub-partition #2.

[0061] Conclusion

[0062] Thus, by using the same reference pixels for multiple sub-partitions of a block, the transcoding and decoding processes can be performed more efficiently. This improves the performance of the transcoder and / or decoder.

[0063] System

[0064] Figure 9 An example of a transcoding system according to some embodiments is depicted. A video codec framework includes a set of basic components: block splitting, inter-frame and intra-frame prediction, transformation and quantization, and entropy coding.

[0065] The transcoder 106 receives video frames, which are first divided into non-overlapping coding blocks for further processing. To cope with different video content characteristics, complex regions will be covered by partitions with smaller sizes, while simple regions will be covered by larger partitions. Multiple block patterns and shapes can be used together. For example, the quadtree pattern, the ternary tree pattern, and the binary tree pattern can be used together, and square blocks and rectangular blocks can also be used together.

[0066] Prediction is used to remove redundancy in the video signal. By subtracting the predicted pixel value from the pixel being processed, the amplitude of the residual signal can be significantly reduced, thereby reducing the size of the resulting bitstream. The intra-prediction block 910 using reference pixels in the current frame aims to reduce spatial redundancy within the frame. The inter-prediction block 912 using reference pixels from adjacent frames attempts to remove temporal redundancy between frames. The motion estimation and compensation block 916 can be a sub-module of inter-prediction on the transcoder side, which captures the motion trajectory of objects between adjacent frames and generates reference pixels for inter-prediction.

[0067] The transform and quantization block 904 uses the residual pixels after intra or inter prediction. The transform and quantization block 904 performs a transform operation that represents the residual signal in the frequency domain. Considering that the human visual system is more sensitive to the low-frequency components of the video signal than the high-frequency components, quantization is designed to further compress the residual signal by reducing the precision of the high-frequency signals.

[0068] To avoid out-of-sync problems between the transcoder 106 and the decoder 112, the transcoder 106 includes a decoding module to ensure that both the transcoder 106 and the decoder 112 use the same mathematical processing. Thus, the inverse transform and inverse quantization block 908 is similar to the same block on the decoder side. The inverse transform and inverse quantization block 908 uses intra and inter prediction to reconstruct pixels.

[0069] The in-loop filter 914 removes any visual artifacts introduced by the above processes. Various filtering methods are applied to the reconstructed frames in a cascaded manner to reduce different artifacts, including but not limited to block artifacts, mosquito artifacts, banding effects, etc.

[0070] The entropy coding module 906 can further compress the bitstream using model-based methods. The transcoder 106 sends the resulting encoded bitstream to the decoder 112 via a network or other types of media.

[0071] Figure 10Illustrates an example of a decoding system according to some embodiments. The decoder 112 receives an encoded bitstream and inputs it into the entropy decoding block 1002 to recover the information required for the decoding process. As described above, the decoded frame can be decoded by using the inverse transform and inverse quantization block 1004, the intra prediction block 1006 or the inter prediction block 1008, the motion compensation block 1010, and the in-loop filtering block 1012 in the same manner as constructing the decoded frame.

[0072] Example embodiment

[0073] In some embodiments, a method includes: receiving, by a computing device, a current block to be encoded; dividing, by the computing device, the current block into a plurality of sub-partitions; determining, by the computing device, a first set of reference pixels; processing, by the computing device in parallel, a first sub-partition among the plurality of sub-partitions using the first set of reference pixels and processing a second sub-partition among the plurality of sub-partitions using the first set of reference pixels; and processing, by the computing device, a third sub-partition among the plurality of sub-partitions based on a second set of reference pixels determined from the processing of the first sub-partition, and processing a fourth sub-partition among the plurality of sub-partitions based on a third set of reference pixels determined from the processing of the second sub-partition.

[0074] In some embodiments, the first set of reference pixels includes pixels from adjacent blocks of the current block in a frame.

[0075] In some embodiments, the first set of reference pixels includes pixels from a left block located to the left of the current block and a top block located above the current block in a frame.

[0076] In some embodiments, the first sub-partition is located at a first position along an edge of the frame, and the second sub-partition is not adjacent to the first sub-partition.

[0077] In some embodiments, the third sub-partition is located between the first sub-partition and the second sub-partition.

[0078] In some embodiments, the fourth sub-partition is adjacent to the second sub-partition.

[0079] In some embodiments, the first set of reference pixels is generated from decoded pixels of another block different from the current block.

[0080] In some embodiments, the second set of reference pixels is generated from decoded pixels of the first sub-partition, and the third set of reference pixels is generated from decoded pixels of the second sub-partition.

[0081] In some embodiments, the second set of reference pixels is generated from a first residual of the first sub-partition, and the third set of reference pixels is generated from a second residual of the second sub-partition.

[0082] In some embodiments, processing a first sub - partition using a first set of reference pixels and processing a second sub - partition using the first set of reference pixels in parallel includes: generating a first predicted pixel from the first set of reference pixels; transcoding the first sub - partition using the first predicted pixel; generating a second predicted pixel from a second set of reference pixels; and transcoding the second sub - partition using the second predicted pixel.

[0083] In some embodiments, processing a third sub - partition among multiple sub - partitions based on a second set of reference pixels determined from the processing of the first sub - partition and processing a fourth sub - partition among multiple sub - partitions based on a third set of reference pixels determined from the processing of the second sub - partition includes: generating a third predicted pixel from the second set of reference pixels; transcoding the third sub - partition using the third predicted pixel; generating a fourth predicted pixel from the third set of reference pixels; and transcoding the fourth sub - partition using the fourth predicted pixel.

[0084] In some embodiments, the second set of reference pixels is generated from first decoded pixels and the first predicted pixel of the first sub - partition, and the third set of reference pixels is generated from second decoded pixels and the second predicted pixel of the second sub - partition.

[0085] In some embodiments, processing a first sub - partition using a first set of reference pixels and processing a second sub - partition using the first set of reference pixels in parallel includes: decoding the first sub - partition; generating a first predicted pixel from the first set of reference pixels; generating a second set of reference pixels from the decoded first sub - partition and the first predicted pixel; decoding the second sub - partition; generating a second predicted pixel from the second set of reference pixels; and generating a third set of reference pixels from the decoded second sub - partition and the second predicted pixel.

[0086] In some embodiments, a non - transitory computer - readable storage medium contains instructions that, when executed, control a computer system to be operable to: receive a current block to be encoded; divide the current block into multiple sub - partitions; determine a first set of reference pixels; process a first sub - partition among multiple sub - partitions using the first set of reference pixels and process a second sub - partition among multiple sub - partitions using the first set of reference pixels in parallel; and process a third sub - partition among multiple sub - partitions based on a second set of reference pixels determined from the processing of the first sub - partition, and process a fourth sub - partition among multiple sub - partitions based on a third set of reference pixels determined from the processing of the second sub - partition.

[0087] In some embodiments, the first set of reference pixels includes pixels from adjacent blocks of the current block in a frame.

[0088] In some embodiments, processing a first sub-division using a first set of reference pixels and processing a second sub-division using the first set of reference pixels in parallel includes: generating a first predicted pixel from the first set of reference pixels; transcoding the first sub-division using the first predicted pixel; generating a second predicted pixel from a second set of reference pixels; and transcoding the second sub-division using the second predicted pixel.

[0089] In some embodiments, processing a third sub-division among a plurality of sub-divisions based on a second set of reference pixels determined from processing of the first sub-division and processing a fourth sub-division among the plurality of sub-divisions based on a third set of reference pixels determined from processing of the second sub-division includes: generating a third predicted pixel from the second set of reference pixels; transcoding the third sub-division using the third predicted pixel; generating a fourth predicted pixel from the third set of reference pixels; and transcoding the fourth sub-division using the fourth predicted pixel.

[0090] In some embodiments, the second set of reference pixels is generated from first decoded pixels and the first predicted pixel of the first sub-division, and the third set of reference pixels is generated from second decoded pixels and the second predicted pixel of the second sub-division.

[0091] In some embodiments, processing a first sub-division using a first set of reference pixels and processing a second sub-division using the first set of reference pixels in parallel includes: decoding the first sub-division; generating a first predicted pixel from the first set of reference pixels; generating a second set of reference pixels from the decoded first sub-division and the first predicted pixel; decoding the second sub-division; generating a second predicted pixel from the second set of reference pixels; and generating a third set of reference pixels from the decoded second sub-division and the second predicted pixel.

[0092] In some embodiments, a device includes: one or more computer processors; and a non-transitory computer-readable storage medium including instructions that, when executed, control the one or more computer processors to be operable to: receive a current block to be encoded; divide the current block into a plurality of sub-divisions; determine a first set of reference pixels; process a first sub-division among the plurality of sub-divisions using the first set of reference pixels and process a second sub-division among the plurality of sub-divisions using the first set of reference pixels in parallel; and process a third sub-division among the plurality of sub-divisions based on a second set of reference pixels determined from processing of the first sub-division and process a fourth sub-division among the plurality of sub-divisions based on a third set of reference pixels determined from processing of the second sub-division.

[0093] Some embodiments may be implemented on a non - transitory computer - readable storage medium for use by or in connection with an instruction - execution system, apparatus, system, or machine. The computer - readable storage medium contains instructions for controlling a computer system to execute the methods described by some embodiments. The computer system may include one or more computing devices. When executed by one or more computer processors, the instructions may be configured to or operative to perform the operations described in some embodiments.

[0094] As used in this description and throughout the appended claims, the singular forms, "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Also, as used in this description and throughout the appended claims, the meaning of "in" includes "in" and "on" unless the context clearly dictates otherwise.

[0095] The above description illustrates various embodiments and examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be considered the only embodiments, and are presented to illustrate the flexibility and advantages of some embodiments as defined by the appended claims. Based on the above disclosure and the appended claims, other arrangements, embodiments, implementations, and equivalents may be employed without departing from the scope of the invention as defined by the claims.

Claims

1. A method, comprising: determining, by a computing device, a first set of reference pixels from processing of a first sub-division of a current block; determining, by the computing device, a second set of reference pixels from processing of a second sub-division of the current block, wherein the first sub-division and the second sub-division are processed in parallel; and processing, by the computing device, a third sub-division of the current block based on the first set of reference pixels; and processing, by the computing device, a fourth sub-division of the current block based on the second set of reference pixels.

2. The method according to claim 1, further comprising: processing the first sub-division of the current block based on a third set of reference pixels; and processing the second sub-division of the current block based on the third set of reference pixels.

3. The method according to claim 2, wherein the first set of reference pixels, the second set of reference pixels, and the third set of reference pixels include different first partial pixels and identical second partial pixels.

4. The method according to claim 2, wherein the third sub-division and the fourth sub-division are processed in parallel after processing the first sub-division and the second sub-division.

5. The method according to claim 2, wherein processing the first sub-division and processing the second sub-division include: transcoding the first sub-division; decoding the first sub-division to generate first decoded pixels of the first sub-division; generating the first set of reference pixels from the first decoded pixels of the first sub-division; transcoding the second sub-division; decoding the second sub-division to generate second decoded pixels of the second sub-division; generating the second set of reference pixels from the second decoded pixels of the second sub-division.

6. The method according to claim 2, wherein processing the first sub-division and processing the second sub-division include: decoding the first sub-division to generate first decoded pixels of the first sub-division; generating the first set of reference pixels from the first decoded pixels of the first sub-division; decoding the second sub-division to generate second decoded pixels of the second sub-division; generating the second set of reference pixels from the second decoded pixels of the second sub-division.

7. The method according to claim 1, wherein the first sub-division is located at a first position of the current block, and the second sub-division is not adjacent to the first sub-division.

8. The method according to claim 1, wherein the first set of reference pixels includes pixels adjacent to the first sub-division.

9. The method according to claim 1, wherein the first set of reference pixels includes pixels adjacent to the fourth sub-division.

10. The method according to claim 1, wherein the first set of reference pixels includes pixels adjacent to the second sub-division and the third sub-division.

11. The method according to claim 1, wherein the third sub-division and the fourth sub-division are located between the first sub-division and the second sub-division.

12. The method according to claim 1, wherein Processing the third sub - partition and processing the fourth sub - partition include: Encoding the third sub - partition and the fourth sub - partition.

13. The method according to claim 1, wherein, Processing the third sub - partition and processing the fourth sub - partition include: Decoding the third sub - partition and the fourth sub - partition.

14. The method according to claim 1, wherein: The first set of reference pixels is adjacent to the third sub - partition, and The second set of reference pixels is adjacent to the fourth sub - partition.

15. The method according to claim 1, wherein: The first set of reference pixels and the second set of reference pixels include different first - part pixels and the same second - part pixels.

16. The method according to claim 1, wherein: The first set of reference pixels and the second set of reference pixels are selected to process the third sub - partition and the fourth sub - partition in parallel.

17. A non - transitory computer - readable storage medium containing instructions that, when executed, control a computer system to operate as: Determine a first set of reference pixels from the processing of a first sub - partition of a current block; Determine a second set of reference pixels from the processing of a second sub - partition of the current block, wherein the first sub - partition and the second sub - partition are processed in parallel; and Process a third sub - partition of the current block based on the first set of reference pixels; and Process a fourth sub - partition of the current block based on the second set of reference pixels.

18. The non - transitory computer - readable storage medium according to claim 17, further operates as: Process the first sub - partition of the current block based on a third set of reference pixels; and Process the second sub - partition of the current block based on the third set of reference pixels.

19. The non - transitory computer - readable storage medium according to claim 17, wherein, The first set of reference pixels, the second set of reference pixels, and the third set of reference pixels include different first - part pixels and the same second - part pixels.

20. An apparatus, comprising: One or more computer processors; and A non - transitory computer - readable storage medium including instructions that, when executed, control the one or more computer processors to operate as: Determine a first set of reference pixels from the processing of a first sub - partition of a current block; Determine a second set of reference pixels from the processing of a second sub - partition of the current block, wherein the first sub - partition and the second sub - partition are processed in parallel; and Process a third sub - partition of the current block based on the first set of reference pixels; and Process a fourth sub - partition of the current block based on the second set of reference pixels.

Images