Parallel intra coding of subpartitions

Abstract

A method includes receiving a current block to be encoded, and dividing the current block into a plurality of sub-partitions. Determining a first set of reference pixels. The method processes a first sub-partition of the plurality of sub-partitions in parallel using the first set of reference pixels and processes a second sub-partition of the plurality of sub-partitions using the first set of reference pixels. Then, the method processes a third sub-partition of the plurality of sub-partitions based on a second set of reference pixels determined from processing of the first sub-partition, and processes a fourth sub-partition of the plurality of sub-partitions based on a third set of reference pixels determined from processing of the second sub-partition.

Description

[0001] [Cross-reference to related applications]

[0002] The present disclosure claims priority to U.S. non-provisional application No. 16 / 886,627, filed on May 28, 2020, entitled “PARALLEL INTRA-CODING OF SUB-PARTITIONS,” which claims priority to U.S. provisional application No. 62 / 854,736, filed on May 30, 2019, all of which are incorporated herein by reference in their entirety for all purposes. [Background technology]

[0003] In video coding, video content is compressed via a set of processes including prediction, transformation, quantization, and entropy coding. In intra-frame prediction, spatial redundancy between adjacent pixels in a video frame is reduced. For example, the original pixels of a video frame may not be encoded and sent to a decoder. Instead, the predicted pixels may be subtracted from the original pixels to form a residual. The residual is a smaller amount of information to be encoded and sent to the decoder than the original pixels. This reduces the bandwidth used to transmit the encoded bitstream.

Brief Description of the Drawings

[0004] With respect to the following discussion and particularly with respect to the drawings, it is emphasized that for the purpose of illustrative discussion, the details shown represent examples and are presented in order to provide a description of the principles and conceptual aspects of the present disclosure. In this regard, no attempt is made to show implementation details beyond those required for a basic understanding of the present disclosure. The following discussion in conjunction with the drawings makes it clear to those skilled in the art how embodiments according to the present disclosure may be practiced. In the various drawings and supporting descriptions, similar or identical reference numerals may be used to identify or otherwise refer to similar or identical elements. In the drawings:

[0005] Figure 1 A simplified system for performing intra prediction according to some embodiments is depicted.

[0006] Figure 2 Depicted is an example of a frame used for intra prediction in accordance with some embodiments.

[0007] Figure 3 An example of a transcoder according to some embodiments is depicted.

[0008] Figure 4 An example of a decoder according to some embodiments is depicted.

[0009] Figure 5Depicted are examples of sub-partitions in a current block according to some embodiments.

[0010] Figure 6 An example of parallel intra prediction processing of sub-partitions according to some embodiments is depicted.

[0011] Figure 7 A more detailed flow diagram of transcoding of blocks according to some embodiments is depicted.

[0012] Figure 8 A more detailed flow chart of the decoding of a block according to some embodiments is depicted.

[0013] Fig. 9 An example of an encoding system according to some embodiments is depicted.

[0014] Fig.10 An example of a decoding system in accordance with some embodiments is depicted. [Specific implementation method]

[0015] This article describes techniques for video encoding systems. In the following description, for the purpose of illustration, numerous examples and specific details are set forth in order to provide a thorough understanding of some embodiments. Some embodiments defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may also include modifications and equivalents of the features and concepts described herein.

[0016] In intra prediction, the transcoder can subtract predicted pixels from the original pixels of the video frame to reduce the amount of values ​​that need to be transcoded. One method of intra prediction coding mode is to divide the intra-coded block into multiple sub-partitions and process the sub-partitions separately. Conventionally, when one sub-partition is transcoded first and then the transcoding process of another sub-partition is started, or when one sub-partition is decoded first and then the decoding process of another sub-partition is started, the sub-partitions are processed serially. However, some embodiments process groups of sub-partitions in parallel. This speeds up the encoding process by at least two times compared to serial encoding of the sub-partitions.

[0017] system

[0018] Figure 1 A simplified system 100 for performing intra-frame prediction according to some embodiments is depicted. The system 100 transcodes a source video asset, which can be any type of video, such as for a television program, a movie, or a video clip. The source video may need to be transcoded into one or more formats, such as one or more bit rates. In some embodiments, a server system 102 sends an encoded bitstream to a client 104. For example, the server system 102 can send a video to a client 104 for playback.

[0019] The server system 102 includes a transcoder 106 that transcodes a video into an encoded bit stream. The transcoder 106 can be a software video processor / transcoder configured on a central processing unit (CPU), a hardware accelerated video processor / transcoder with a graphics processing unit (GPU), a field programmable gate array (FPGA), and / or a hardware processor / transcoder implemented in an application specific integrated circuit (ASIC). Transcoding can be a conversion from one digital format to another digital format. Transcoding can involve decoding the source format and encoding the source video into another digital format, or converting the source content into a video with a specific resolution, frame rate, bit rate, codec, etc. Moreover, encoding can be converting analog source content into a digital format. As used, the term transcoding can include encoding.

[0020] The transcoder 106 includes a transcoder intra prediction block 108 that performs intra prediction. The transcoder side and the decoder side use intra coding modes to compress video by removing spatial redundancy of adjacent pixels in a video frame. Traditionally, intra coding can intra-code a current block with a pixel size equal to NxN. In some embodiments, pixels inside the current block are intra-coded using pixels from neighboring blocks of the current block, such as the top, left, and diagonal blocks of the current block.

[0021] The client 104 includes a decoder 112 that decodes the encoded bitstream. During the decoding process, the decoder intra prediction block 110 performs intra prediction to decode the block using pixels from neighboring blocks of the current block, such as blocks above, to the left, and diagonally opposite the current block.

[0022] Figure 2 An example of a frame 200 for intra prediction according to some embodiments is depicted. The frame 200 may be an image that is transcoded or decoded. The frame 200 includes a plurality of blocks, and a current block 202 (#16) is being transcoded or decoded. A top block 204 (#10) at the top of the current block 202 and a left block 206 (#15) at the left of the current block 202 have been decoded. Pixels from the left block 206 and the top block 204 may be used to predict the content of the current block 202. Because these pixels have been decoded, these pixels may be used to predict the content of the current block. For example, neighboring pixels may be similar to some pixels of the current block and provide a good prediction of the current pixel in the current block 202. Pixels from other blocks, such as pixels from blocks #9, #11, and #21 (diagonally positioned), may also be used if pixels from these blocks are available (e.g., have been transcoded and then decoded).

[0023] Transcoder

[0024] Figure 3An example of a transcoder 106 is depicted in accordance with some embodiments. Figure 3 A simplified version of the transcoder is included, and a more detailed version will be described below. The transcoder 106 receives the original pixels from the current block 202 to be transcoded. At 302, the transcoder 106 then subtracts the predicted pixels from the original pixels to produce a residual, which includes the pixels to be transcoded. For example, as part of the transcoding process, the transform and quantization block 304 performs a transform step and a quantization step (T / Q) on the residual value to further compress the size of the data to be transcoded. Finally, the entropy coding block 306 processes the resulting data by entropy coding the data. The transcoder 106 packages the data into a coded bit stream, which is sent to the decoder side by the transcoder 106.

[0025] To ensure that the decoder side performs the decoding process with the same behavior as the transcoder side, the transcoder 106 includes an inverse quantization and inverse transform block 308 and an intra prediction block (e.g., transcoder intra prediction block 108) that mirrors the decoding pipeline. This loop generates reconstructed pixels for the decoded current block 202, which are used as reference pixels for the next coded block in the coding order. For example, at 312, the transcoder 106 adds the predicted pixels used to transcode the current block to the decoded pixels of the residual to generate reference pixels.

[0026] The transcoder intra prediction block 108 generates predicted pixel values ​​using reference pixel values ​​adjacent to the next current block to be transcoded. For example, the transcoder intra prediction block 108 receives reference pixel values ​​and generates predicted pixel values ​​according to the intra mode used. The intra prediction mode may include a predicted direction and use different reference pixels based on the direction used. That is, the horizontal intra prediction direction may use the reference pixel on the left of the current block, and the vertical intra prediction direction may use the reference pixel on the top of the current block. The direction may also include a diagonal direction. The transcoder intra prediction block 108 may generate predicted pixels from the reference pixels and the prediction direction. The transcoder intra prediction block 108 may determine the predicted direction in different ways. In some embodiments, the transcoder intra prediction block 108 may check all possible intra prediction directions and select the best one based on rules such as rate-distortion optimization metrics. Predicted pixels may be used because some directions may require some interpolation of reference pixel values; for example, diagonal directions may interpolate pixel values ​​from one or more reference pixels.

[0027] Decoder

[0028] Figure 4 A more detailed example of the decoder 112 is depicted according to some embodiments. Figure 4A simplified version of the decoder is included, a more detailed version will be described below. On the decoder side, the decoder 112 receives the encoded bitstream and the entropy decoding block 402 performs entropy decoding. Then, the inverse quantization and inverse transform block 404 performs an inverse quantization step and an inverse transform step (Q -1 / T -1 ). The output of the inverse quantization and inverse transform steps is the residual. At 408, the decoder 112 then combines the residual with the predicted pixels that were used to generate the residual at the transcoder 106 to generate decoded pixels. The decoded pixels are reconstructed representations of the original pixels.

[0029] The decoder intra prediction block 406 generates predicted pixels from reference pixels, such as decoded reference pixels from the current block being decoded. The decoded pixels are used as reference pixels for the next coded block in the decoding order. Another coded block in the coding order can use the predicted pixels generated from the reference pixels to decode the coded bitstream of another block.

[0030] Intra-frame prediction

[0031] During intra prediction, the transcoder side and the decoder side can use the intra sub-partitioning (ISP) mode to perform intra sub-partitioning. Intra sub-partitioning divides an intra-coded block into multiple sub-partitions (SPs) and transcodes or decodes these sub-partitions separately.

[0032] Figure 5 An example of subpartitions 502 in a current block 202 according to some embodiments is depicted. The current block 202 is divided into four subpartitions: subpartition #1 502-1, subpartition #2 502-2, subpartition #3 502-3, and subpartition #4 502-4. Although four subpartitions are described, other numbers of subpartitions may be used, such as two subpartitions, six subpartitions, eight subpartitions, etc. Also, the subpartitions may be split horizontally or vertically. Figure 5 In , the current block is split horizontally; however, the current block may be split vertically or in other patterns.

[0033] During the transcoding process, the transcoder 106 processes the sub-partitions 502 separately, which means that the transcoder 106 first transcodes the pixels of the sub-partition 502-1, then transcodes the pixels of the next sub-partition 502-2, and so on. As described above, the transcoder 106 uses reference pixels to transcode each sub-partition 502. For example, reference pixels from a block that has already been transcoded are used to transcode each sub-partition 502.

[0034] The transcoder 106 selects reference pixels for the various sub-partitions 502 in a manner that enables the transcoder to transcode some of the sub-partitions in parallel. Conventionally, the sub-partitions may be transcoded serially; for example, the transcoder first transcodes sub-partition #1 502-1, then transcodes sub-partition #2 502-2 using reference pixels from sub-partition #1 502-1, then transcodes sub-partition #3 502-3 using reference pixels from sub-partition #2 502-2, and finally transcodes sub-partition #4 502-4 using reference pixels from sub-partition #3 502-3. This forces the transcoder to transcode the sub-partitions 502 serially. Instead of transcoding the sub-partitions 502 serially, the transcoder transcodes groups of sub-partitions 502 in parallel, which speeds up the transcoding time of the current block. The transcoder 106 may include multiple transcoding processes capable of performing transcoding of the sub-partitions 502 in parallel, or separate transcoders may be used.

[0035] In order to transcode the sub-partitions 502 in parallel, the reference pixels for some of the sub-partitions 502 are changed from the reference pixels used to transcode the sub-partitions in series. For example, the reference pixels for sub-partition #1 502-1 and sub-partition #3 502-3 are shared. Then, the reference pixels for sub-partition #2 502-2 come from sub-partition #1 502-1, and the reference pixels for sub-partition #4 502-4 come from sub-partition #3 502-3. This allows the transcoder 106 to transcode sub-partition #1 502-1 and sub-partition #3 502-3 in parallel. Once the transcoder 106 transcodes and decodes subpartition #1 502-1 and subpartition #3 502-3, the transcoder 106 may transcode subpartition #2 502-2 using reference pixels from subpartition #1 502-1 and transcode subpartition #4 502-4 using reference pixels from subpartition #3 502-3 in parallel.

[0036] Three groups of reference pixels 504-1 to 504-3 are shown for four sub-partitions 502. If there are different numbers of sub-partitions 502, there may be different numbers of reference pixel 504 groups. However, at least two sub-partitions 502 may share reference pixels to allow parallel processing. Note that for clarity, the reference pixels represented are drawn. In some embodiments, the reference pixels may include one or more pixel rows closest to the current block 202 or adjacent thereto. That is, the reference pixel 504-1 may include reference pixels from the pixel row closest to the current block 202, rather than the pixel row farther away compared to the reference pixels 504-2 and 504-4. The reference pixels are separated to clearly show the individual reference pixels. However, the reference pixels of the overlapping portion on the left in the block may include the same pixels. For example, the pixels at 506 may be the same, and the pixels at 508 may be the same. Moreover, the reference pixels used are not limited to the reference pixels shown. Additional reference pixels may be used, such as more reference pixels from the top block or the left block, or reference pixels from blocks other than the top block and the left block (eg, blocks located diagonally opposite the current block).

[0037] When transcoding subpartition #1 502-1 and subpartition #3 502-3, the transcoder 106 uses reference pixels 504-1. The reference pixels 504-1 may include reference pixels from a block at the top of the current block 202 and reference pixels from a block to the left of the current block 202. In some embodiments, when transcoding subpartition #3 502-3, the transcoder 106 may prefer to use an intra direction that uses reference pixels from a block to the left of the current block 202 rather than reference pixels at the top of the current block 202. The reference pixels to the left of subpartition #3 502-3 are closer and may provide more accurate predictions.

[0038] Also, the reference pixels 504-2 may include reference pixels from sub-partition #2 502-2 and a block to the left of the current block 202. The reference pixels for sub-partition #2 502-2 may be the same as a conventional process.

[0039] The reference pixels 504-3 may include reference pixels from sub-partition #3 502-3 and a block to the left of the current block 202. The reference pixels for sub-partition #4 502-2 may be the same as the conventional process.

[0040] Note that if the current block 202 is partitioned vertically, the reference pixels used may be slightly different. For example, the current block 202 may be rotated 90 degrees to illustrate a vertical partition. However, the concept is the same as the reference pixels shared for sub-partition #1 502-1 and sub-partition #3 502-3. Then, the reference pixels for sub-partition #2 502-2 come from sub-partition #1 502-1, and the reference pixels for sub-partition #4 502-4 come from sub-partition #3 502-3.

[0041] Decoder 112 may use sub-partitions in a similar manner during the decoding process, which will be described in more detail below.

[0042] Figure 6 An example of parallel intra prediction processing of sub-partitions 502 according to some embodiments is depicted. The following processing pipeline can be used in a transcoding process or a decoding process. In the transcoding process, the pipeline generates predicted pixels for a sub-partition, and then after transcoding the sub-partition, generates decoded pixels for the sub-partition to be used as reference pixels for another sub-partition. If the process is performed serially, the above process of generating predicted pixels for sub-partitions and then constructing decoded pixels for the sub-partitions is performed serially for each sub-partition. For example, first, the transcoder 106 generates predicted pixels for sub-partition #1 502-1, generates residuals and transcodes the residuals, and then the transcoder 106 constructs decoded pixels for sub-partition #1 502-1. The transcoder 106 uses these decoded pixels as reference pixels to generate predicted pixels for sub-partition #2 502-2. After intra-coding sub-partition #2 502-2, the transcoder 106 constructs decoded pixels for sub-partition #2 502-2. The transcoder 106 uses these decoded pixels to generate predicted pixels for sub-partition #3 502-3, and the process continues serially until the transcoder 106 transcodes the current block. Therefore, when processing sub-partitions 502 serially, the transcoder 106 must routinely complete transcoding a sub-partition and decode the pixels of the sub-partition before continuing to transcode another sub-partition.

[0043] As described above, the transcoder 106 does not need to wait for a previous sub-partition to be transcoded and then decoded before starting the transcoding process of another sub-partition. Figure 6 In the example, when transcoding of the current block is started, the reference pixels of sub-partition #1 502-1 and sub-partition #3 502-3 are the same, and the transcoder 106 starts transcoding of sub-partition #1 502-1 at 602 and starts transcoding of sub-partition #3 502-3 at 610 in parallel. For example, transcoding process #1 600-1 transcodes sub-partition #1 502-1, and transcoding process #2 600-2 transcodes sub-partition #3 502-3.

[0044] At 602, transcoding process #1 600-1 generates predicted pixels for sub-partition #1 502-1 using reference pixels from neighboring blocks, such as a block on top of the current block 202 and a block to the left of the current block 202. Transcoding process #1 600-1 then performs a transcoding process for sub-partition #1 502-1. After transcoding the pixels, at 604, transcoding process #1 600-1 then reconstructs (e.g., decodes) the pixels of sub-partition #1 502-1. In parallel, at 610, transcoding process #2 600-2 generates predicted pixels for sub-partition #3 502-3 using the same reference pixels as used for the reference pixels for sub-partition #1 502-1. Transcoding process #2 600-2 then performs a transcoding process for sub-partition #3 502-3. After transcoding the pixels, at 612, transcoding process #2 600-2 then reconstructs (eg, decodes) the pixels of sub-partition #3 502-3.

[0045] Sub-partition #2 502-2 uses reference pixels from sub-partition #1 502-1 and from neighboring blocks such as the block to the left. After reconstructing the decoded pixels of sub-partition #1 502-1, at 606, transcoding process #1 600-1 can generate predicted pixels from the reference pixels of sub-partition #2 502-2 and the block to the left. Transcoding process #1 600-1 performs the transcoding process of sub-partition #2 502-2. After transcoding the pixels, at 608, transcoding process #1 600-1 then reconstructs (e.g., decodes) the pixels of sub-partition #2 502-2.

[0046] Also, sub-partition #4 502-4 uses reference pixels from sub-partition #3 502-3. After reconstructing the transcoded pixels of sub-partition #3 502-3, at 614, transcoding process #2 600-2 can generate predicted pixels from reference pixels from sub-partition #4 502-4 and neighboring blocks such as blocks to the left. Transcoding process #2 600-2 performs a transcoding process for sub-partition #4 502-4. After transcoding the pixels, at 616, transcoding process #2 600-2 then reconstructs (e.g., decodes) the pixels of sub-partition #4 502-4.

[0047] After reconstructing the transcoded pixels of sub-partition #2 502-2 and reconstructing the transcoded pixels of sub-partition #4 502-4, the transcoding process ends. Therefore, due to the parallel processing of the two groups of sub-partitions 502, the transcoder 106 can perform the transcoding process twice as fast as performing the process in series. Although two groups are described, the transcoder can divide the current block 202 into a different number of groups. For example, if eight sub-partitions 502 are used, two groups can be used; however, the transcoder can also divide the sub-partitions 502 into four groups to perform the process four times faster than in series and twice as fast as using two groups.

[0048] During the decoding process, the sub-partitions 502 may also be decoded in parallel. For example, when the decoder 112 starts decoding the current block 202, the reference pixels of sub-partition #1 502-1 and sub-partition #3 502-3 are the same, and the decoder 112 starts decoding sub-partition #1 502-1 and sub-partition #3 502-3 in parallel. The decoder 112 may also use two decoding processes, decoding process #1 601-1 and decoding process #2 600-2, similar to the transcoder 106.

[0049] Decoding process #1 601-1 generates predicted pixels for sub-partition #1 502-1 using pixels from neighboring blocks, such as a block at the top of the current block 202 and a block to the left of the current block 202. Decoding process #1 601-1 then performs a decoding process for sub-partition #1 502-1 to generate a residual. After decoding the pixels, decoding process #1 601-1 then uses the predicted pixels and the residual to reconstruct the pixels of sub-partition #1 502-1. In parallel, decoding process #2 600-2 generates predicted pixels for sub-partition #3 502-3 using the same reference pixels as those used for sub-partition #1 502-1. Decoding process #2 600-2 then performs a decoding process for sub-partition #3 502-3. After decoding the pixels, decoding process #2 600-2 then uses the predicted pixels and the residual to reconstruct the reference pixels of sub-partition #3 502-3.

[0050] Sub-partition #2 502-2 uses reference pixels from sub-partition #1 502-1 and from neighboring blocks such as the left block. After decoding the pixels of sub-partition #1 502-1, decoding process #1 601-1 can generate predicted pixels from the decoded reference pixels of sub-partition #1 502-1 and neighboring blocks such as the left block. Decoding process #1 601-1 performs the decoding process of sub-partition #2 502-2 to generate residuals. After decoding the pixels, decoding process #1 601-1 then uses the predicted pixels and the residuals to reconstruct the reference pixels of sub-partition #2 502-2.

[0051] Sub-partition #3 502-3 uses the same reference pixels as sub-partition #1 502-1, and the decoding process can be performed in parallel with sub-partition #1 502-1. Decoding process #2 600-2 performs the decoding process of sub-partition #3 502-3 to generate a residual. After decoding the pixels, decoding process #2 600-2 then uses the predicted pixels and the residual to reconstruct the reference pixels of sub-partition #3 502-3.

[0052] Sub-partition #4 502-4 uses reference pixels from sub-partition #3 502-3 and from neighboring blocks such as the left block. After reconstructing the reference pixels of sub-partition #3 502-3, decoding process #2 600-2 can generate predicted pixels from the reference pixels of sub-partition #3 502-3 and the left block. Decoding process #2 600-2 then performs the decoding process of sub-partition #4 502-4 to generate residuals. After decoding the pixels, decoding process #2 600-2 then uses the predicted pixels and the residuals to reconstruct the reference pixels of sub-partition #4 502-4.

[0053] Figure 7 A more detailed flow chart 700 of transcoding of a block according to some embodiments is depicted. The above flow discusses the generation of predicted pixels. The following flow chart describes the transcoding process in more detail.

[0054] At 702, the transcoding process #1 600-1 receives the original pixels of sub-partition #1 502-1 and the predicted pixels of sub-partition #1 502-1. At 704, the transcoding process #1 600-1 generates the residual of sub-partition #1 from the original pixels and the predicted pixels. At 706, the transcoding process #1 600-1 transcodes the residual of sub-partition #1 502-1 by performing transformation and quantization and entropy coding. At 708, the transcoding process #1 600-1 decodes the residual to generate the decoded pixels of sub-partition #1 502-1. At 710, the transcoding process #1 600-1 combines the decoded pixels with the predicted pixels to generate the reference pixels of sub-partition #1 502-1. At 712, the transcoding process of sub-partition #2 502-2 is performed using steps similar to the steps described in 702 to 710. However, sub-partition #2 uses reference pixels from sub-partition #1.

[0055] The following operations are performed in parallel. Note that the corresponding steps may not be performed completely simultaneously. That is, steps 702 and 712 do not need to start and end at the same time. Instead, steps 702 to 712 are performed in parallel with steps 714 to 724, where each step can be performed at its own pace. At 714, transcoding process #2 600-2 receives the original pixels of sub-partition #3 502-3 and the predicted pixels of sub-partition #3 502-3. At 716, transcoding process #2 600-2 generates the residual of sub-partition #1 from the original pixels and the predicted pixels. At 718, transcoding process #2 600-2 transcodes the residual of sub-partition #3 502-3 by performing transformation and quantization and entropy coding. At 720, transcoding process #2 600-2 decodes the residual to generate decoded pixels of sub-partition #3 502-3. At 722, transcoding process #2 600-2 combines the decoded pixels with the predicted pixels to generate reference pixels for sub-partition #3 502-3. At 724, a transcoding process for sub-partition #4 502-4 is performed using steps similar to those described in 714 to 724. However, sub-partition #4 uses reference pixels from sub-partition #2.

[0056] Figure 8 A more detailed flowchart 800 of decoding of a block according to some embodiments is depicted. At 802, the decoding process #1 601-1 generates predicted pixels from reference pixels of a previous block. The previous block may be a block on top of the current block 202 and / or to the left of the current block 202. Also, the previous block may be a sub-partition of the current block 202 or another block.

[0057] At 804, decoding process #1 601-1 receives the encoded pixels of sub-partition #1. At 806, decoding process #1 601-1 generates residuals for sub-partition #1 from the encoded pixels. For example, decoding process #1 601-1 may perform entropy decoding and inverse quantization and transformation. At 808, decoding process #1 601-1 generates decoded pixels from the predicted pixels and the residuals. For example, decoding process #1 601-1 may combine the decoded pixels with the predicted pixels.

[0058] At 810, decoding process #1 601-1 generates predicted pixels for sub-partition #1 from reference pixels. At 812, decoding process #1 601-1 performs a prediction process for sub-partition #2. The process may be similar to steps 802-810. However, sub-partition #2 uses reference pixels from sub-partition #1.

[0059] The following operations are performed in parallel. Note that the corresponding steps may not be performed completely simultaneously. That is, the steps of 802 and 812 do not need to start and end at the same time. Instead, the steps of 802 to 812 are performed in parallel with the steps of 814 to 824, wherein each step can be performed at its own pace. At 814, decoding process #2 601-2 generates predicted pixels from the reference pixels of the previous block. At 816, decoding process #2 601-2 receives the encoded pixels of sub-partition #3. At 818, decoding process #2 601-2 generates residuals for sub-partition #3 from the encoded pixels. At 820, decoding process #2 601-2 generates decoded pixels from the predicted pixels and residuals. At 822, decoding process #2 601-2 generates predicted pixels for sub-partition #3 from the reference pixels. At 824, decoding process #2 601-2 performs a prediction process for sub-partition #4. The process can be similar to steps 814-822. However, sub-partition #4 uses reference pixels from sub-partition #2.

[0060] in conclusion

[0061] Therefore, by using the same reference pixels for multiple sub-partitions of a block, the transcoding and decoding process can be performed more efficiently. This improves the performance of the transcoder and / or decoder.

[0062] system

[0063] Fig. 9 An example of a transcoding system according to some embodiments is depicted.A video codec framework includes a set of basic components: block partitioning, inter- and intra-frame prediction, transform and quantization, and entropy coding.

[0064] The transcoder 106 receives a video frame, which is first divided into non-overlapping coding blocks for further processing. In order to cope with different video content characteristics, complex areas will be covered by partitions with smaller sizes, while simple areas will be covered by larger partitions. Multiple block modes and shapes can be used together, for example, quadtree mode, ternary tree mode and binary tree mode can be used together, and square blocks and rectangular blocks can also be used together.

[0065] Prediction is used to remove redundancy of the video signal. By subtracting the predicted pixel value from the processed pixel, the amplitude of the residual signal can be significantly reduced, thereby reducing the resulting bitstream size. The intra-frame prediction block 910 using reference pixels in the current frame is intended to reduce spatial redundancy within the frame. The inter-frame prediction block 912 using reference pixels from adjacent frames attempts to remove temporal redundancy between frames. The motion estimation and compensation block 916 can be a submodule of inter-frame prediction on the transcoder side, which captures the motion trajectory of objects between adjacent frames and generates reference pixels for inter-frame prediction.

[0066] The transform and quantization block 904 uses the remaining pixels after intra-frame or inter-frame prediction. The transform and quantization block 904 performs a transform operation to represent the residual signal in the frequency domain. Considering that the human visual system is more sensitive to low-frequency components of video signals than high-frequency components, quantization is designed to further compress the residual signal by reducing the accuracy of high-frequency signals.

[0067] To avoid desynchronization issues between the transcoder 106 and the decoder 112, the transcoder 106 includes a decoding module to ensure that both the transcoder 106 and the decoder 112 use the same mathematical processing. Thus, the inverse transform and inverse quantization block 908 is similar to the same block on the decoder side. The inverse transform and inverse quantization block 908 uses intra-frame and inter-frame prediction to reconstruct pixels.

[0068] An in-loop filter 914 removes any visual artifacts introduced by the above process. Various filtering methods are applied to the reconstructed frame in a cascaded manner to reduce different artifacts, including but not limited to blocking artifacts, mosquito artifacts, color banding effects, etc.

[0069] The entropy coding module 906 may further compress the bitstream using a model-based approach. The transcoder 106 sends the resulting encoded bitstream to the decoder 112 over a network or other type of medium.

[0070] Fig.10 An example of a decoding system according to some embodiments is depicted. The decoder 112 receives the encoded bitstream and inputs it into the entropy decoding block 1002 to recover the information required for the decoding process. As described above, the decoded frame can be decoded by using the inverse transform and inverse quantization block 1004, the intra-frame prediction block 1006 or the inter-frame prediction block 1008, the motion compensation block 1010 and the in-loop filtering block 1012 in the same manner as the decoded frame is constructed.

[0071] Example Embodiments

[0072] In some embodiments, a method includes: receiving, by a computing device, a current block to be encoded; dividing, by the computing device, the current block into a plurality of sub-partitions; determining, by the computing device, a first set of reference pixels; processing, by the computing device, a first sub-partition of the plurality of sub-partitions using the first set of reference pixels in parallel and processing a second sub-partition of the plurality of sub-partitions using the first set of reference pixels; and processing, by the computing device, a third sub-partition of the plurality of sub-partitions based on a second set of reference pixels determined from processing of the first sub-partition, and processing a fourth sub-partition of the plurality of sub-partitions based on a third set of reference pixels determined from processing of the second sub-partition.

[0073] In some embodiments, the first set of reference pixels includes pixels from neighboring blocks of the current block in the frame.

[0074] In some embodiments, the first set of reference pixels includes pixels from a left block located to the left of the current block and a top block located on top of the current block in the frame.

[0075] In some embodiments, the first subpartition is located at a first position along an edge of the frame, and the second subpartition is not immediately adjacent to the first subpartition.

[0076] In some embodiments, the third sub-partition is located between the first sub-partition and the second sub-partition.

[0077] In some embodiments, the fourth sub-partition is adjacent to the second sub-partition.

[0078] In some embodiments, the first set of reference pixels is generated from decoded pixels of another block different from the current block.

[0079] In some embodiments, the second set of reference pixels is generated from decoded pixels of the first sub-partition and the third set of reference pixels is generated from decoded pixels of the second sub-partition.

[0080] In some embodiments, the second set of reference pixels is generated from the first residual of the first sub-partition and the third set of reference pixels is generated from the second residual of the second sub-partition.

[0081] In some embodiments, processing a first sub-partition using a first set of reference pixels and processing a second sub-partition using the first set of reference pixels in parallel includes: generating first predicted pixels from the first set of reference pixels; transcoding the first sub-partition using the first predicted pixels; generating second predicted pixels from the second set of reference pixels; and transcoding the second sub-partition using the second predicted pixels.

[0082] In some embodiments, processing a third sub-partition of multiple sub-partitions based on a second set of reference pixels determined from processing of the first sub-partition and processing a fourth sub-partition of multiple sub-partitions based on a third set of reference pixels determined from processing of the second sub-partition includes: generating third predicted pixels from the second set of reference pixels; transcoding the third sub-partition using the third predicted pixels; generating fourth predicted pixels from the third set of reference pixels; and transcoding the fourth sub-partition using the fourth predicted pixels.

[0083] In some embodiments, a second set of reference pixels is generated from first decoded pixels and first predicted pixels of the first sub-partition, and a third set of reference pixels is generated from second decoded pixels and second predicted pixels of the second sub-partition.

[0084] In some embodiments, processing a first sub-partition using a first set of reference pixels in parallel and processing a second sub-partition using the first set of reference pixels includes: decoding the first sub-partition; generating first predicted pixels from the first set of reference pixels; generating a second set of reference pixels from the decoded first sub-partition and the first predicted pixels; decoding the second sub-partition; generating second predicted pixels from the second set of reference pixels; and generating a third set of reference pixels from the decoded second sub-partition and the second predicted pixels.

[0085] In some embodiments, a non-transitory computer-readable storage medium contains instructions that, when executed, control a computer system to: receive a current block to be encoded; divide the current block into a plurality of sub-partitions; determine a first set of reference pixels; process a first sub-partition of the plurality of sub-partitions in parallel using the first set of reference pixels and process a second sub-partition of the plurality of sub-partitions using the first set of reference pixels; and process a third sub-partition of the plurality of sub-partitions based on a second set of reference pixels determined from processing the first sub-partition, and process a fourth sub-partition of the plurality of sub-partitions based on a third set of reference pixels determined from processing the second sub-partition.

[0086] In some embodiments, the first set of reference pixels includes pixels from neighboring blocks of the current block in the frame.

[0087] In some embodiments, processing a first sub-partition using a first set of reference pixels and processing a second sub-partition using the first set of reference pixels in parallel includes: generating first predicted pixels from the first set of reference pixels; transcoding the first sub-partition using the first predicted pixels; generating second predicted pixels from the second set of reference pixels; and transcoding the second sub-partition using the second predicted pixels.

[0088] In some embodiments, processing a third sub-partition of multiple sub-partitions based on a second set of reference pixels determined from processing of the first sub-partition and processing a fourth sub-partition of multiple sub-partitions based on a third set of reference pixels determined from processing of the second sub-partition includes: generating third predicted pixels from the second set of reference pixels; transcoding the third sub-partition using the third predicted pixels; generating fourth predicted pixels from the third set of reference pixels; and transcoding the fourth sub-partition using the fourth predicted pixels.

[0089] In some embodiments, a second set of reference pixels is generated from first decoded pixels and first predicted pixels of the first sub-partition, and a third set of reference pixels is generated from second decoded pixels and second predicted pixels of the second sub-partition.

[0090] In some embodiments, processing a first sub-partition using a first set of reference pixels in parallel and processing a second sub-partition using the first set of reference pixels includes: decoding the first sub-partition; generating first predicted pixels from the first set of reference pixels; generating a second set of reference pixels from the decoded first sub-partition and the first predicted pixels; decoding the second sub-partition; generating second predicted pixels from the second set of reference pixels; and generating a third set of reference pixels from the decoded second sub-partition and the second predicted pixels.

[0091] In some embodiments, a device includes: one or more computer processors; and a non-transitory computer-readable storage medium including instructions that, when executed, control the one or more computer processors to: receive a current block to be encoded; divide the current block into a plurality of sub-partitions; determine a first set of reference pixels; process a first sub-partition of the plurality of sub-partitions in parallel using the first set of reference pixels and process a second sub-partition of the plurality of sub-partitions using the first set of reference pixels; and process a third sub-partition of the plurality of sub-partitions based on a second set of reference pixels determined from processing the first sub-partition, and process a fourth sub-partition of the plurality of sub-partitions based on a third set of reference pixels determined from processing the second sub-partition.

[0092] Some embodiments may be implemented in a non-transitory computer-readable storage medium for use with or in conjunction with an instruction execution system, device, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform the methods described by some embodiments. The computer system may include one or more computing devices. When executed by one or more computer processors, the instructions may be configured or operable to perform the operations described in some embodiments.

[0093] As used in the description herein and throughout the appended claims, the singular forms, "a," "an," and "the" include plural references unless the context clearly dictates otherwise. Also, as used in the description herein and throughout the appended claims, the meaning of "in" includes "in" and "on," unless the context clearly dictates otherwise.

[0094] The above description illustrates various embodiments and examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be considered to be the only embodiments, and are presented to illustrate the flexibility and advantages of some embodiments as defined by the appended claims. Based on the above disclosure and the appended claims, other arrangements, embodiments, implementations and equivalents may be adopted without departing from the scope of the invention as defined by the claims.

Claims

1. A method for video encoding, the method comprising: The computing device divides the current block into a plurality of sub-partitions; processing, by the computing device, in parallel, a first sub-partition of the plurality of sub-partitions using a first set of reference pixels and processing a second sub-partition of the plurality of sub-partitions using the first set of reference pixels; generating, by the computing device, a second set of reference pixels determined from processing of the first sub-partition; generating, by the computing device, a third set of reference pixels determined from processing of the second sub-partition; as well as A third sub-partition of the plurality of sub-partitions is processed, by the computing device, based on the second set of reference pixels and a fourth sub-partition of the plurality of sub-partitions is processed based on the third set of reference pixels determined from processing of the second sub-partition.

2. The method according to claim 1, wherein: The first set of reference pixels includes pixels from neighboring blocks of the current block in a frame.

3. The method according to claim 1, wherein: The first set of reference pixels includes pixels from one or more neighboring blocks of the current block in a frame.

4. The method according to claim 1, wherein: The first sub-partition is located at a first position along an edge of the frame, and The second sub-partition is not adjacent to the first sub-partition.

5. The method according to claim 4, wherein: The third sub-partition is located between the first sub-partition and the second sub-partition.

6. The method according to claim 4, wherein: The fourth sub-partition is adjacent to the second sub-partition.

7. The method according to claim 1, wherein: The first set of reference pixels is generated from decoded pixels of another block different from the current block.

8. The method according to claim 1, wherein: The second set of reference pixels is generated from decoded pixels of the first sub-partition, and The third set of reference pixels is generated from decoded pixels of the second sub-partition.

9. The method according to claim 1, wherein: The second set of reference pixels is generated from the first residual of the first sub-partition, and The third set of reference pixels is generated from a second residual of the second sub-partition.

10. The method according to claim 1, wherein: Processing the first sub-partition using the first set of reference pixels and processing the second sub-partition using the first set of reference pixels in parallel includes: generating first predicted pixels from the first set of reference pixels; transcoding the first sub-partition using the first predicted pixels; generating second predicted pixels from the second set of reference pixels; and The second sub-partition is transcoded using the second predicted pixels.

11. The method according to claim 10, wherein: Processing the third of the plurality of sub-partitions based on the second set of reference pixels determined from processing of the first sub-partition and processing the fourth of the plurality of sub-partitions based on the third set of reference pixels determined from processing of the second sub-partition comprises: generating a third predicted pixel from the second set of reference pixels; transcoding the third sub-partition using the third predicted pixel; generating fourth predicted pixels from the third set of reference pixels; and The fourth sub-partition is transcoded using the fourth predicted pixels.

12. The method according to claim 11, wherein: The second set of reference pixels is generated from the first decoded pixels of the first sub-partition and the first predicted pixels, and The third set of reference pixels is generated from second decoded pixels of the second sub-partition and the second predicted pixels.

13. The method according to claim 1, wherein: Processing the first sub-partition using the first set of reference pixels and processing the second sub-partition using the first set of reference pixels in parallel includes: decoding the first sub-partition; generating first predicted pixels from the first set of reference pixels; generating the second set of reference pixels from the decoded first sub-partition and the first predicted pixels; decoding the second sub-partition; generating second predicted pixels from the second set of reference pixels; and The third set of reference pixels is generated from the decoded second sub-partition and the second predicted pixels.

14. A non-transitory computer readable storage medium comprising instructions which, when executed, control a computer system to: Divide the current block into multiple sub-partitions; processing, in parallel, a first sub-partition of the plurality of sub-partitions using a first set of reference pixels and processing a second sub-partition of the plurality of sub-partitions using the first set of reference pixels; generating a second set of reference pixels determined from processing of the first sub-partition; generating a third set of reference pixels determined from processing of the second sub-partition; as well as A third sub-partition of the plurality of sub-partitions is processed based on the second set of reference pixels, and a fourth sub-partition of the plurality of sub-partitions is processed based on the third set of reference pixels determined from processing of the second sub-partition.

15. The non-transitory computer-readable storage medium of claim 14, wherein: The first set of reference pixels includes pixels from neighboring blocks of the current block in a frame.

16. The non-transitory computer-readable storage medium of claim 14, wherein: Processing the first sub-partition using the first set of reference pixels and processing the second sub-partition using the first set of reference pixels in parallel includes: generating first predicted pixels from the first set of reference pixels; transcoding the first sub-partition using the first predicted pixels; generating second predicted pixels from the second set of reference pixels; and The second sub-partition is transcoded using the second predicted pixels.

17. The non-transitory computer-readable storage medium of claim 16, wherein: Processing the third of the plurality of sub-partitions based on the second set of reference pixels determined from processing of the first sub-partition and processing the fourth of the plurality of sub-partitions based on the third set of reference pixels determined from processing of the second sub-partition comprises: generating a third predicted pixel from the second set of reference pixels; transcoding the third sub-partition using the third predicted pixel; generating fourth predicted pixels from the third set of reference pixels; and The fourth sub-partition is transcoded using the fourth predicted pixels.

18. The non-transitory computer-readable storage medium of claim 17, wherein: The second set of reference pixels is generated from the first decoded pixels of the first sub-partition and the first predicted pixels, and The third set of reference pixels is generated from second decoded pixels of the second sub-partition and the second predicted pixels.

19. The non-transitory computer-readable storage medium of claim 14, wherein: Processing the first sub-partition using the first set of reference pixels and processing the second sub-partition using the first set of reference pixels in parallel includes: decoding the first sub-partition; generating first predicted pixels from the first set of reference pixels; generating the second set of reference pixels from the decoded first sub-partition and the first predicted pixels; decoding the second sub-partition; generating second predicted pixels from the second set of reference pixels; and The third set of reference pixels is generated from the decoded second sub-partition and the second predicted pixels.

20. A device for video encoding, the device comprising: one or more computer processors; and A non-transitory computer-readable storage medium comprising instructions that, when executed, control the one or more computer processors to: Divide the current block into multiple sub-partitions; processing, in parallel, a first sub-partition of the plurality of sub-partitions using a first set of reference pixels and processing a second sub-partition of the plurality of sub-partitions using the first set of reference pixels; generating a second set of reference pixels determined from processing of the first sub-partition; generating a third set of reference pixels determined from processing of the second sub-partition; as well as A third sub-partition of the plurality of sub-partitions is processed based on the second set of reference pixels, and a fourth sub-partition of the plurality of sub-partitions is processed based on the third set of reference pixels determined from processing of the second sub-partition.

Images