Wide-angle intra prediction with subpartitions

Abstract

To provide a method and an apparatus for performing prediction for encoding and decoding.SOLUTION: An encoding method determines an intra-prediction direction of a rectangular sub-partition of a video block on the basis of a dimension of the sub-partition when a ratio of dimensions of the sub-partition is within a certain range and on the basis of a dimension of the video block when the ratio of the dimensions of the sub-partition is outside the certain range, maps reference samples from an upper right or lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, predicts samples of the rectangular sub-partition using reference samples from an upper row of the video block or reference samples from a left column of the video block, and encodes the video block using the prediction in an intra-coding mode.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] Technical Field At least one of the present embodiments generally relates to a method or apparatus for video encoding or decoding. [Background technology]

[0002] background To achieve high compression efficiency, image and video coding schemes typically utilize prediction, including spatial and / or motion vector prediction, and transform to exploit spatial and temporal redundancy within the video content. Typically, intra- or inter-prediction is used to exploit intra- or inter-frame correlation, and then the difference between the original and predicted image, often referred to as the prediction error or prediction residual, is transformed, quantized, and entropy coded. To reconstruct the video, the compressed data is decoded by an inverse process corresponding to entropy coding, quantization, transformation, and prediction. Summary of the Invention

[0003] overview The shortcomings and disadvantages of the prior art are addressed by the general aspects described herein directed to block-shape adaptive intra-prediction direction in encoding and decoding.

[0004] According to a first aspect, a method is provided, comprising: determining an intra-prediction direction for a rectangular sub-partition of a video block based on a dimension of the sub-partition when a ratio of the dimensions of the sub-partition is within a specific range, and based on a dimension of the video block when the ratio of the dimensions of the sub-partition is outside the specific range, mapping reference samples from an upper right or a lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, predicting samples of the rectangular sub-partition using reference samples from an upper row of the video block or a left column of the video block, wherein a number of reference samples in the upper row of the video block or the left column of the video block is determined based on the dimension of the rectangular sub-partition, and encoding the rectangular sub-partition of the video block using the prediction in an intra-coding mode.

[0005] According to a second aspect, a method is provided, comprising: determining an intra-prediction direction of a rectangular sub-partition of a video block based on dimensions of the sub-partition when a ratio of dimensions of the sub-partition is within a specific range, and based on dimensions of the video block when a ratio of dimensions of the sub-partition is outside the specific range, mapping reference samples from an upper right or a lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, predicting samples of the rectangular sub-partition using reference samples from an upper row of the video block or reference samples from a left column of the video block, wherein a number of reference samples in the upper row of the video block or the left column of the video block is determined based on dimensions of the rectangular sub-partition, and decoding the rectangular sub-partition of the video block using the prediction in an intra-coding mode.

[0006] According to another aspect, there is provided an apparatus having a processor, the processor being configured to encode blocks of video or decode a bitstream by performing any of the methods described above.

[0007] According to another general aspect of at least one embodiment, there is provided an apparatus having an apparatus according to any of the decoding embodiments and at least one of (i) an antenna configured to receive a signal, the signal including a video block, (ii) a band limiter configured to limit the received signal to a band of frequencies including the video block, or (iii) a display configured to display an output representing the video block.

[0008] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that includes data content generated according to any of the described encoding embodiments or variations.

[0009] According to another general aspect of at least one embodiment, there is provided a signal having video data generated according to any of the described encoding embodiments or variations.

[0010] According to another general aspect of at least one embodiment, the bitstream is formatted to include data content generated according to any of the described encoding embodiments or variations.

[0011] According to another general aspect of at least one embodiment, there is provided a computer program product having instructions that, when executed by a computer, cause the computer to perform any of the described decoding embodiments or variations.

[0012] These and other aspects, features, and advantages of the general aspects will become apparent from the following detailed description of illustrative embodiments, which is to be read in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0013] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1] 1 shows a reference sample for intra prediction in VTM. [Figure 2] 1 shows multiple reference lines for intra prediction in VTM. [Figure 3] 1 shows the intra prediction direction in VTM for a square target block. [Figure 4] An example of division into 4x8 and 8x4 blocks is shown below. [Figure 5] Examples of divisions for all blocks except 4x8, 8x4, and 4x4 are shown. [Figure 6] 1 illustrates wide-angle intra prediction for non-square blocks. [Figure 7] Indicates the ISP wide angle calculated from the height and width of the CU. [Figure 8] The reference samples used in the wide angle and horizontal split ISP cases are shown. [Figure 9] 1 shows an example of a proposed array of reference samples for predicting subpartitions. [Figure 10] 1 shows an example of reference array padding in a VTM for the illustrative case of two subpartitions. [Figure 11] 10 shows the mapping of reference samples from the top reference of a coding unit in the case of a horizontal split. [Figure 12] 10 shows the mapping of reference samples from the top reference of a coding unit in the case of a horizontal split. [Figure 13] Here are some common standard encoding methods: [Figure 14] A common standard decoding scheme is shown. [Figure 15] 1 illustrates a representative processor configuration in which the described embodiments may be implemented. [Figure 16] Here are some common standard encoding methods: [Figure 17] A common standard decoding scheme is shown. [Figure 18] 1 illustrates a representative processor configuration in which the described embodiments may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0014] Detailed Description The embodiments described herein are in the field of video compression and relate to video compression and video encoding and decoding.

[0015] The Versatile Video Coding (VVC) Test Model 4.0 (VTM) supports Intra Prediction with Sub-partitioning (ISP), where the target block is encoded with a choice of horizontal or vertical sub-partitioning. Prediction for each sub-partition uses the same prediction direction as that of the target coding unit. Sub-partition prediction can be improved by utilizing wide-angle prediction adapted to the aspect ratio of each sub-partition independent of the parent coding unit. Second, sub-partition prediction uses decoded pixels in the previous sub-partition as reference samples. This may result in the most significant right or least significant left reference sample of a sub-partition being unavailable. Reference samples from the parent CU can be used to improve prediction along directions requiring these samples.

[0016] In the Versatile Video Coding (VVC) Test Model (VTM), any target block in intra prediction can have one of 67 prediction modes. Similar to HEVC, one is planar mode, one is DC mode, and the remaining 65 are directional modes. The 65 directional modes are selected from 95 directions, including 65 normal angles ranging from 45° to −135° when the target is square, and potentially 28 wide-angle directions when the block is rectangular. VTM encodes the prediction mode of a block using a Most Probable Mode (MPM) set of six prediction modes. If a prediction mode does not belong to the MPM set, it is truncated to 5 or 6 bits and binary encoded.

[0017] Intra prediction in video compression refers to the spatial prediction of a block of pixels using information from causal neighboring blocks, i.e., neighboring blocks within the same frame that have already been decoded. It is a powerful coding tool because it allows high compression efficiency not only in INTRA frames but also in INTER frames whenever a relatively good temporal prediction does not exist. Therefore, intra prediction is included as a core coding tool in all video compression standards, including H.264 / AVC, HEVC, etc. For the purpose of explanation below, we will refer to intra prediction in the Versatile Video Coding (VVC) Software Test Model (VTM).

[0018] In VTM, encoding of frames of a video sequence is based on a Quad-Tree (QT) / Multi-Type Tree (MTT) block structure. Frames are divided into non-overlapping square Coding Tree Units (CTUs), all of which undergo QT / MTT-based splitting into multiple Coding Units (CUs) based on a rate-distortion criterion. For the purpose of relative ease of reference, we will use the terms "CU" and "block" interchangeably throughout this specification.

[0019] In intra prediction, a CU is spatially predicted from its neighboring CUs, i.e., the top and left CUs. For this purpose, VTM uses a simple spatial model called a prediction mode. Based on the decoded pixel values ​​in the top and left CUs, called reference pixels, the encoder constructs different predictions of the target block and selects the one that yields the best RD performance. Of the 95 defined modes, one is a planar mode (indexed as mode 0), one is a DC mode (indexed as mode 1), and the remaining 93 are angular modes (indexed as modes -14...-1, 2...80). Of the 93 angular modes, only 65 neighboring modes are selected for any target CU according to its shape. The angular modes aim to model the directional structure of objects in a frame. Therefore, to fill the target CU, the decoded pixel values ​​in the top and left CUs are simply repeated along the defined direction. Some prediction modes may introduce discontinuities along the top and left reference boundaries, and therefore these prediction modes may include a subsequent post-processing step called Position Dependent Intra Prediction Combination (PDPC), which aims to smooth the pixel values ​​along the boundaries.

[0020] The intra prediction process in VTM consists of three steps: (1) reference sample generation, (2) intra sample prediction, and (3) post-processing of predicted samples. The reference sample generation process is illustrated in Figure 1, which shows reference samples for intra prediction in VTM. H and W represent the height and width of the current block, respectively. For a CU of size H x W, a row of 2W decoded samples at the top is formed from the pre-reconstructed top and top-right pixels of the current CU. Similarly, a column of 2H samples at the left is formed from the reconstructed left and lower-left pixels. In addition, the corner pixel at the top-left position is used to fill the gap between the top row and the left column reference. If some samples at the top or left are unavailable because the corresponding CU does not exist in the same slice, or the current CU is located at a frame boundary, or for some other reason, a method called reference sample substitution can be performed.

[0021] VTM also supports intra prediction with multiple reference lines (MRL). The idea is to perform prediction based on several sets of reference lines as shown in Figure 2, and then select the reference line that gives the best rate-distortion performance. The reference line to be used is signaled to the decoder by a variable length code.

[0022] The next step, intra-sample prediction, involves predicting the pixels of the target CU based on the reference sample. As mentioned above, VTM supports a series of prediction models to efficiently predict different types of content. Planar and DC prediction modes are used to predict smooth and gradually changing regions, while angular prediction modes are used to capture different directional structures. VTM supports 95 directional prediction modes, indexed from -14 to -1 and from 2 to 80. For square CUs, only prediction modes 2 to 66 are used. These prediction modes correspond to different prediction directions ranging from 45 degrees to -135 degrees in a clockwise direction, as shown in Figure 3, which shows the intra-prediction directions in VTM for square target blocks. Although non-square blocks can generally be used with extended prediction directions, this figure shows square blocks. The numbers indicate the prediction mode index associated with the corresponding direction. Modes 2 to 33 indicate horizontal prediction, and modes 34 to 66 indicate vertical prediction.

[0023] Modes with indices from -14 to -1 and from 67 to 80 are wide-angle modes used for rectangular blocks of different shapes. Modes -14 to -1 are defined beyond mode 2 (angles greater than 45 degrees) and are used for tall rectangular blocks (blocks with height greater than width). Similarly, modes 67 to 80 are defined beyond mode 66 (angles greater than -135 degrees) and are used for flat rectangular blocks (blocks with width greater than height). The number of wide-angle modes used for a rectangular block depends on the aspect ratio of the block. In either case, the total number of angular modes used for any block is 65, and the modes are always contiguous in direction.

[0024] The general aspects described herein aim to improve prediction efficiency in intra prediction involving subpartitions. First, it proposes predicting each subpartition using its own wide-angle mode rather than the wide-angle mode of the parent coding unit. Second, it also proposes using the reference samples of the parent CU for predicting the subpartition when those samples are unavailable. These two proposals can be incorporated into current VTM code to improve coding performance. The added complexity and memory requirements for incorporating these changes are minimal, with the potential for some coding gain.

[0025] In the Versatile Video Coding (VVC) Test Model (VTM), encoding of frames of a video sequence is based on a quad-tree (QT) / multi-type tree (MTT) block structure. A frame is divided into non-overlapping square coding tree units (CTUs), all of which undergo QT / MTT-based splitting into multiple coding units (CUs) based on a rate-distortion criterion. In intra prediction, a CU is spatially predicted from its causal neighboring CUs, i.e., the top and left CUs. For this purpose, the VTM uses a simple spatial model called a prediction mode. Based on the decoded pixel values ​​in the top and left CUs, called reference pixels, the encoder constructs different predictions for the target block and selects the one that results in the best RD performance.

[0026] In VTM 4.0, a target block has the option to choose between intra prediction of the entire CU and intra prediction with subpartitions of the CU (ISP). In the latter prediction, the target CU is divided into two or four equally sized subpartitions, which are decoded sequentially using one prediction mode. That is, each subpartition is decoded independently. Therefore, a subpartition can benefit from the availability of decoded samples from neighboring subpartitions, which are direct neighbors of the current subpartition. This improves prediction and therefore compression efficiency.

[0027] VTM4.0 also supports Wide Angle Intra Prediction (WAIP), which allows the use of wide-angle prediction directions for rectangular CUs. Depending on the aspect ratio of the target block, some normal prediction directions are replaced by corresponding wide-angle directions. The prediction directions are also modified to properly adapt to different rectangular block shapes, so that the defined prediction directions are aligned on the secondary diagonal of the block. In effect, the method aims to provide better predictions that result in higher compression efficiency by using wide angles instead of normal angles.

[0028] In VTM4.0, it is easy to combine ISP with WAIP. For each subpartition in an ISP, the prediction direction is the same as that of the parent CU. However, this is not required. Since subpartitions are decoded independently, WAIP allows the prediction direction of each subpartition to be determined independently. Second, except for the first subpartition, the remaining subpartitions use decoded pixels from previous subpartitions as reference samples for prediction. This undesirably results in some reference samples being unavailable depending on the prediction direction. This situation can be alleviated by using reference samples from the parent CU. Before describing the proposed mode, we briefly present ISP and WAIP in VTM4.0 below. For ease of reference, we will use the terms "CU" and "block" interchangeably throughout this specification.

[0029] The ISP tool in VTM 4.0 divides luma intra-predicted blocks into two or four subpartitions vertically or horizontally, depending on the block size. A subpartition must have at least 16 samples. Thus, a 4x4 block is not divided into subpartitions, while 4x8 and 8x4 blocks have only two partitions. All other sized blocks have only four subpartitions. The subpartitions can be horizontal or vertical. A 4x8 block can have only two 4x4 vertical partitions, while an 8x4 block can have only two 4x4 horizontal partitions. Similarly, as another example, a 4x16 block can have four 4x4 vertical subpartitions, or four 1x16 horizontal subpartitions. Figures 4 and 5 show two possible examples.

[0030] [Table 1]

[0031] For each of these subpartitions, a prediction is constructed using the decoded prediction mode of the parent CU. This prediction signal is added to the decoded residual signal generated by entropy decoding the coefficients sent by the encoder and then inverse quantizing and inverse transforming them to reconstruct the pixels within the subpartition. Except for the first subpartition, the reconstructed values ​​of each subpartition can be used to generate a prediction for the next one.

[0032] Subpartitions are processed in normal order regardless of the intra mode and split used, i.e., the first subpartition to be processed is the one containing the most significant left sample of the CU, and then continues consecutively downwards (horizontal split) or to the right (vertical split).

[0033] VTM4.0 also supports intra prediction with multiple reference lines (MRL). The target block can select from the first, second, and fourth reference lines to use the one that gives the best rate-distortion performance. The selected reference line is signaled by a 1-bit (0) or 2-bit (10 or 11) flag to indicate between the first reference line or the second and fourth reference lines, respectively. In VTM4.0, ISP is only applied to blocks that use the first reference line. Therefore, if a block has a non-zero MRL index, the ISP coding mode will be assumed to be 0 and will not be transmitted to the decoder.

[0034] The ISP method is tested with intra modes that are part of the MPM list, which consists of six individual modes out of the 67 prediction modes. For any block tested by ISP, the MPM list is modified to exclude DC modes and to prioritize horizontal intra modes for horizontal splits and vertical intra modes for vertical splits.

[0035] The basic idea behind wide-angle prediction is to adapt the prediction directions according to the block shape while keeping the total number of prediction modes the same. This is done by adding some prediction directions on the larger side of the block and reducing the prediction directions on the shorter side. The overall aim is to improve prediction accuracy and result in higher compression efficiency. The newly introduced directions are called wide-angle directions because they are beyond the usual range of 180 degrees, from 45 degrees to -135 degrees.

[0036] When the target block is square, the wide angle has no role to play because the defined modes of the block remain unchanged. When the target block is flat, i.e., its width W exceeds its height H, some modes close to 45 degrees are removed and an equal number of wide-angle modes above -135 degrees are added. The added directions are indexed as prediction modes 67, 68, ..., etc. Similarly, when the target block is tall, some modes close to -135 degrees are removed and an equal number of wide-angle modes above 45 degrees are added. Because prediction modes 0 and 1 are reserved for planar and DC prediction, the added directions are indexed as -1, -2, ..., etc.

[0037] Since the direction of the secondary diagonal depends on the block shape, the total number of wide-angle modes utilized depends on the block shape. Table 2 shows the number of normal modes replaced by wide-angle modes for different block shapes.

[0038] [Table 2]

[0039] For any target block, the mapping from permuted normal modes to wide-angle modes is performed as follows: modeShift[5] = {0, 6, 10, 12, 14}; ratio = Abs(Log2(W / H)) if W > H and 1 < predMode < 2 + modeShift[ratio] predMode = predMode + 65; else if H > W and (66 - modeShift[ratio]) < predMode <= 66 predMode = predMode - 67;

[0040] In the case of ISP, the mapping from normal mode to wide-angle mode is performed using the height and width of the parent CU, not the height and width of the subpartition. Regardless of whether the new mode is wide-angle mode or normal mode, the same mode is used for prediction in each subpartition. This is shown by an example in Figure 7.

[0041] As shown above, the current approach for ISP with WAIP is to determine the prediction angle from the aspect ratio of the parent CU and use it for prediction in each subpartition. This constraint is not necessarily binding because the subpartitions are processed sequentially but independently. A second problem resulting from individual subpartition processing is the unavailability of some reference samples, which may affect prediction in certain directions. We provide two solutions to these problems below.

[0042] For the sake of generality, in what follows we will assume rectangular blocks with width W and height H. A square target block is a special case with W=H.

[0043] WAIP for intra-subpartition For blocks with ISP, the current VTM code (VTM4.0) processes each subpartition individually in the normal order. In the prediction process, the predictor first generates one reference array of samples at the top of the partition and one reference array of samples at the left. If the ISP split is horizontal, the reference array at the top is constructed using the decoded samples in the last row of the previous subpartition. Similarly, if the ISP split is vertical, the reference array at the left is constructed using the decoded samples in the last column of the previous subpartition. This is shown in Figure 8.

[0044] The current VTM code determines the size of the reference array for each partition from what is required by the parent CU. We propose to determine the prediction direction from the dimensions of the subpartition. As an implementation advantage, we also propose to determine the length of the partition's reference array from the partition's dimensions.

[0045] W p and H p Let denote the width and height of each partition. As a result, the lengths of the reference array at the top and left are 2*W, respectively, by excluding the top-left pixel. p and 2*H p It is judged as:

[0046] For any sub-partition, as shown in FIG. 9, the mapping from normal mode to wide-angle mode is performed as follows: modeShift[5] = {0, 6, 10, 12, 14}; whRatio = Abs(Log2(Wp / Hp)) if Wp > Hp and 1 < predMode < 2 + modeShift[whRatio] predMode = predMode + 65; else if Hp > Wp and (66 - modeShift[whRatio]) < predMode <= 66 predMode = predMode - 67;

[0047] The above mapping is valid as long as the aspect ratio of the subpartitions is within the valid range given in Table 2. If Wp / Hp>16 or Wp / Hp<1 / 16, we suggest using the parent CU dimensions for wide-angle derivation, as is done in VTM.

[0048] In a simple variant of the above method, if a subpartition has a square shape, that is, if Wp=Hp, then a normal prediction mode is applied; otherwise, a wide angle is obtained using the dimensions of the CU.This simplification avoids the use of a wide-angle prediction mode (derived from a parent rectangular CU) for a square subpartition.In other words, wide angles are derived from subpartition dimensions only when the subpartition is square; otherwise, they are derived from CU dimensions.Since there are no wide angles in the case of a square shape, this is equivalent to not having a wide-angle derivation for square subpartitions and preserving the wide-angle derivation for rectangular subpartitions.This is also presented in the following embodiment 1.

[0049] Since all subpartitions in an ISP have equal dimensions, the wide-angle mode calculated for one subpartition is the same for all subpartitions.

[0050] Replacing missing reference samples The second improvement over ISP is to use the reference sample from the top or left of the parent CU. Let's assume that subpartitions are processed in normal order, as in VTM 4.0. In the case of a horizontal split, the top reference sample for subpartitions 2 and above uses the decoded pixel from the previous subpartition. Because the previous partition has the same width as the parent CU, the top right reference sample is simply unavailable. Current VTM codecs would use padding in this case, where the last available reference pixel is repeated throughout the top-right portion of length W. This is shown in Figure 10 for the two subpartition case. A similar case applies for a vertical split, where the bottom left reference sample is lost in subpartitions 2 and above. In this case, current VTM codecs would use padding, where the last available reference pixel is repeated throughout the bottom-left portion of length H.

[0051] These missing reference samples can be replaced from the top reference of the parent CU, as shown in Figure 11 for a horizontal split into two subpartitions. That is, the missing samples are mapped from the reference array of the corresponding parent CU along the prediction direction. If some reference samples are still missing after the replacement process, a normal padding process can be applied, in which the last reference sample is repeated to fill the reference array.

[0052] The missing samples mapped from the reference array of the corresponding parent CU along the prediction direction can use the same interpolation process as the prediction (linear, 4-tap...), or to reduce complexity, the nearest neighbors can be used (no interpolation). In the last case, this corresponds to shifting the reference samples by the number of pixels corresponding to the prediction angle.

[0053] This reference sample replacement process has the undesirable consequence of creating intensity discontinuities where pixels are copied from the parent CU's top reference. To mitigate this effect, a simple low-pass filter can be used. For example, we can use a 3-tap filter ∇ ...

[0054] In Figure 8, the mapped samples on the top reference are arranged along the prediction direction. Another method is to copy all the top right reference samples of a CU onto the top right reference samples of the second or subsequent subpartitions, and then perform low-pass filtering at the top corner to reduce discontinuities. This is shown in Figure 12.

[0055] This replacement process is independent of whether the WAIP changes in the previous section are applied: it can be applied without or in conjunction with the WAIP changes for each subpartition.

[0056] In the following, we assume any common video codec that ISP and WAIP apply, the VTM codec being one example of such a codec.

[0057] Embodiment 1: In this embodiment, if the target block is split in ISP, in any prediction mode, we use the dimensions of the parent CU to determine the wide angle unless the sub-partitions have a square shape. In that case, the normal prediction mode is applied to them without mapping to a wide angle. All other codec parameters remain unchanged, as in VTM4.

[0058] Embodiment 2: In this embodiment, if the target block is split in ISP, in any prediction mode, we use the dimension of the parent CU to determine the wide angle, as in VTM4. However, from the second subpartition onwards, we replace the missing reference samples in the top right (bottom left) from the top (left) reference array of the parent CU in case of horizontal (vertical) split. The reference samples in the top (left) reference array are arranged along the prediction direction, as in Figure 11. All other codec parameters remain unchanged, as in VTM4.

[0059] Embodiment 3: In this embodiment, if the target block is split in ISP, in any prediction mode, we use the dimension of the parent CU to determine the wide angle, as in VTM4. However, from the second subpartition onwards, we replace the missing reference samples of the top-right (bottom-left) from the top-left (left-most) reference array of the parent CU in case of horizontal (vertical) split. The reference samples on the top-left (left-most) reference array are directly copied onto the top-right (bottom-left) part of the reference array in case of the second subpartition onwards, as in Figure 12. All other codec parameters remain unchanged, as in VTM4.0.

[0060] Embodiment 4: In this embodiment, if the target block is split in ISP, in any prediction mode, we use the dimensions of the parent CU to determine the wide angle unless the subpartition has a square shape. If the subpartition has a square shape, normal prediction mode is applied to them without mapping to the wide angle. In addition, from the second subpartition onwards, we replace the missing reference samples of the top right (bottom left) from the top (left) reference array of the parent CU in case of horizontal (vertical) split. The reference sample mapping is performed as in embodiment 2 or embodiment 3. All other codec parameters remain unchanged as in VTM4.0.

[0061] Embodiment 5: In this embodiment, if the target block is split in ISP, in any prediction mode, we use the dimensions of the subpartition instead of the dimensions of the parent CU to determine the wide angle. If the width-to-length ratio (Wp / Hp) of a subpartition is greater than 16 or less than 1 / 16, we will use the W / H ratio of the parent CU to calculate the wide angle, and use it for all subpartitions. In addition, from the second subpartition onwards, we replace the missing reference samples of the top right (bottom left) from the top (left) reference array of the parent CU in case of a horizontal (vertical) split. The reference samples in the top (left) reference array are arranged along the prediction direction, as in Figure 11. All other codec parameters remain unchanged, as in VTM4.0. To speed up the prediction step, the reference array can be used for subpartitions with only twice the dimension of the parent CU, rather than twice the dimension of the parent CU. When using the CU size, the original nominal array length can be preserved.

[0062] Embodiment 6: In this embodiment, if the target block is split in ISP, in any prediction mode, we use the dimensions of the subpartition instead of the dimensions of the parent CU to determine the wide angle. If the width-to-height ratio (Wp / Hp) of a subpartition is greater than 16 or less than 1 / 16, we will use the W / H ratio of the parent CU to calculate the wide angle, and will use this for all subpartitions. In addition, from the second subpartition onwards, we replace the missing reference samples of the top right (bottom left) from the top (left) reference of the parent CU in case of horizontal (vertical) split. The reference samples on the top (left) reference array are directly copied onto the top right (bottom left) part of the reference array in case of the second subpartition onwards, as in Figure 12. All other codec parameters remain unchanged, as in VTM4.0. To speed up the prediction step, the reference array can be used for the subpartitions only twice its size, rather than twice the size of the parent CU. When using the CU size, the original reference array length can be preserved.

[0063] Embodiment 7: VTM4.0 uses an ISP with two or four subpartitions, with the condition that each subpartition must have a minimum of 16 pixels. In this embodiment, we relax these constraints and apply the ISP to all block shapes, with the condition that each subpartition can be at least a single row or a single column. Depending on the CU size, the number of subpartitions within a CU can also be larger, such as 8, 16, or 32. All subpartitions are constrained to be identical in shape, regardless of horizontal or vertical splits. If the width-to-height ratio (Wp / Hp) of a subpartition is greater than 16 or less than 1 / 16, we will use the W / H ratio of the parent CU to calculate the wide angle, and use this for all subpartitions. From the second subpartition onwards, we replace the missing reference sample from the top (left) reference of the parent CU in the case of a horizontal (vertical) split. The mapping of the missing samples is performed in the same manner as in the second or third embodiment.

[0064] Embodiment 8: In this embodiment, we relax the constraint that all subpartitions must have the same shape. Subpartitions can have unequal shapes, but are constrained to have heights and widths as powers of two. Thus, the number of subpartitions does not need to be a power of two, as in embodiments 1-7. For example, an 8x8 block can be split into three partitions of sizes 4x8, 2x8, and 2x8. Any of embodiments 1-7 can be modified accordingly.

[0065] Embodiment 9: VTM4.0 uses ISP for blocks that do not use MRL. In this embodiment, we relax this restriction. In the general case, ISP can also be applied when a CU uses multiple reference lines. In one variant, if the split is horizontal and the prediction direction is vertical, or if the split is vertical and the prediction direction is horizontal, only the first subpartition may use MRL. In another variant, all subpartitions may use MRL regardless of split type and prediction direction. In all these variants, we can use any of embodiments 1 to 8 to calculate the wide angle of the subpartition and / or to map missing reference samples from the second subpartition onwards.

[0066] Embodiment 10: VTM4.0 uses ISP only for the luma component. In this embodiment, we relax this constraint and apply any of embodiments 1 to 9 to both the luma and chroma components.

[0067] By incorporating the modifications in the VTM4.0 reference software, the wide-angle mode derivation with the proposed ISP was implemented similarly to Embodiment 1. Tests were performed with one frame from the JVET test sequence under the All-Intra (AI) configuration. Table 3 shows the BD rate performance of the tested method relative to the VTM4.0 anchor results. It can be observed that the luma BD rate is improved by 0.04% without any additional encoder or decoder complexity. The gain in Class C and Class E sequences is notable, at 0.11% for luma.

[0068] [Table 3]

[0069] The general aspects described herein aim to improve intra prediction efficiency in ISP by modifying the wide-angle derivation and by replacing missing reference samples for some sub-partitions with those from the reference samples of the parent CU. One of the advantages is relatively high compression efficiency without significant additional complexity.

[0070] Figure 13 shows one embodiment of a method 1300 for encoding a block of video data using the general aspects described herein. The method begins at start block 1301, with control passing to function block 1310, which determines the intra-prediction direction of a rectangular sub-partition of a video block based on a dimension of the sub-partition when the ratio of the sub-partition dimensions is within a particular range, and based on a dimension of the video block when the ratio of the sub-partition dimensions is outside the particular range. Control then passes from block 1310 to block 1320, which maps reference samples from the upper right or lower left of the video block as upper right or lower left reference samples for prediction of a sub-partition of the video block. Control passes from block 1320 to block 1330, which predicts samples of the rectangular sub-partition using reference samples from the upper row of the video block or the left column of the video block, where the number of reference samples in the upper row of the video block or the left column of the video block was determined based on the dimension of the rectangular sub-partition. Control passes from block 1330 to block 1340, which encodes the video block using prediction in an intra-coding mode.

[0071] Figure 14 shows one embodiment of a method 1400 for decoding a block of video data using the general aspects described herein. The method begins at start block 1401, and control passes to function block 1410, which determines the intra-prediction direction of a rectangular sub-partition of the video block based on a dimension of the sub-partition when the ratio of the dimensions of the sub-partition is within a particular range, and based on a dimension of the video block when the ratio of the dimensions of the sub-partition is outside of the particular range. Control then passes from block 1410 to block 1420, which maps reference samples from the upper right or lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block. Control passes from block 1420 to block 1430, which predicts samples of the rectangular sub-partition using reference samples from the upper row of the video block or reference samples from the left column of the video block, where the number of reference samples in the upper row of the video block or the left column of the video block was determined based on the dimension of the rectangular sub-partition. Control passes from block 1430 to block 1440, which encodes the video block using prediction in an intra-coding mode.

[0072] 15 shows one embodiment of an apparatus 1500 for encoding or decoding blocks of video data. The apparatus includes a processor 1510, which may be interconnected through at least one port to a memory 1520. Both the processor 1510 and the memory 1520 may also have one or more further interconnections to external connections.

[0073] The processor 1510 is configured to encode or decode video data by using an intra-prediction mode with sub-partitions, by encoding or decoding blocks of video data using an intra-prediction mode with sub-partitions.

[0074] The general aspects described aim to improve intra-prediction efficiency through intra-prediction with sub-partitions and the necessary modifications required for these modes. The advantage is relatively high compression efficiency without much additional complexity.

[0075] This application describes various aspects, including tools, features, embodiments, models, methods, and the like. Many of these aspects are described with technicality and in a manner that may be construed as limiting, to at least indicate their individual characteristics. However, this is for ease of explanation and does not limit the applicability or scope of the aspects. In fact, all of the different aspects can be combined and interchanged to provide additional aspects. Furthermore, aspects can likewise be combined and interchanged with aspects described in prior applications.

[0076] The aspects described and contemplated in this application can be implemented in many different forms. While Figures 16, 17, and 18 provide some embodiments, other embodiments are contemplated, and the descriptions of Figures 16, 17, and 18 are not intended to limit the breadth of implementations. At least one of the aspects relates generally to video encoding and decoding, and at least one other aspect relates generally to transmitting generated or encoded bitstreams. These and other aspects can be implemented as a method, an apparatus, a computer-readable storage medium having stored thereon instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium having stored thereon a bitstream generated according to any of the described methods.

[0077] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably. Typically, but not necessarily, the term "reconstructed" is used on the encoder side, while "decoded" is used on the decoder side.

[0078] Various methods are described herein, each of which comprises one or more steps or actions for achieving the described method. Unless a specific order of steps or actions is required for the proper operation of the method, the order and / or use of specific steps and / or actions may be varied or combined.

[0079] Various methods and other aspects described in this application may be used to modify modules such as, for example, the intra-prediction, entropy coding, and / or decoding modules (160, 360, 145, 330) of the video encoder 100 and decoder 200 shown in Figures 16, 17, and 18. Furthermore, the aspects are not limited to VVC or HEVC but may be applied, for example, to other standards and recommendations, whether existing or developed in the future, and extensions of any such standards and recommendations (including VVC and HEVC). Unless otherwise indicated or technically precluded, the aspects described in this application may be used individually or in combination.

[0080] Various numerical values ​​are used in this application. The particular values ​​are for illustrative purposes only and the described aspects are not limited to these particular values.

[0081] 16 shows an encoder 100. Variations of this encoder 100 are envisioned, but for the sake of clarity, the encoder 100 will be described below without describing all possible variations.

[0082] Before being encoded, a video sequence may undergo a pre-encoding process (101), such as applying a color transformation to an input color picture (e.g., from RGB 4:4:4 to YCbCr 4:2:0) or performing a remapping of the input picture components to obtain a signal distribution that is relatively resilient to compression (e.g., using histogram equalization of one of the color components). Metadata can be associated with the pre-processing and can be attached to the bitstream.

[0083] In the encoder 100, a picture is encoded by the encoder elements, as described below. The picture to be encoded is partitioned (102) and processed, for example, in units of CUs. Each unit is encoded, for example, using intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (160). In inter mode, motion estimation (175) and compensation (170) are performed. The encoder decides (105) which of the intra or inter modes to use for encoding the unit, and signals the intra / inter decision, for example, with a prediction mode flag. For example, a prediction residual is calculated by subtracting (110) the predicted block from the original image block.

[0084] The prediction residual is then transformed (125) and quantized (130). To output a bitstream, the quantized transform coefficients, as well as motion vectors and other syntax elements, are entropy coded (145). The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can also bypass both the transform and quantization, i.e., the residual is coded directly without applying a transform or quantization process.

[0085] The encoder decodes the encoded block to provide a reference for further prediction. Quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. Image blocks are reconstructed by combining (155) the decoded prediction residual and the predicted block. For example, an in-loop filter (165) is applied to the reconstructed picture to perform deblocking / Sample Adaptive Offset (SAO) filtering, for example, to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (180).

[0086] Figure 17 shows a block diagram of a video decoder 200. In the decoder 200, the bitstream is decoded by a decoder element, as described below. The video decoder 200 generally performs a decoding pass that is the inverse of the encoding pass, as described in Figure 16. Also, the encoder 100 generally performs video decoding as part of the encoding of the video data.

[0087] Specifically, the decoder's input includes a video bitstream, such as may be generated by video encoder 100. The bitstream is first entropy decoded (230) to obtain transform coefficients, motion vectors, and other coded information. Picture partition information indicates how the picture is partitioned. The decoder then divides the picture according to the decoded picture partitioning information (235). To decode the prediction residual, the transform coefficients are dequantized (240) and inverse transformed (250). Image blocks are reconstructed by combining the decoded prediction residual and the predicted block (255). The predicted block can be obtained from intra prediction (260) or motion-compensated prediction (i.e., inter prediction) (275) (270). An in-loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference picture buffer (280).

[0088] The decoded picture may further undergo a post-decoding process (285), such as an inverse color conversion (e.g., YCbCr 4:2:0 to RGB 4:4:4) or an inverse remapping that performs the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process can use metadata derived in the pre-encoding process and signaled in the bitstream.

[0089] FIG. 18 illustrates a block diagram of an example system in which various aspects and embodiments can be implemented. System 1000 can be implemented as a device including various components, described below, configured to perform one or more of the aspects described herein. Examples of such devices include, without limitation, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television sets, personal video recording systems, connected home appliances, and servers. Elements of system 1000, singly or in combination, can be implemented on a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices, for example, via a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more of the aspects described herein.

[0090] The system 1000 includes at least one processor 1010 configured to execute instructions loaded therein to implement various aspects described herein, for example. The processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. The system 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). The system 1000 includes a storage device 1040, which may include non-volatile and / or volatile memory including, without limitation, electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drives, and / or optical disk drives. Storage devices 1040 may include, by way of non-limiting example, internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0091] System 1000 includes, for example, an encoder / decoder module 1030 configured to process data to provide encoded or decoded video, which may include its own processor and memory. Encoder / decoder 1030 represents one or more modules that may be included within a device to perform encoding and / or decoding functions. As is known, a device may include one or both encoding and decoding modules. Additionally, encoder / decoder module 1030 may be implemented as a separate element of system 1000 or may be integrated within processor 1010 as a combination of hardware and software, as known to those skilled in the art.

[0092] Program code loaded onto the processor 1010 or the encoder / decoder 1030 to perform various aspects described herein may be stored in the storage device 1040 and later loaded onto the memory 1020 for execution by the processor 1010. According to various embodiments, one or more of the processor 1010, the memory 1020, the storage device 1040, and the encoder / decoder module 1030 may store one or more of various items during the operation of the processes described herein. Such stored items may include, without limitation, input video, decoded video or portions of decoded video, bitstreams, matrices, variables, and intermediate or final results resulting from the processing of expressions, formulas, operations, and operational logic.

[0093] In some embodiments, memory internal to the processor 1010 and / or the encoder / decoder module 1030 is used to store instructions and provide working memory for processing required in encoding or decoding, although in other embodiments, memory external to the processing device (e.g., the processing device may be the processor 1010 or the encoder / decoder module 1030) is used for one or more of these functions. The external memory may be, for example, memory 1020 and / or storage device 1040, such as dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used to store, for example, the television's operating system. In at least one embodiment, high-speed external dynamic volatile memory such as RAM is used as working memory for video coding and decoding operations such as MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard is being developed by the Joint Video Experts Team (JVET)).

[0094] Input to the elements of system 1000 can be provided through various input devices, shown in block 1130. Such input devices include, without limitation, (i) a radio frequency (RF) section that receives, for example, an RF signal transmitted wirelessly by a broadcaster, (ii) a component (COMP) input terminal (or set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Other examples not shown in FIG. 18 include composite video.

[0095] In various embodiments, the input devices of block 1130 have associated individual input processing elements known in the art. For example, the RF section may be associated with elements suitable for (i) selecting a desired frequency (and selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting again to a narrow band of frequencies to select a signal frequency band, which in certain embodiments may be referred to as a channel (for example), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF section of various embodiments includes one or more elements for performing these functions, such as, for example, a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF section may include, for example, a tuner that performs various of these functions, including down-converting a received signal to a relatively lower frequency (e.g., an intermediate frequency or a frequency near baseband) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted over a wired (e.g., cable) medium and perform frequency selection by filtering, downconverting, and filtering again to a desired frequency band. Various embodiments rearrange the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include, for example, inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0096] Additionally, the USB and / or HDMI terminals may include respective interface processors for connecting system 1000 to other electronic devices across the USB and / or HDMI connections. For example, it should be understood that various aspects of input processing, such as Reed-Solomon error correction, may be implemented in a separate input processing IC or within processor 1010, as desired. Similarly, aspects of USB or HDMI interface processing may also be implemented in a separate interface IC or within processor 1010, as desired. The demodulated, error corrected, and demultiplexed stream is provided to various processing elements, including processor 1010 and encoder / decoder 1030, operating in combination with memory and storage elements, to process the data stream as desired for presentation on an output device.

[0097] The various elements of system 1000 may be provided within an integrated housing in which the various elements may be interconnected and data may be transmitted therebetween using any suitable connection arrangement, such as an internal bus known in the art, including, for example, an Inter-IC (I2C) bus, wiring, and printed circuit boards.

[0098] System 1000 includes a communication interface 1050 that enables communication with other devices over a communication channel 1060. Communication interface 1050 may include, without limitation, a transceiver configured to transmit and receive data over communication channel 1060. Communication interface 1050 may include, without limitation, a modem or a network card, and communication channel 1060 may be implemented, for example, within a wired and / or wireless medium.

[0099] In various embodiments, data is streamed or otherwise provided to system 1000 using a wireless network, such as a Wi-Fi network, e.g., an IEEE 802.11 (IEEE stands for Institute of Electrical and Electronics Engineers) network. In these embodiments, the Wi-Fi signal is received over communication channel 1060 and communication interface 1050, which are adapted for Wi-Fi communication. In these embodiments, communication channel 1060 is typically connected to an access point or router that provides access to external networks, including the Internet, that allow streaming applications and other over-the-top communications. Other embodiments provide streamed data to system 1000 by using a set-top box that supplies data over the HDMI connection of input block 1130. Still other embodiments provide streamed data to system 1000 by using the RF connection of input block 1130. As noted above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi, such as, for example, a cellular network or a Bluetooth network.

[0100] System 1000 can provide output signals to various output devices, including a display 1100, speakers 1110, and other peripherals 1120. Display 1100 in various embodiments includes, for example, one or more of a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 1100 may be for a television, a tablet, a laptop, a cell phone, or other device. Display 1100 may also be integrated with other components (e.g., as in a smartphone) or may be separate (e.g., an external monitor for a laptop). Other peripherals 1120, in various example embodiments, include one or more of a standalone digital video disc (or digital versatile disc) (DVR, for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripherals 1120 to provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of the system 1000 .

[0101] In various embodiments, control signals are communicated between system 1000 and display 1100, speakers 1110, or other peripheral devices 1120 using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable inter-device control with or without user intervention. Output devices can be communicatively coupled to system 1000 via dedicated connections through individual interfaces 1070, 1080, and 1090. Alternatively, output devices can be connected to system 1000 using communication channel 1060 via communication interface 1050. Display 1100 and speakers 1110 can be integrated with other components of system 1000 in a single unit, for example, in an electronic device such as a television. In various embodiments, display interface 1070 includes a display driver, for example, a timing controller (T Con) chip.

[0102] Display 1100 and speakers 1110 may alternatively be separate from one or more of the other components, for example, if the RF portion of input 1130 is part of a separate set-top box. In various embodiments in which display 1100 and speakers 1110 are external components, the output signal may be provided via a dedicated output connection including, for example, an HDMI port, a USB port, or a COMP output.

[0103] The embodiments may be performed by computer software implemented by the processor 1010, by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, by way of non-limiting examples, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 1010 may be of any type suitable for the technical environment and may include, by way of non-limiting examples, one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture.

[0104] Various implementations involve decoding. As used herein, "decoding" may encompass all or a portion of the processes performed on a received encoded sequence to generate a final output suitable for display, for example. In various embodiments, such processes include one or more of the processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. Additionally or alternatively, in various embodiments, such processes may include processes performed by decoders of various implementations described herein.

[0105] As a further example, in one embodiment, "decoding" refers to entropy decoding only, in another embodiment, "decoding" refers to differential decoding only, and in another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to refer specifically to a subset of operations or to refer generally to the relatively broad decoding process will be clear based on the context of the particular description and is believed to be well understood by those skilled in the art.

[0106] Various implementations involve encoding. Similar to the above description of "decoding," "encoding," as used herein, may encompass, for example, all or a portion of the processes performed on an input video sequence to generate an encoded bitstream. In various embodiments, such processes include one or more of the processes typically performed by an encoder, such as, for example, partitioning, differential encoding, transform, quantization, and entropy encoding. Additionally or alternatively, in various embodiments, such processes also include processes performed by the encoders of the various implementations described herein.

[0107] As a further example, in one embodiment, "encoding" refers only to entropy encoding, in another embodiment, "encoding" refers only to differential encoding, and in another embodiment, "encoding" refers to a combination of differential encoding and entropy encoding. Whether the phrase "encoding process" is intended to refer specifically to a subset of operations or to refer generally to a relatively broad encoding process will be clear based on the context of the particular description and is believed to be well understood by one of ordinary skill in the art.

[0108] It should be noted that the syntax elements used herein are descriptive terms, and therefore do not preclude the use of other syntax element names.

[0109] When a figure is presented as a flow diagram, it should be understood that this also provides a block diagram of the corresponding apparatus, and similarly, when a figure is presented as a block diagram, it should be understood that this also provides a flow diagram of the corresponding method / process.

[0110] Various embodiments may refer to parametric models or rate-distortion optimization. Specifically, the encoding process typically considers a balance or trade-off between rate and distortion, often subject to computational complexity constraints. This can be measured through a rate-distortion optimization (RDO) metric, or through least mean squares (LMS), mean absolute error (MAE), or other such measures. Rate-distortion optimization is typically formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist for solving the rate-distortion optimization problem. For example, an approach may be based on a collective examination of all encoding options, including all considered modes or coding parameter values, with a full evaluation of their coding costs and associated distortions of the reconstructed signal after coding and decoding. Also, relatively fast approaches can be used to reduce encoding complexity, particularly those involving the computation of approximate distortion based on a predicted or prediction residual signal rather than a reconstructed one. It is also possible to use a hybrid of these two approaches, for example, by using approximate distortion for only some of the possible encoding choices and full distortion for other encoding choices. Other approaches evaluate only a subset of the possible encoding choices. More generally, many approaches utilize any of a variety of techniques to perform optimization, but the optimization is not necessarily a complete evaluation of both the coding cost and the distortion involved.

[0111] Implementations and aspects described herein may be implemented in, for example, a method or process, an apparatus, a software program, a data stream, or a signal. Even if described in the context of only a single form of implementation (e.g., described only as a method), the described implementation of the features may also be implemented in other forms (e.g., an apparatus or a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. For example, a method may be implemented in a processor, which generally refers to a processing device including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include, for example, communication devices such as computers, cell phones, handheld / personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end users.

[0112] Reference to "one embodiment" or "an embodiment" or "one implementation" or "an implementation," as well as other variations thereof, means that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment. Thus, the appearances of the phrases "in one embodiment" or "in one embodiment" or "in one implementation" or "in one implementation" in various places throughout this application, as well as any other variations thereof, are not necessarily all referring to the same embodiment.

[0113] Additionally, the application may refer to "determining" various pieces of information. Determining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from memory.

[0114] Additionally, the application may refer to "accessing" various pieces of information. Accessing information may include, for example, one or more of receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or inferring information.

[0115] Additionally, the application may refer to "receiving" various pieces of information. Receiving, like "accessing," is intended to be a broad term. Receiving information may include, for example, one or more of accessing information or retrieving information (e.g., from memory). Furthermore, "receiving" typically involves, in one manner or another, an action such as storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or inferring information.

[0116] For example, it should be understood that the use of any of " / ," "and / or," and "at least one of" in the cases of "A / B," "A and / or B," and "at least one of A and B" is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such phrases are intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), or the selection of only the first and third listed alternatives (A and C), or the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be extended for as many items as are listed, as will be apparent to those skilled in this and related arts.

[0117] Also, as used herein, the term "signal" means, among other things, informing a corresponding decoder of something. For example, in certain embodiments, an encoder signals a particular one of multiple transforms, coding modes, or flags. As a result, in one embodiment, the same transform, parameter, or mode is used at both the encoder and decoder sides. Thus, for example, the encoder can transmit a particular parameter to the decoder so that the decoder can use the same particular parameter (explicit signaling). Conversely, if the decoder already has not only a particular parameter but also others, signaling can be used without transmission to simply allow the decoder to know and select the particular parameter (implicit signaling). By avoiding the transmission of any actual function, bit savings are realized in various embodiments. It should be understood that signaling can be realized in various manners. For example, one or more syntax elements, flags, etc. are used to signal information to a corresponding decoder in various embodiments. While the above relates to the verb form of the term "signal," the term "signal" may also be used herein as a noun.

[0118] As will be apparent to those skilled in the art, implementations can generate a variety of signals formatted to carry information that can be, for example, stored or transmitted. Information can include, for example, instructions for performing a method or data generated by one of the described implementations. For example, a signal can be formatted to carry a bitstream of the described embodiments. Such a signal can be formatted, for example, as an electromagnetic wave (e.g., using a high-frequency portion of the spectrum) or as a baseband signal. Formatting can include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal can be, for example, analog or digital information. The signal can be transmitted over a variety of different wired or wireless links, as is known. The signal can be stored on a processor-readable medium.

[0119] The inventors have described several embodiments across various claim categories and types. Features of these embodiments may be provided alone or in any combination. Furthermore, embodiments may include one or more of the following features, devices, or aspects across various claim categories and types, alone or in any combination. Process or apparatus for performing intra-encoding and decoding with intra-coding sub-partitions Process or apparatus for performing intra-encoding and decoding with intra-coding sub-partitions using multiple reference lines Process or apparatus for performing intra-encoding and decoding with intra-coding sub-partitions and missing sample replacement Process or apparatus for performing intra-encoding and decoding with mapping of reference pixels above intra-coding sub-partitions and parent video blocks Intra-coding sub-partitions and process or apparatus for performing intra-encoding and decoding with unequal sub-partition sizes A bitstream or signal containing one or more of the described syntax elements or their variants A bitstream or signal containing syntax conveying information generated according to any of the described embodiments. · generating and / or transmitting and / or receiving and / or decoding according to any of the described embodiments A method, process, apparatus, medium, instruction storage medium, data storage medium, or signal according to any of the described embodiments. Insertion in signaling syntax elements that allow the decoder to determine the coding mode in a manner that corresponds to that used by the encoder · Generating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variants thereof A TV, set-top box, cell phone, tablet, or other electronic device that performs one or more conversion methods according to any of the described embodiments. A TV, set-top box, cell phone, tablet, or other electronic device that performs the transformation method determination according to any of the described embodiments and displays the resulting image (e.g., using a monitor, screen, or other type of display). A TV, set-top box, cell phone, tablet, or other electronic device that selects, bandlimits, or tunes (e.g., using a tuner) a channel to receive a signal containing an encoded image and performs one or more conversion methods according to any of the described embodiments. A TV, set-top box, cell phone, tablet, or other electronic device that receives a signal containing the encoded image wirelessly (e.g., using an antenna) and performs one or more conversion methods.

Claims

1. 1. A method comprising: determining an intra-prediction direction for rectangular sub-partitions of a video block based on dimensions of the sub-partitions when a ratio of the dimensions of the sub-partitions is within a particular range, and based on dimensions of the video block when a ratio of the dimensions of the sub-partitions is outside the particular range; mapping reference samples from the upper right or lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, wherein missing reference samples are mapped from the video block; predicting samples of the rectangular sub-partition using reference samples from a row above the video block or a column to the left of the video block, wherein a number of reference samples in the row above the video block or the column to the left of the video block is determined based on a dimension of the rectangular sub-partition; encoding the rectangular sub-partition of a video block using the prediction in an intra-coding mode; and A method having the following.

2. 1. An apparatus comprising: determining an intra-prediction direction for rectangular sub-partitions of a video block based on dimensions of the sub-partitions when a ratio of the dimensions of the sub-partitions is within a particular range, and based on dimensions of the video block when a ratio of the dimensions of the sub-partitions is outside the particular range; mapping reference samples from the upper right or lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, wherein missing reference samples are mapped from the video block; predicting samples of the rectangular sub-partition using reference samples from a row above the video block or a column to the left of the video block, wherein a number of reference samples in the row above the video block or the column to the left of the video block is determined based on a dimension of the rectangular sub-partition; encoding the rectangular sub-partition of a video block using the prediction in an intra-coding mode; and 1. An apparatus having a processor configured to execute

3. 1. A method comprising: determining an intra-prediction direction for rectangular sub-partitions of a video block based on dimensions of the sub-partitions when a ratio of the dimensions of the sub-partitions is within a particular range, and based on dimensions of the video block when a ratio of the dimensions of the sub-partitions is outside the particular range; mapping reference samples from the upper right or lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, wherein missing reference samples are mapped from the video block; predicting samples of the rectangular sub-partition using reference samples from a row above the video block or a column to the left of the video block, wherein a number of reference samples in the row above the video block or the column to the left of the video block is determined based on a dimension of the rectangular sub-partition; decoding the rectangular sub-partition of a video block using the prediction in an intra-coding mode; A method having the following.

4. 1. An apparatus comprising: determining an intra-prediction direction for rectangular sub-partitions of a video block based on dimensions of the sub-partitions when a ratio of the dimensions of the sub-partitions is within a particular range, and based on dimensions of the video block when a ratio of the dimensions of the sub-partitions is outside the particular range; mapping reference samples from the upper right or lower left of the video block as upper right or lower left reference samples for prediction of the sub-partition of the video block, wherein missing reference samples are mapped from the video block; and predicting samples of the rectangular sub-partition using reference samples from a row above the video block or a column to the left of the video block, wherein a number of reference samples in the row above the video block or the column to the left of the video block is determined based on a dimension of the rectangular sub-partition; decoding the rectangular sub-partition of a video block using the prediction in an intra-coding mode; 1. An apparatus having a processor configured to execute

5. The method of claim 1 , wherein a non-wide angle prediction mode is applied to sub-partitions that are square.

6. 2. The method of claim 1, wherein missing reference samples are mapped from the video block for subpartitions beyond the one closest to the left reference array in the case of vertical subpartitions or beyond the one closest to the top edge in the case of horizontal subpartitions.

7. The method or apparatus of claim 6 , wherein the intra-prediction direction is based on a dimension of the video block for sub-partitions having a width / height ratio greater than 16 or less than 1 / 16.

8. 5. The apparatus of claim 4, wherein the intra-prediction direction is based on the dimensions of the video block for subpartitions having a width / height ratio greater than 16 or less than 1 / 16, and wherein missing reference samples are directly mapped from the video block in the horizontal or vertical direction for subpartitions beyond the one closest to the left reference array for vertical subpartitions or the one closest to the top edge for horizontal subpartitions.

9. The apparatus of claim 6 , wherein a sub-partition is at least a single row or a single column of the video block.

10. 5. The apparatus of claim 4, wherein the subpartitions have non-square shapes but have heights and widths that are equal to a power of two.

11. The apparatus of claim 4 , wherein a sub-partition can use multiple reference lines.

12. 1. An apparatus comprising: An apparatus according to claim 4; at least one of: (i) an antenna configured to receive a signal, the signal including the video block; (ii) a band limiter configured to limit the received signal to a frequency band including the video block; and (iii) a display configured to display an output representing the video block; A device having:

13. A computer program having instructions which, when executed by a computer, cause the computer to carry out the method of claim 3.

14. The method of claim 3 , wherein a non-wide angle prediction mode is applied to sub-partitions that are square.

15. 4. The method of claim 3, wherein missing reference samples are mapped from the video block for subpartitions beyond the one closest to the left reference array in the case of vertical subpartitions or beyond the one closest to the top edge in the case of horizontal subpartitions.

16. 5. The apparatus of claim 4, wherein the intra-prediction direction is based on the dimensions of the video block for subpartitions having a width / height ratio greater than 16 or less than 1 / 16, and wherein missing reference samples are directly mapped from the video block in the horizontal or vertical direction for subpartitions beyond the one closest to the left reference array for vertical subpartitions or the one closest to the top edge for horizontal subpartitions.

17. 5. The apparatus of claim 4, wherein the subpartitions have unequal shapes but have heights and widths that are equal to a power of two.

18. The apparatus of claim 4 , wherein a sub-partition can use multiple reference lines.

Images