Methods and computer readable memories or storage devices for video encoding, decoding

Abstract

Methods and computer readable memory or storage for video encoding, decoding. A method of video encoding includes defining, within an encoding region of a video frame, a rectangular coding unit CU having CU x and CU y coordinates based on coding tree units including quad-tree partitioning; selectively defining side and primary reference pixels, selectively determining primary and side weight values, generating a prediction CU if the CU is to be encoded using weighted angular prediction for intra prediction of a particular angle, filtering the prediction CU based on post-generation filtering and delivering the filtered prediction CU for entropy encoding; generating a prediction CU based at least in part on another prediction if the CU is not to be encoded using weighted angular prediction for intra prediction of the particular angle; the generating of the prediction CU based at least in part on the other prediction is not based on at least one of a weighted primary reference pixel and a weighted side reference pixel; the prediction CU is not filtered based on post-generation filtering; delivering the prediction CU, which is not filtered, for entropy encoding.

Description

[0001] This application is a divisional application of PCT application number PCT / US2018 / 040862, international application date July 5, 2018, Chinese application number 201880045213.2, entitled "Post-filtering for weighted angle prediction".

[0002] Cross-references to related applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 528,724, filed earlier on July 5, 2017, under 35 U.S.SC §119(e), which is incorporated herein by reference. Technical Field

[0004] This disclosure relates to the field of video coding, and more particularly to improving coding efficiency by modifying the intra-prediction coding process using weighted angle prediction coding, thereby achieving higher bit rates, resolutions and better quality video. Background Technology

[0005] Technological improvements in evolving video coding standards illustrate the trend towards higher coding efficiency to achieve higher bit rates, higher resolutions, and better video quality. The Joint Video Exploration Group (JVET) is developing a new video coding scheme called JVET. Similar to other video coding schemes such as HEVC (High Efficiency Video Coding), JVET is a block-based hybrid spatiotemporal prediction coding scheme. However, compared to HEVC, JVET includes numerous modifications to the bitstream structure, syntax, constraints, and mappings for generating decoded pictures. JVET has been implemented in the Joint Exploration Model (JEM) encoder and decoder, which utilizes various coding techniques, including weighted angular prediction.

[0006] In the current JVET design, three steps are involved in generating the final intra-predictor block: pre-filtering, predictor generation, and post-filtering. Typically, pre-filtering applies filtering operations such as mode-dependent intra-smoothing or MDIS to neighboring pixels to prepare for the predictor generation step. Then, the predictor generation step computes predictor pixels according to rules associated with the selected intra-prediction mode. Finally, post-filtering applies filtering operations such as boundary smoothing to predictor pixels along the block boundaries to reduce discontinuities across block boundaries. Summary of the Invention

[0007] This disclosure provides a method for encoding a JVET bitstream, the method comprising: defining a coding unit (CU) having CU x and CU y coordinates within a coding region of a video frame; then defining a primary reference pixel within the coding region having primary x and primary y coordinates associated with the primary reference; and defining a side reference pixel within the coding region having side x and side y coordinates associated with the side reference. Additionally, a primary weight value associated with the primary reference pixel can be determined, and a side weight value associated with the side reference pixel can also be determined. The method can then generate a predicted CU for the coding unit based at least in part on a combination of the primary reference pixel combined with the primary weight value and the side reference pixel combined with the side weight value.

[0008] In some embodiments, post-predictive filtering can be applied unbiasedly relative to the main reference pixel and / or the side reference pixel. In alternative embodiments, post-predictive filtering can be bypassed or an all-pass filter can be applied. Attached Figure Description

[0009] Further details of the invention are explained with the aid of the accompanying drawings, in which:

[0010] Figure 1 It describes dividing a frame into multiple coding tree units (CTUs).

[0011] Figure 2 An example partitioning of the CTU into a coding unit (CU) is depicted using quadtree partitioning and symmetric binary partitioning.

[0012] Figure 3 Depicting Figure 2 The partition is represented by a quadtree plus binary tree (QTBT).

[0013] Figure 4 The asymmetric binary partitioning of the CU into two smaller CUs is described.

[0014] Figure 5 Example partitioning of CTU to CU using quadtree partitioning, symmetric binary partitioning, and asymmetric binary partitioning is described.

[0015] Figure 6 Depicting Figure 5 The QTBT representation of the partition.

[0016] Figure 7A and 7B A simplified block diagram is depicted for CU encoding in the JVET encoder.

[0017] Figure 8 Sixty-seven possible intra-frame prediction modes for the luminance component in JVET were described.

[0018] Figure 9 A simplified block diagram is depicted for CU decoding in a JVET encoder.

[0019] Figure 10 An embodiment of a method for CU encoding in a JVET encoder is described.

[0020] Figure 11 A simplified block diagram is depicted for CU encoding in the JVET encoder.

[0021] Figure 12 A simplified block diagram is depicted for CU decoding in the JVET decoder.

[0022] Figure 13 Embodiments of computer systems suitable for and / or configured to process CU encoding methods are described.

[0023] Figure 14 An embodiment of an encoder / decoder system for CU encoding / decoding in a JVET encoder / decoder is described. Detailed Implementation

[0024] Figure 1 The diagram describes dividing a frame into multiple coding tree units (CTUs) 100. A frame can be an image in a video sequence. A frame can include a matrix or a set of matrices having pixel values ​​representing intensity measures in the image. Therefore, the set of these matrices can generate a video sequence. Pixel values ​​can be defined as representing color and luminance in panchromatic video coding, where a pixel is divided into three channels. For example, in the YCbCr color space, a pixel can have a luminance value Y representing the grayscale intensity in the image and two chrominance values ​​Cb and Cr representing the degree of difference between colors from gray to blue and red. In other embodiments, pixel values ​​can be represented using values ​​in different color spaces or models. The resolution of the video determines the number of pixels in a frame. Higher resolution may mean more pixels and better image sharpness, but may also lead to higher bandwidth, storage, and transmission requirements.

[0025] JVET can be used to encode and decode frames of a video sequence. JVET is a video coding scheme developed by the Joint Video Exploration Group. Versions of JVET have been implemented in the JEM (Joint Exploration Model) encoder and decoder. Similar to other video coding schemes (such as HEVC (High Efficiency Video Coding)), JVET is a block-based hybrid spatiotemporal predictive coding scheme. During encoding with JVET, the frame is first divided into square blocks called CTU 100, such as... Figure 1 As shown. For example, CTU 100 can be a block of 128×128 pixels.

[0026] Figure 2An exemplary partitioning of CTU 100 into CU 102 is depicted. Each CTU 100 in a frame can be divided into one or more CUs (coding units) 102. CU 102 can be used for prediction and transform as described below. Unlike HEVC, in JVET, CU 102 can be rectangular or square and can be encoded without further partitioning into prediction or transform units. CU 102 can be as large as their root CTU 100, or can be a smaller subdivision of the root CTU 100, as small as a 4×4 block.

[0027] In JVET, CTU 100 can be divided into CU 102 using a Quadtree Plus Binary Tree (QTBT) scheme. In this scheme, CTU 100 is recursively split into square blocks based on the quadtree, and these square blocks can then be recursively split horizontally or vertically based on the binary tree. Parameters can be set to control the splitting based on QTBT, such as the CTU size, the minimum size of leaf nodes in the quadtree and binary tree, the maximum size of the root node in the binary tree, and the maximum depth of the binary tree.

[0028] In some embodiments, JVET may restrict the binary partitioning in the binary tree portion of the QTBT to symmetrical partitioning, wherein the block may be divided into two halves vertically or horizontally along the midline.

[0029] By way of non-restrictive examples, Figure 2 The diagram shows a CTU 100 divided into CUs 102, where solid lines indicate quadtree splits and dashed lines indicate symmetric binary tree splits. As shown, binary splits allow for symmetric horizontal and vertical splits to define the structure of the CTU and its subdivisions into CUs.

[0030] Figure 3 Show Figure 2 The partition is represented by a QTBT. The root node of the quadtree represents CTU 100, and each child node in the quadtree section represents one of the four square blocks split from the parent square block. The square blocks represented by the leaf nodes of the quadtree can then be symmetrically partitioned zero or more times using a binary tree, where the leaf node of the quadtree is the root node of the binary tree. At each level of the binary tree section, blocks can be partitioned symmetrically either vertically or horizontally. A flag set to "0" indicates that the block is partitioned symmetrically in the horizontal direction, while a flag set to "1" indicates that the block is partitioned symmetrically in the vertical direction.

[0031] In other embodiments, JVET may allow symmetric or asymmetric binary partitioning within the binary tree portion of the QTBT. Asymmetric motion partitioning (AMP) is allowed in different contexts within HEVC when partitioning prediction units (PUs). However, for the case of partitioning CU 102 in JVET according to the QTBT structure, asymmetric binary partitioning can result in improved partitioning compared to symmetric binary partitioning when the relevant region of CU 102 is not located on either side of the midline passing through the center of CU 102. By way of a non-limiting example, when CU 102 depicts an object near the center of CU and another object on the side of CU 102, CU 102 can be asymmetrically partitioned to place each object in a separate, smaller CU 102 of a different size.

[0032] Figure 4 Four possible types of asymmetric binary partitioning are depicted, in which CU 102 is divided into two smaller CU 102 along a line spanning the length or height of CU 102, such that one of the smaller CU 102 is 25% of the size of the parent CU 102, and the other is 75% of the size of the parent CU 102. Figure 4 The four types of asymmetric binary partitions shown allow CU 102 to split along lines from 25% of the left side of CU 102, 25% of the right side of CU 102, 25% of the top of CU 102, or 25% of the bottom of CU 102. In alternative embodiments, the asymmetric partition lines at the split points of CU 102 can be located in any other position, such that CU 102 is not symmetrically divided into two halves.

[0033] Figure 5 A non-restrictive example of CTU 100 partitioned into CU 102 is depicted using a scheme that allows both symmetric and asymmetric binary partitioning in the binary tree portion of the QTBT. Figure 5 In the diagram, the dashed lines represent asymmetric binary partition lines, where... Figure 4 One of the partitioning types shown is used to partition the parent CU 102.

[0034] Figure 6 It shows Figure 5 The QTBT representation of the partition. Figure 6 In the diagram, two solid lines extending from a node indicate a symmetrical partition in the binary tree portion of the QTBT, while two dashed lines extending from a node indicate an asymmetrical partition in the binary tree portion.

[0035] A syntax indicating how CTU 100 is divided into CU 102 can be encoded in the bitstream. By way of a non-restrictive example, a syntax indicating which nodes are partitioned using a quadtree partition, which nodes are partitioned using a symmetric binary partition, and which nodes are partitioned using an asymmetric binary partition can be encoded in the bitstream. Similarly, a syntax indicating the type of asymmetric binary partition used can be encoded in the bitstream of nodes split using an asymmetric binary partition, for example... Figure 4 One of the four types shown.

[0036] In some embodiments, the use of asymmetric partitioning may be limited to partitioning CU 102 at the leaf nodes of the quadtree portion of the QTBT. In these embodiments, the CU 102 at the child nodes split from the parent node using quadtree partitioning in the quadtree portion may be the final CU 102, or they may be further split using quadtree partitioning, symmetric binary partitioning, or asymmetric binary partitioning. Child nodes in the binary tree portion split using symmetric binary partitioning may be the final CU 102, or they may be recursively split once or multiple times using only symmetric binary partitioning. Child nodes in the binary tree portion split from the QT leaf nodes using asymmetric binary partitioning may be the final CU 102, and further splitting is not allowed.

[0037] In these embodiments, restricting the use of asymmetric partitioning to splitting quadtree leaf nodes reduces search complexity and / or limits overhead bits. Because only quadtree leaf nodes can be split using asymmetric partitioning, using asymmetric partitioning directly indicates the end of a QT partial branch without additional syntax or further signaling. Similarly, since nodes with asymmetric partitions cannot be further split, using asymmetric partitioning on a node also directly indicates that its asymmetric partitioned child nodes are the final CU 102 without additional syntax or further signaling.

[0038] In alternative embodiments, for example when there are fewer considerations regarding limiting search complexity and / or limiting the number of overhead bits, asymmetric partitioning can be used to split nodes generated by quadtree partitioning, symmetric binary partitioning, and / or asymmetric binary partitioning.

[0039] After quadtree and binary tree splitting using any of the QTBT structures described above, the blocks represented by the leaf nodes of the QTBT represent the final CU 102 to be encoded, for example, using inter-prediction or intra-prediction coding. For slices or full frames coded with inter-prediction, different partitioning structures can be used for the luma and chroma components. For example, for inter-slice, the CU 102 may have coded blocks (CBs) for different color components, such as one luma CB and two chroma CBs. For slices or full frames coded with intra-prediction, the partitioning structure can be the same for both luma and chroma components.

[0040] In an alternative embodiment, JVET can use a two-level coding block structure as an alternative to or extension of the QTBT partitioning described above. In the two-level coding block structure, CTU 100 can first be partitioned into basic units (BUs) at a higher level. Then, the BUs can be partitioned into operation units (OUs) at a lower level.

[0041] In embodiments employing a two-level coding block structure, at the higher level, the CTU 100 can be divided into BUs according to one of the QTBT structures described above or according to a quadtree (QT) structure such as that used in HEVC, wherein a block can only be divided into four sub-blocks of equal size. By way of non-limiting examples, it is possible to divide the CTU 100 into BUs according to the above... Figure 5-6 The described QTBT structure partitions CTU 102 into BUs, allowing leaf nodes in the quadtree portion to be split using quadtree partitioning, symmetric binary partitioning, or asymmetric binary partitioning. In this example, the final leaf node of the QTBT can be a BU instead of a CU.

[0042] At the lower level of the two-level coded block structure, each BU (Block Entity) partitioned from CTU 100 can be further divided into one or more OUs. In some embodiments, when a BU is square, it can be split into OUs using quadtree partitioning or binary partitioning (e.g., symmetric or asymmetric binary partitioning). However, when a BU is not square, it can only be split into OUs using binary partitioning. Restricting the partition types available for non-square BUs limits the number of bits used to signal the partition type used to generate the BU.

[0043] Although the discussion below describes encoding CU 102, in embodiments using a two-level coded block structure, BUs and OUs can be encoded instead of CU 102. By way of non-limiting example, BUs can be used for higher-level coding operations such as intra-frame prediction or inter-frame prediction, while smaller OUs can be used for lower-level coding operations such as transforms and generating transform coefficients. Thus, the syntax to be encoded for BUs indicates whether they are encoded with intra-frame prediction or inter-frame prediction, or information identifies the specific intra-frame prediction mode or motion vector used to encode the BU. Similarly, the syntax for OUs can identify the specific transform operations or quantized transform coefficients used to encode the OU.

[0044] Figure 7A A simplified block diagram for CU encoding in a JVET encoder is depicted. The main stages of video encoding include the partitioning described above to identify CU 102, followed by encoding CU 102 using prediction at 704 or 706, generating residual CU 710 at 708, performing transform at 712, quantizing at 716, and entropy coding at 720. Figure 7A The encoder and encoding process shown also include the decoding process, which is described in more detail below.

[0045] Given the current CU 102, the encoder can obtain a predicted CU 702 by using intra-frame prediction spatially at 704 or inter-frame prediction temporally at 706. The basic idea of ​​predictive coding is to send a differential or residual signal between the original signal and its prediction. On the receiver side, the original signal can be reconstructed by adding the residual and the prediction, as described below. Since the differential signal has lower correlation than the original signal, it requires fewer bits to transmit.

[0046] A slice (such as an entire picture or a portion of a picture) fully encoded with intra-frame prediction using CU 102 can be an I-slice, which can be decoded without reference to other slices, and thus can be a possible point where decoding can begin. A slice encoded with at least some inter-frame prediction using CU can be a prediction (P) or bidirectional prediction (B) slice that can be decoded based on one or more reference pictures. A P-slice can use intra-frame and inter-frame prediction with previously encoded slices. For example, by using inter-frame prediction, a P-slice can be further compressed than an I-slice, but requires encoding of the previously encoded slices to encode them. A B-slice can use intra-frame or inter-frame prediction that utilizes interpolation prediction from two different frames, using data from previous and / or subsequent slices for its encoding, thereby improving the accuracy of the motion estimation process. In some cases, P-slices and B-slices can also be encoded or alternatively encoded using intra-block duplication, where data from other parts of the same slice is used.

[0047] As will be discussed below, intra-frame prediction or inter-frame prediction can be performed based on the reconstructed CU 734 from previously encoded CU 102 (such as adjacent CU 102 or CU 102 in the reference image).

[0048] When spatially coding CU 102 using intra-frame prediction in 704, an intra-frame prediction pattern can be found that optimally predicts the pixel values ​​of CU 102 based on samples from neighboring CU 102 in the image.

[0049] When encoding the luma component of the CU, the encoder can generate a list of candidate intra-prediction modes. While HEVC has 35 possible intra-prediction modes for the luma component, JVET has 67. These include a planar mode using a three-dimensional plane of values ​​generated from neighboring pixels, a DC mode using values ​​averaged from neighboring pixels, and a mode using values ​​copied from neighboring pixels along an indicated direction. Figure 8 The 65 orientation patterns shown.

[0050] When generating a list of candidate intra-prediction modes for the luma component of the CU, the number of candidate modes in the list can depend on the size of the CU. The candidate list may include: a subset of the 35 modes of HEVC with the lowest SATD (Sum of Absolute Transform Differences) cost; new directional modes added for JVET that are adjacent to candidates found from HEVC modes; and modes from the set of six most probable modes (MPMs) of CU 102 identified based on intra-prediction modes used for previously encoded neighboring blocks and a default mode list.

[0051] When encoding the chroma components of a CU, a list of candidate intra-prediction modes can also be generated. This list can include: modes generated using cross-component linear model projection from luma samples; intra-prediction modes found for luma CBs at specific juxtaposition locations within a chroma block; and chroma prediction modes previously found for adjacent blocks. The encoder can find the candidate modes with the lowest rate distortion cost from the list and use these intra-prediction modes when encoding the luma and chroma components of the CU. A syntax indicating the intra-prediction modes used for encoding each CU 102 can be encoded in the bitstream.

[0052] After selecting the optimal intra-frame prediction mode for CU 102, the encoder can use those modes to generate predictions for CU 402. When the selected mode is a directional mode, a 4-tap filter can be used to improve directional accuracy. Boundary prediction filters (such as 2-tap or 3-tap filters) can be used to adjust the top or left columns or rows of the prediction block.

[0053] The prediction CU 702 can be further smoothed using a position-dependent intra prediction combination (PDPC) process that adjusts the prediction CU 702 generated based on smoothing of filtered samples of adjacent blocks using unfiltered samples of adjacent blocks or adaptive reference samples using a 3-tap or 5-tap low-pass filter to process the reference samples in step 705b. In some embodiments, PDPC can be implemented according to equation (1) below:

[0054] P’[x,y]=((A*Recon[x,-1]-B*Recon[-1,-1]+C*Recon[-1,y]+D*P[x,y]+Round) / Denom

[0055] Equation (1)

[0056] where A = (Cv1 >> int(y / dy)), B = ((Cv2 >> int(y / dy)) + (Ch2 >> int(x / dx))), C = (Ch1 >> int(x / dx)) and D = (1 << Denom) – A – C + B. Thus, P'[x, y] is the filtered pixel after the post-filtering operation at the coordinates (x, y) of the current CU. Cv1, Cv2, Ch1, and Ch2 are PDPC parameters that determine the filtering effect, "Round" is the rounding parameter, and "Denom" is the normalization factor.

[0057] In some embodiments, weighted angular prediction can be employed, which uses pixels at the projected positions on both the top reference row and the left reference column to generate predictor pixels for angular prediction. In embodiments employing weighted angular prediction, prediction generation can be completed in three steps: primary reference projection prediction, side reference projection prediction, and combination of projection predictions.

[0058] In some embodiments employing weighted angular prediction, the system and method can define along the primary reference projected pixel positions according to the angular direction of the coded intra prediction mode and use linear interpolation between two adjacent reconstructed pixels to determine the pixel values at the projected positions. The system and method can also define along the side reference projected pixel positions according to the angle of the same coding mode and use linear interpolation between two adjacent reconstructed pixels to determine the pixel values at the projected positions. Then, the system and method can combine the projected pixel values of the primary reference with those of the side reference. A non-limiting exemplary combination is shown in equation (2) below. In the exemplary combination shown in equation (2), the values are weighted according to the distance between the predictor pixels and the projected pixel positions on the primary reference and the side reference. However, in alternative embodiments, alternative values can be used to weight the values associated with the primary reference pixels and the side reference pixels.

[0059] P[x,y]=(((w1*MainRecon[x',y'])+(w2*SideRecon[x”,y”])+(w1+w2) / 2) / (w1+w2))

[0060] Equation (2)

[0061] In the exemplary equation (1) above, MainRecon[x', y'] is the neighboring pixel value along the main reference at the projection position (x', y') corresponding to the predicted pixel (x, y). SideRecon[x”, y”] is the neighboring pixel value along the side reference at the projection position (x”, y”) corresponding to the neighboring pixel (x, y).

[0062] Equation (3) below shows a non-limiting exemplary combination of using weighted angle prediction (which uses HEVC mode 2 or mode 66) and the predictor pixel at coordinates (x, y). Thus, P[x, y] will be determined as shown and stated in Equation (3), where Recon[0, 0] is the reconstructed pixel at the top-left coordinate (0, 0) of the current CU.

[0063] P[x,y]=((((x+1)*Recon[x+y+2,-1])+((y+1)*(Recon[-1,x+y+2]))+(y+x+2) / 2) / (y+x+2))

[0064] Equation (3)

[0065] When the projected reference position on the side reference points to a non-feasible or unavailable reconstructed position, an exception may occur in the system and process in which weighted angle prediction may not be employed. In such cases, there may be multiple options for handling the exception when weighted angle prediction may not be employed. In some embodiments, the exception may be handled by using the value of the last available reconstructed pixel or a default value of the projected position. In other alternative embodiments, the exception may be handled by disabling weighted angle prediction and / or using only the projected pixel position of the master reference. Therefore, in step 705a, it can be determined whether weighted angle prediction has been used as the intra-prediction mode in step 704. If the intra-prediction mode is determined to be using weighted angle prediction in step 705a, the prediction coding unit 702 can be delivered for entropy coding without filtering. However, if it is determined in step 705a that the intra-prediction mode is not weighted angle prediction, a post-intra-prediction filter 705b (e.g., PDPC filter) can be applied to the prediction coding unit before delivery for entropy coding.

[0066] like Figure 7B As shown, in some embodiments, a post-intra-prediction filter 705b may be applied for all intra-prediction after step 704. Figure 7B In such embodiments, if the intra-frame prediction mode is based on something other than weighted angle prediction, the applied filter can be applied as normally applied in step 705b. However, if the intra-frame prediction mode is based on weighted angle prediction, the filtering in step 705b can be bypassed and / or in some embodiments, the applied filter may not be biased towards the primary reference, side reference, or both. By way of non-limiting example, the values ​​of Cv1 and Ch1 can be equal and / or the values ​​of Cv2 and Ch2 can be equal.

[0067] When inter-frame prediction is used to temporally encode CU 102 at 706, a set of motion vectors (MVs) can be found that point to samples in the reference image that best predict the pixel values ​​of CU 102. Inter-frame prediction utilizes the temporal redundancy between slices by representing the displacement of pixel blocks within a slice. The displacement is determined based on the pixel values ​​in the previous or next slice through a process called motion compensation. The motion vectors indicating the displacement of a pixel relative to a specific reference image and the associated reference index can be provided to the decoder in the bitstream along with the residual between the original pixel and the motion-compensated pixel. The decoder can then reconstruct the pixel blocks in the reconstructed slice using the residual, the motion vectors transmitted as signals, and the reference index.

[0068] In JVET, the precision of motion vectors can be stored in 1 / 16 pixel increments, and the difference between the motion vector and the predicted motion vector of the CU can be encoded in either quarter-pixel or integer pixel resolution.

[0069] In JVET, techniques such as Advanced Temporal Motion Vector Prediction (ATMVP), Spatiotemporal Motion Vector Prediction (STMVP), Affine Motion Compensation Prediction, Pattern Matching Motion Vector Derivation (PMMVD), and / or Bidirectional Optical Flow (BIO) can be used to find motion vectors for multiple sub-CUs in CU 102.

[0070] Using ATMVP, the encoder can find the time vector of CU 102, which points to the corresponding block in the reference image. The time vector can be found based on the motion vectors found for previously encoded neighboring CUs 102 and the reference image. Using the reference block pointed to by the time vector for the entire CU 102, motion vectors can be found for each sub-CU within the entire CU 102.

[0071] STMVP can find the motion vector of a sub-CU by scaling and averaging the motion vectors found for neighboring blocks previously coded with inter-frame prediction, as well as the time vector.

[0072] Affine motion compensation prediction can be used to predict the field of motion vectors for each sub-CU in a block based on two control motion vectors found for the vertices of the block. For example, the motion vectors of sub-CUs can be derived based on the vertices motion vectors found for each 4x4 block within CU 102.

[0073] PMMVD can use bilateral matching or template matching to find the initial motion vector of the current CU 102. Bilateral matching can examine the current CU 102 and reference blocks in two different reference images along the motion trajectory, while template matching can examine the current CU 102 and the corresponding blocks in the reference images identified by the template. The initial motion vector found for CU 102 can then be refined for each sub-CU.

[0074] When performing inter-frame prediction using bidirectional prediction based on earlier and later reference images, BIO can be used, and BIO allows motion vectors to be found for sub-CUs based on the gradients of the differences between the two reference images.

[0075] In some cases, the values ​​of the scaling factor and offset parameters can be found at the CU level using Local Illumination Compensation (LIC) based on samples adjacent to the current CU 102 and corresponding samples adjacent to the reference block identified by the candidate motion vectors. In JVET, the LIC parameters can be changed and transmitted as signals at the CU level.

[0076] For some of the methods described above, the motion vectors found for each sub-CU of a CU can be sent as signals to the decoder at the CU level. For other methods (such as PMMVD and BIO), motion information is not sent as signals in the bitstream to save overhead, and the decoder can derive the motion vectors through the same process.

[0077] After finding the motion vectors for CU 102, the encoder can use those motion vectors to generate the prediction CU 702. In some cases, when motion vectors for a single sub-CU have already been found, overlapping block motion compensation (OBMC) can be used when generating the prediction CU 702 by combining those motion vectors with those previously found for one or more neighboring sub-CUs.

[0078] When using bidirectional prediction, JVET can use decoder-side motion vector refinement (DMVR) to find motion vectors. DMVR allows the use of a bilateral template matching process to find motion vectors based on the two motion vectors found for bidirectional prediction. In DMVR, a weighted combination of the predicted CU 702 generated by each of the two motion vectors can be found, and they can be refined by replacing the two motion vectors with a new motion vector representing the predicted CU 702 of the optimal pointing combination. These two refined motion vectors can then be used to produce the final predicted CU 702.

[0079] At 708, as described above, once the prediction CU 702 has been found using intra-frame prediction at 704 or inter-frame prediction at 706, the encoder can subtract the prediction CU 702 from the current CU 102 to find the residual CU 710.

[0080] The encoder can use one or more transform operations at 712 to convert the residual CU 710 into transform coefficients 714 that express the residual CU 710 in the transform domain, such as using Discrete Cosine Block Transform (DCT transform) to transform the data into the transform domain. Compared to HEVC, JVET allows for more types of transform operations, including DCT-II, DST-VII, DST-VII, DCT-VIII, DST-I, and DCT-V operations. The allowed transform operations can be grouped into subsets, and the encoder can signal indications of which subsets were used and which specific operations within those subsets. In some cases, large-block-size transforms can be used to zero out high-frequency transform coefficients in CU 102 larger than a certain size, so that only those CU 102 retain low-frequency transform coefficients.

[0081] In some cases, after the forward core transformation, a mode-dependent inseparable quadratic transform (MDNSST) can be applied to the low-frequency transform coefficients 714. The MDNSST operation can be performed using the Hypercube-Givens transform (HyGT) based on rotational data. When used, the encoder can send an index value that identifies a specific MDNSST operation.

[0082] At 716, the encoder can quantize the transform coefficients 714 into quantized transform coefficients 716. The quantization of each coefficient can be calculated by dividing the value of the coefficient by the quantization step size, which is derived from the quantization parameter (QP). In some embodiments, Qstep is defined as 2. (QP-4) / 6Because high-precision transform coefficients 714 can be converted into quantized transform coefficients 716 with a finite number of possible values, quantization can assist in data compression. Therefore, the quantization of transform coefficients limits the number of bits produced and transmitted by the transform process. However, although quantization is a lossy operation and the loss cannot be compensated for, the quantization process strikes a trade-off between the quality of the reconstructed sequence and the amount of information required to represent that sequence. For example, a lower QP value can produce better quality decoded video, although it may require a higher amount of data for representation and transmission. Conversely, a high QP value results in a lower quality reconstructed video sequence, but with lower data and bandwidth requirements.

[0083] JVET can utilize variance-based adaptive quantization, which allows each CU 102 to use different quantization parameters for its encoding process (instead of using the same frame QP for every CU 102 of a frame). Variance-based adaptive quantization adaptively decreases the quantization parameters in some blocks and increases them in others. To select a specific QP for a CU 102, the variance of the CU is calculated. In short, if the variance of the CU is higher than the average variance of the frame, a higher QP can be set for the CU 102 than the frame's QP. If the CU 102 exhibits a lower variance than the frame's average variance, a lower QP can be assigned.

[0084] At 720, the encoder can find the final compressed bits 722 by entropy coding the quantized transform coefficients 718. Entropy coding aims to remove statistical redundancy from the information to be transmitted. In JVET, the quantized transform coefficients 718 can be encoded using CABAC (Context Adaptive Binary Arithmetic Coding), which uses a probabilistic metric to remove statistical redundancy. For CU 102 with non-zero quantized transform coefficients 718, the quantized transform coefficients 718 can be converted to binary. Each bit (“bin”) of the binary representation can then be encoded using a context model. CU 102 can be divided into three regions, each with its own set of context models for the pixels within that region.

[0085] Multiple scan processes can be performed to encode a bin. During the encoding of the first three bins (bin0, bin1, and bin2), an index value can be found by finding the sum of the bin positions in up to five previously encoded adjacent quantization transform coefficients 718 identified by the template. This index value indicates the context model to be used for that bin.

[0086] The context model can be based on the probability that a bin value is "0" or "1". Because values ​​are encoded, the probabilities in the context model can be updated based on the actual number of "0" and "1" values ​​encountered. While HEVC uses a fixed table to reinitialize the context model for each new image, in JVET, the probabilities of the context model for a new inter-frame prediction image can be initialized based on the context model developed for previously encoded inter-frame prediction images.

[0087] The encoder can produce a bitstream containing: 722 bits of entropy-encoded residual CU 710; prediction information, such as the selected intra-frame prediction mode or motion vector; indicators of how to partition CU 102 from CTU 100 according to the QTBT structure; and / or other information about the encoded video. The bitstream can be decoded by the decoder, as described below.

[0088] In addition to using quantized transform coefficients 718 to find the final compressed bit 722, the encoder can also generate the reconstructed CU 734 using the quantized transform coefficients 718 by following the same decoding process that the decoder will use to generate the reconstructed CU 734. Therefore, once the encoder has calculated and quantized the transform coefficients, the quantized transform coefficients 718 can be sent to the decoding loop in the encoder. After quantizing the transform coefficients of the CU, the decoding loop allows the encoder to generate the same reconstructed CU 734 as the decoder generates during the decoding process. Therefore, the encoder can use the same reconstructed CU 734 that the decoder will use for adjacent CU 102 or reference images when performing intra-frame or inter-frame prediction on a new CU 102. The reconstructed CU 102, reconstructed slice, or fully reconstructed frame can be used as a reference for further prediction stages.

[0089] In the encoder's decoding loop (see below for the same operation in the decoder), to obtain the pixel values ​​of the reconstructed image, dequantization can be performed. To dequantize a frame, for example, the quantization value of each pixel in the frame is multiplied by the quantization step size described above, for example, (Qstep), to obtain the reconstructed dequantized transform coefficients 726. For example, in the encoder... Figure 7A In the decoding process shown, the quantization transform coefficients 718 of the residual CU 710 can be dequantized at 724 to find the dequantized transform coefficients 726. If the MDNSST operation was performed during encoding, this operation can be reversed after dequantization.

[0090] At 728, the dequantized transform coefficients 726 can be inversely transformed, such as by applying DCT to these values ​​to obtain the reconstructed image, to find the reconstructed residual CU 730. At 732, the reconstructed residual CU 730 can be added to the corresponding prediction CU 702 found at 704 using intra-frame prediction or at 706 using inter-frame prediction, to find the reconstructed CU 734.

[0091] At 736, one or more filters can be applied to the reconstructed data at the image level or CU level during the decoding process (in the encoder, or, as described below, in the decoder). For example, the encoder can apply a deblocking filter, a Sample Adaptive Offset (SAO) filter, and / or an Adaptive Loop Filter (ALF). The encoder's decoding process may implement filters to estimate the optimal filter parameters that can resolve potential artifacts in the reconstructed image and transmit them to the decoder. Such improvements enhance the objective and subjective quality of the reconstructed video. In deblocking filtering, pixels near the boundaries of sub-CUs can be modified, while in SAO, pixels in the CTU 100 can be modified using edge offset or frequency band offset classification. JVET's ALF can use a filter with a circularly symmetrical shape for each 2x2 block. Indications of the size and identity of the filter used for each 2x2 block can be signaled. Alternatively, in some embodiments where weighted angle prediction is implemented for the predicted CU, alternative filters or no filters may be applied to the reconstructed CU.

[0092] If the reconstructed images are reference images, they can be stored in reference buffer 738 for inter-frame prediction of future CU 102 at 706.

[0093] In the steps described above, JVET allows the use of content-adaptive clipping to adjust color values ​​to fit between upper and lower clipping limits. The clipping limits can be changed for each slice, and parameters identifying the limits can be signaled in the bitstream.

[0094] Figure 9 A simplified block diagram of CU encoding for use in a JVET decoder is depicted. The JVET decoder can receive a bitstream containing information about the encoded CU 102. This bitstream can indicate how to partition the CU 102 of the image from CTU 100 according to the QTBT structure. By way of non-limiting examples, the bitstream can identify how to partition the CU 102 from each CTU 100 in the QTBT using quadtree partitioning, symmetric binary partitioning, and / or asymmetric binary partitioning. The bitstream can also indicate prediction information for the CU 102 such as intra-frame prediction modes or motion vectors, as well as bits 902 for representing the residual CU of entropy coding.

[0095] At 904, the decoder can decode the entropy-encoded bits 902 using the CABAC context model sent by the encoder as signals in the bitstream. The decoder can update the probabilities of the context model using the parameters emitted by the encoder as signals, just as it did during the encoding process.

[0096] After inverting the entropy code at 904 to find the quantized transform coefficient 906, the decoder can dequantize it at 908 to find the dequantized transform coefficient 910. If the MDNSST operation was performed during encoding, this operation can be inverted by the decoder after dequantization.

[0097] At 912, the dequantized transform coefficients 910 can be inversely transformed to find the reconstructed residual CU 914. At 916, the reconstructed residual CU 914 can be added to the corresponding prediction CU 926 found at 922 using intra-frame prediction or at 924 using inter-frame prediction to find the reconstructed CU 918.

[0098] Therefore, in step 923a, it can be determined whether weighted angle prediction has been used as the intra-prediction mode in step 922. If it is determined in step 923a that the intra-prediction mode is weighted angle prediction, then prediction coding unit 926 can be delivered for entropy coding without filtering. However, if it is determined in step 923a that the intra-prediction mode is not weighted angle prediction, then a post-intra-prediction filter 923b (e.g., PDPC filter) can be applied to the prediction coding unit before delivery for entropy coding.

[0099] At 920, one or more filters can be applied to the reconstructed data at the image level or CU level. For example, the decoder can apply a deblocking filter, a Sample Adaptive Offset (SAO) filter, and / or an Adaptive Loop Filter (ALF). As mentioned above, the in-loop filters located in the encoder's decoding loop can be used to estimate optimal filter parameters to increase the objective and subjective quality of the frame. These parameters are sent to the decoder to filter the reconstructed frame at 920 to match the filtered reconstructed frame in the encoder.

[0100] After generating a reconstructed picture by finding the reconstructed CU 918 and applying a filter for signal transmission, the decoder can output the reconstructed picture as output video 928. If the reconstructed picture is used as a reference picture, it can be stored in a reference buffer 930 for inter-frame prediction of future CU 102 at 924.

[0101] Figure 10 An embodiment of a method for encoding 1000 using the CU in a JVET decoder is described. Figure 10In the illustrated embodiment, in step 1002, an encoded bitstream 902 may be received, and then in step 1004, a CABAC context model associated with the encoded bitstream 902 may be determined. Then, in step 1006, the determined CABAC context model may be used to decode the encoded bitstream 902.

[0102] In step 1008, quantization transform coefficients 906 associated with the encoded bitstream 902 can be determined, and then in step 1010, dequantization transform coefficients 910 can be determined from the quantization transform coefficients 906.

[0103] In step 1012, it can be determined whether an MDNSST operation was performed during encoding and / or whether bitstream 902 contains an indication that an MDNSST operation was applied to bitstream 902. If it is determined that an MDNSST operation was performed during the encoding process or bitstream 902 contains an indication to apply an MDNSST operation to bitstream 902, then the inverse MDNSST operation 1014 can be performed before the inverse transform operation 912 on bitstream 902 in step 1016. Alternatively, the inverse transform operation 912 can be performed on bitstream 902 in step 1016 without applying the inverse MDNSST operation in step 1014. The inverse transform operation 912 in step 1016 can determine and / or construct the reconstructed residual CU 914.

[0104] In step 1018, the reconstruction residual CU 914 from step 1016 can be combined with the prediction CU 918. The prediction CU 918 can be one of the intra-prediction CU 922 determined in step 1020 and the inter-prediction unit 924 determined in step 1022.

[0105] Therefore, in step 1023a, it can be determined whether weighted angle prediction has been used as the intra-prediction mode in step 1020. If it is determined in step 1023a that the intra-prediction mode is weighted angle prediction, then prediction coding unit 926 can be delivered for entropy coding without filtering, and / or the filtering performed in step 1024 can be modified and / or not present. However, if it is determined in step 1023a that the intra-prediction mode is not weighted angle prediction, then post-intra-prediction filtering 1023b and / or the filtering performed in step 1024 (e.g., PDPC filtering) can be applied to the prediction coding unit before delivery for entropy coding.

[0106] like Figure 10 As shown, in some embodiments, step 1023b may be omitted, and a post-intra-prediction filter 1024 may be applied for all predictions after step 1018. Figure 10In the illustrated embodiments, if the intra-frame prediction mode is based on something other than weighted angle prediction, the applied filter can be applied as normally as applied in step 1024. However, if the intra-frame prediction mode is based on weighted angle prediction, the filtering in step 1024 can be bypassed, and / or in some embodiments, in step 1026, before the output reconstructed CU, the applied filter may not be biased towards the primary reference, side reference, or primary and side reference. By way of non-limiting example, the values ​​of Cv1 and Ch1 can be equal and / or the values ​​of Cv2 and Ch2 can be equal.

[0107] In step 1024, any one or more filters 920 can be applied to the reconstructed CU 914 and output in step 1026. In some embodiments, filters 920 may not be applied in step 1024.

[0108] In some embodiments, in step 1028, the reconstructed CU 918 may be stored in the reference buffer 930.

[0109] Figure 11 A simplified block diagram 1100 is depicted for CU encoding in a JVET encoder. In step 1102, a JVET encoding tree unit can be represented as the root node in a quadtree plus binary tree (QTBT) structure. In some embodiments, the QTBT can have quadtrees branching from the root node and / or binary trees branching from the leaf nodes of one or more quadtrees. The representation from step 1102 can proceed to steps 1104, 1106, or 1108.

[0110] In step 1104, an asymmetric binary partition can be used to split the represented quadtree node into two unequal-sized blocks. In some embodiments, the split blocks can be represented as leaf nodes that can represent the final coding unit in the binary tree branching from the quadtree node. In some embodiments, the final coding unit is represented in the binary tree branching from the quadtree node as a leaf node, wherein further splitting is not allowed. In some embodiments, the asymmetric partition can split the coding unit into unequal-sized blocks, with the first block representing 25% of the quadtree node and the second block representing 75% of the quadtree node.

[0111] In step 1106, quadtree partitioning can be used to split the represented quadtree symbol into four square blocks of equal size. In some embodiments, the split blocks can be represented as quadtree symbols representing the final coding unit, or they can be represented as child nodes that can be further divided by quadtree partitioning, symmetric binary partitioning, or asymmetric binary partitioning.

[0112] In step 1108, quadtree partitioning can be used to split the represented quadtree symbol into two blocks of equal size. In some embodiments, the split blocks can be represented as quadtree symbols representing the final coding unit, or they can be represented as child nodes that can be further split by quadtree partitioning, symmetric binary partitioning, or asymmetric binary partitioning.

[0113] In step 1110, the child node from step 1106 or step 1108 may be represented as a child node configured to be encoded. In some embodiments, the child node may be represented by a leaf symbol of a binary tree with JVET.

[0114] In step 1112, the encoding unit from step 1104 or 1110 can be encoded using JVET.

[0115] Figure 12 A simplified block diagram 1200 for CU decoding in a JVET decoder is depicted. Figure 12 In the illustrated embodiment, in step 1202, a bitstream instructing how to divide the coding tree unit into coding units according to the QTBT structure can be received. The bitstream may instruct how to split the quadtree node using at least one of quadtree partitioning, symmetric binary partitioning, or asymmetric binary partitioning.

[0116] In step 1204, coding units represented by leaf nodes of a QTBT structure can be identified. In some embodiments, the coding unit may use asymmetric binary partitioning to indicate whether the node is split from the leaf nodes of the quadtree. In some embodiments, the coding unit may indicate that the node represents the final coding unit to be decoded.

[0117] In step 1206, the identified coding unit can be decoded using JVET.

[0118] The execution of the instruction sequence required for implementing the embodiments can be performed by, for example... Figure 13 The computer system 1300 shown performs the execution. In this embodiment, the execution of the instruction sequence is performed by a single computer system 1300. According to other embodiments, two or more computer systems 1300 coupled via communication link 1315 can coordinately execute the instruction sequence with each other. Although a description of only one computer system 1300 will be presented below, it should be understood that any number of computer systems 1300 can be used to practice the embodiments.

[0119] Now refer to Figure 13 A computer system 1300 according to an embodiment is described. Figure 13This is a block diagram of the functional components of computer system 1300. As used herein, the term computer system 1300 is widely used to describe any computing device capable of storing and independently running one or more programs.

[0120] Each computer system 1300 may include a communication interface 1314 coupled to a bus 1306. The communication interface 1314 provides bidirectional communication between the computer systems 1300. The communication interface 1314 of the respective computer system 1300 sends and receives electrical, electromagnetic, or optical signals, including data streams representing various types of signal information (e.g., instructions, messages, and data). A communication link 1315 links one computer system 1300 to another. For example, the communication link 1315 may be a LAN, in which case the communication interface 1314 may be a LAN card; or the communication link 1315 may be a PSTN, in which case the communication interface 1314 may be an Integrated Services Digital Network (ISDN) card or modem; or the communication link 1315 may be the Internet, in which case the communication interface 1314 may be a dial-up, cable, or wireless modem.

[0121] Computer system 1300 can send and receive messages, data, and instructions, including programs, i.e., applications, code, through its respective communication links 1315 and communication interfaces 1314. The received program code can be executed by the corresponding processor 1307 and / or stored in storage device 1310 or other associated non-volatile media for later execution when it is received.

[0122] In one embodiment, computer system 1300 operates in combination with data storage system 1331 (e.g., data storage system 1331 containing database 1332), which is easily accessible by computer system 1300. Computer system 1300 communicates with data storage system 1331 via data interface 1333. Data interface 1333, coupled to bus 1306, sends and receives electrical, electromagnetic, or optical signals, including data streams representing various types of signal information (e.g., instructions, messages, and data). In this embodiment, the functionality of data interface 1333 may be performed by communication interface 1314.

[0123] Computer system 1300 includes a bus 1306 or other communication mechanisms for communicating instructions, messages, and data (collectively, information) and one or more processors 1307 coupled to the bus 1306 to process information. Computer system 1300 also includes main memory 1308, such as random access memory (RAM) or other dynamic storage device, coupled to the bus 1306, for storing dynamic data and instructions to be executed by processor 1307. Main memory 1308 can also be used to store temporary data, i.e., variables or other intermediate information, during instruction execution by processor 1307.

[0124] The computer system 1300 may also include a read-only memory (ROM) 1309 or other static storage device coupled to the bus 1306 for storing static data and instructions for the processor 1307. A storage device 1310, such as a magnetic disk or optical disk, may also be provided and coupled to the bus 1306 for storing data and instructions for the processor 1307.

[0125] Computer system 1300 can be coupled to display device 1311 via bus 1306, such as, but not limited to, a cathode ray tube (CRT) or liquid crystal display (LCD) monitor, for displaying information to a user. Input device 1312 (e.g., alphanumeric keys and other keys) is coupled to bus 1306 for transmitting information and command selections to processor 1307.

[0126] According to one embodiment, a single computer system 1300 performs a specific operation by executing one or more sequences of one or more instructions contained in main memory 1308 via its respective processor 1307. These instructions may be read into main memory 1308 from another computer-usable medium (e.g., ROM 1309 or storage device 1310). Executing the sequence of instructions contained in main memory 1308 causes processor 1307 to perform the processes described herein. In alternative embodiments, hardwired circuitry may be used instead of or in combination with software instructions. Therefore, the embodiments are not limited to any particular combination of hardware circuitry and / or software.

[0127] As used herein, the term "computer-usable medium" refers to any medium that provides information or can be used by processor 1307. Such media can take many forms, including but not limited to non-volatile, volatile, and transmission media. Non-volatile media, i.e., media that can retain information without power, include ROM 1309, CD ROM, magnetic tape, and disks. Volatile media, i.e., media that cannot retain information without power, include main memory 1308. Transmission media include coaxial cables, copper wires, and optical fibers, including cables that include bus 1306. Transmission media can also be in the form of carrier waves; i.e., electromagnetic waves that can be modulated by frequency, amplitude, or phase to transmit information signals. Additionally, transmission media can be in the form of sound waves or light waves, such as sound waves or light waves generated during radio wave and infrared data communication.

[0128] In the foregoing description, embodiments have been described with reference to their specific elements. However, it will be apparent that various modifications and changes can be made to the embodiments without departing from their broader spirit and scope. For example, the reader will understand that the specific order and combination of process actions shown in the process flow diagrams described herein are merely illustrative, and different or additional process actions may be used, or different combinations or orders of process actions may be used to formulate embodiments. Therefore, the description and drawings should be considered illustrative rather than restrictive.

[0129] It should also be noted that the present invention can be implemented in various computer systems. The various techniques described herein can be implemented in hardware or software, or a combination of both. Preferably, the techniques are implemented in a computer program that executes on a programmable computer, each programmable computer including a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. Program code is applied to data input using the input device to perform the functions described above and generate output information. The output information is applied to one or more output devices. Each program is preferably implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly language or machine language. In any case, the language can be a compiled language or an interpreted language. Each such computer program is preferably stored on a general-purpose or special-purpose programmable computer-readable storage medium or device (e.g., ROM or disk) to configure and operate the computer when the computer reads the storage medium or device to perform the processes described above. The system can also be considered as a computer-readable storage medium configured with a computer program, wherein such a configured storage medium causes the computer to operate in a specific and predetermined manner. Furthermore, the storage element of an exemplary computing application can be a computing database capable of storing relational or sequential (flat file) types of data in various combinations and configurations.

[0130] Figure 14 This is a high-level view of the source device 1412 and the destination device 1410, which can be combined with the features of the systems and apparatus described herein. For example... Figure 14 As shown, the exemplary video encoding system 1410 includes a source device 1412 and a destination device 1414, wherein, in this example, the source device 1412 generates encoded video data. Therefore, the source device 1412 can be referred to as a video encoding device. The destination device 1414 can decode the encoded video data generated by the source device 1412. Therefore, the destination device 1414 can be referred to as a video decoding device. The source device 1412 and the destination device 1414 can be examples of video encoding devices.

[0131] Destination device 1414 can receive encoded video data from source device 1412 via channel 1416. Channel 1416 may include a medium or device capable of moving encoded video data from source device 1412 to destination device 1414. In one example, channel 1416 may include a communication medium that enables source device 1412 to transmit encoded video data directly to destination device 1414 in real time.

[0132] In this example, source device 1412 may modulate encoded video data according to communication standards such as wireless communication protocols, and may transmit the modulated video data to destination device 1414. The communication medium may include wireless or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium may form part of a packet-based network such as a local area network, wide area network, or a global network such as the Internet. The communication medium may include routers, switches, base stations, or other devices that facilitate communication from source device 1412 to destination device 1414. In another example, channel 1416 may correspond to a storage medium storing the encoded video data generated by source device 1412.

[0133] exist Figure 14 In the example, source device 1412 includes a video source 1418, a video encoder 1420, and an output interface 1422. In some cases, the output interface 1428 may include a modulator / demodulator (modem) and / or a transmitter. In source device 1412, video source 1418 may include a source such as a video capture device (e.g., a camera), a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and / or a computer graphics system for generating video data, or a combination of such sources.

[0134] Video encoder 1420 can encode captured, pre-captured, or computer-generated video data. Input images can be received by video encoder 1420 and stored in input frame memory 1421. General-purpose processor 1423 can load information from here and perform encoding. Programs for driving the general-purpose processor can be obtained from sources such as... Figure 14 The example storage module shown is loaded into a storage device. A general-purpose processor can use the processing memory 1422 to perform encoding, and the output of the information encoded by the general-purpose processor can be stored in a buffer, such as output buffer 1426.

[0135] The video encoder 1420 may include a resampling module 1425, which may be configured to encode (e.g., decode) video data using a scalable video coding scheme that defines at least one base layer and at least one enhancement layer. The resampling module 1425 may resample at least some of the video data as part of the encoding process, wherein resampling may be performed adaptively using a resampling filter.

[0136] Encoded video data (e.g., encoded bitstream) can be directly transmitted to the destination device 1414 via the output interface 1428 of the source device 1412. Figure 14 In the example, destination device 1414 includes an input interface 1438, a video decoder 1430, and a display device 1432. In some cases, input interface 1428 may include a receiver and / or a modem. Input interface 1438 of destination device 1414 receives encoded video data via channel 1416. The encoded video data may include various syntax elements representing video data generated by video encoder 1420. Such syntax elements may be included in encoded video data transmitted on a communication medium, stored on a storage medium, or stored on a file server.

[0137] The encoded video data can also be stored on a storage medium or file server for later access by the destination device 1414 for decoding and / or playback. For example, the encoded bitstream can be temporarily stored in the input buffer 1431 and then loaded into the general-purpose processor 1433. A program to drive the general-purpose processor can be loaded from a storage device or memory. The general-purpose processor can use the processing memory 1432 to perform decoding. The video decoder 1430 may also include a resampling module 1435 similar to the resampling module 1425 used in the video encoder 1420.

[0138] Figure 14A resampling module 1435 separate from the general-purpose processor 1433 is depicted; however, those skilled in the art will recognize that the resampling function can be performed by a program executed by the general-purpose processor, and that one or more processors can be used to perform the processing in the video encoder. The decoded image can be stored in the output frame buffer 1436 and then sent to the input interface 1438.

[0139] Display device 1438 may be integrated with destination device 1414 or may be external to destination device 1414. In some examples, destination device 1414 may include an integrated display device and may also be configured to interface with an external display device. In other examples, destination device 1414 may be a display device. Typically, display device 1438 displays decoded video data to the user.

[0140] The video encoder 1420 and video decoder 1430 can operate according to video compression standards. ITU-T VCEG (Q6 / 16) and ISO / IEC MPEG (JTC 1 / SC 29 / WG 11) are investigating the potential need to standardize future video coding technologies with compression capabilities significantly exceeding the current High Efficiency Video Coding (HEVC) standard (including its current extensions and recent extensions to screen content coding and high dynamic range coding). This exploration is being conducted jointly by the group as a joint collaborative working group called the Joint Video Exploration Group (JVET) to evaluate compression technology designs proposed by their experts in the field. The latest developments of JVET are described below: "Algorithm Description of Joint Exploration Test Model 5 (JEM 5)" published by J. Chen, E. Alshina, G. Sullivan, J. Ohm, and J. Boyce, JVET-E1001-V2.

[0141] Alternatively or concurrently, the video encoder 1420 and video decoder 1430 may operate according to other proprietary or industry standards that function with the disclosed JVET features. Thus, other standards such as ITU-T H.264, or MPEG-4, Part 10, Advanced Video Coding (AVC), or extensions of these standards. Therefore, although newly developed for JVET, the techniques disclosed herein are not limited to any particular coding standard or technology. Other examples of video compression standards and technologies include MPEG-2, ITU-T H.263, and proprietary or open-source compression formats and related formats.

[0142] The video encoder 1420 and video decoder 1430 can be implemented in hardware, software, firmware, or any combination thereof. For example, the video encoder 1420 and decoder 1430 can employ one or more processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), discrete logic, or any combination thereof. When the video encoder 1420 and decoder 1430 are partially implemented in software, the device can store instructions for the software in a suitable non-transitory computer-readable storage medium, and the instructions can be executed in hardware using one or more processors to perform the techniques of this disclosure. Each of the video encoder 1420 and video decoder 1430 can be included in one or more encoders or decoders, any of which can be integrated as part of a combined encoder / decoder (CODEC) in the respective device.

[0143] The aspects of the subject matter described herein can be described within the general context of computer-executable instructions (e.g., program modules) that are executed by a computer (e.g., the aforementioned general-purpose processors 1423 and 1433). Typically, program modules include routines, programs, objects, components, and data structures that perform specific tasks or implement specific abstract data types. The aspects of the subject matter described herein can also be practiced in distributed computing environments where tasks are performed by remote processing devices linked via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including memory storage devices.

[0144] Examples of memory include random access memory (RAM), read-only memory (ROM), or both. Memory can store instructions used to perform the techniques described above, such as source code or binary code. Memory can also be used to store variables or other intermediate information during the execution of instructions carried out by a processor (such as processors 1423 and 1433).

[0145] The storage device may also store instructions for performing the aforementioned techniques, such as source code or binary code. The storage device may additionally store data used and manipulated by a computer processor. For example, the storage device in video encoder 1420 or video decoder 1430 may be a database accessed by computer system 1423 or 1433. Other examples of such storage devices include random access memory (RAM), read-only memory (ROM), hard disk drive, magnetic disk, optical disk, CD-ROM, DVD, flash memory, USB memory card, or any other medium that a computer can read.

[0146] The memory or storage device may be an example of a non-transitory computer-readable storage medium for use by or in combination with a video encoder and / or decoder. The non-transitory computer-readable storage medium contains instructions for controlling a computer system configured to perform the functions described in the particular embodiments. When executed by one or more computer processors, the instructions may be configured to perform the instructions described in the particular embodiments.

[0147] Furthermore, it should be noted that some embodiments have been described as processes that can be depicted as flowcharts or block diagrams. Although each operation can be described as a sequential process, many operations can be performed in parallel or simultaneously. Additionally, the order of operations can be rearranged. The process may have other steps not included in the figures.

[0148] Specific embodiments may be implemented in a non-transitory computer-readable storage medium for use by or in combination with an instruction execution system, apparatus, system, or machine. The computer-readable storage medium contains instructions for controlling a computer system to perform the methods described in the specific embodiments. The computer system may include one or more computing devices. When executed by one or more computer processors, the instructions may be configured to perform the instructions described in the specific embodiments.

[0149] As used herein and throughout the claims, “a,” “an,” and “the” include plural references unless the context clearly indicates otherwise. Furthermore, as used herein and throughout the claims, “in” includes “among” and “on” unless the context clearly indicates otherwise.

[0150] Although exemplary embodiments of the invention have been described in detail using language specific to the structural features and / or methodological actions described above, it should be understood that those skilled in the art will readily recognize that many additional modifications are possible in the exemplary embodiments without substantially departing from the novel teachings and advantages of the invention. Furthermore, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Therefore, these and all such modifications are intended to be included within the scope of the invention as interpreted according to the breadth and scope of the appended claims.

Claims

1. A method for encoding video, comprising: (a) Define a rectangular coding unit (CU) with CU x and CU y coordinates within the coding region of a video frame, based on coding tree units including quadtree partitioning; (b) Determine if the coding unit is to be encoded using weighted angle prediction for a specific angle used for intra-frame prediction, then (i) Selectively define side reference pixels within the encoding region, the side reference pixels having side x and side y coordinates associated with the side reference pixels; (ii) Selectively define a primary reference pixel within the encoding region, the primary reference pixel having primary x and primary y coordinates associated with the primary reference pixel; (iii) Selectively determine the main weight value associated with the main reference pixel; (iv) Selectively determine the side weight values ​​associated with the side reference pixels; (v) Generate a prediction CU for the coding unit based at least in part on at least one of the main reference pixel combined with the sovereign weight value and the side reference pixel combined with the side weight value; (vi) wherein the predicted CU is filtered based on a post-generative filter using position-related prediction combination (PDPC) after the predicted CU is generated for the coding unit, and is subsequently delivered for entropy coding; (c) Determine if the coding unit will not be encoded using the weighted angle prediction for the specific angle used for the intra-frame prediction, then (i) Generate a prediction CU for the coding unit based at least in part on another prediction; (ii) wherein the generation based at least in part on the other prediction is not based on at least one of the primary reference pixel weighted by the sovereign weight and the side reference pixel weighted by the side weight; (iii) wherein the predicted CU is not filtered based on the post-generation filtering using position-related prediction combination (PDPC) after the predicted CU is generated for the coding unit; (iv) wherein the predicted CU, which is not filtered based on the generated post-filter using the location-related prediction combination (PDPC), is subsequently delivered for entropy coding.

2. One or more computer-readable storage devices that store encoded data as part of a bitstream, the encoded data being organized to facilitate decoding by a video decoder performing operations, the video decoder being implemented using a memory and one or more processing units, the operations comprising: (a) The bitstream contains data that is suitable for defining rectangular coding units (CUs) with CU x and CU y coordinates within the coding region of a video frame, based on coding tree units including quadtree partitioning. (b) The bitstream contains data adapted to determine if the coding unit will be decoded using weighted angle prediction for a specific angle, used for intra-frame prediction. (i) Selectively define side reference pixels within the encoding region, the side reference pixels having side x and side y coordinates associated with the side reference pixels; (ii) Selectively define a primary reference pixel within the encoding region, the primary reference pixel having primary x and primary y coordinates associated with the primary reference pixel; (iii) Selectively determine the main weight value associated with the main reference pixel; (iv) Selectively determine the side weight values ​​associated with the side reference pixels; (v) Generate a prediction CU for the coding unit based at least in part on at least one of the main reference pixel combined with the sovereign weight value and the side reference pixel combined with the side weight value; (vi) wherein the predicted CU is filtered based on a post-generative filter using position-related prediction combination (PDPC) after the predicted CU is generated for the coding unit, and is subsequently delivered for entropy decoding; (c) The bitstream contains data adapted to determine if the coding unit would be decoded without using the weighted angle prediction for the specific angle used for intra-frame prediction, then... (i) Generate a prediction CU for the coding unit based at least in part on another prediction; (ii) wherein the generation based at least in part on the other prediction is not based on at least one of the primary reference pixel weighted by the sovereign weight and the side reference pixel weighted by the side weight; (iii) wherein the predicted CU is not filtered based on the post-generation filtering using position-related prediction combination (PDPC) after the predicted CU is generated for the coding unit; (iv) wherein the predicted CU, which is not filtered based on the generated post-filter using the location-related prediction combination (PDPC), is subsequently delivered for entropy decoding.

3. A method for decoding video, comprising: (a) Define a rectangular coding unit (CU) with CU x and CU y coordinates within the coding region of a video frame, based on coding tree units including quadtree partitioning; (b) Determine if the coding unit is to be decoded using weighted angle prediction for a specific angle used for intra-frame prediction, then (i) Selectively define side reference pixels within the encoding region, the side reference pixels having side x and side y coordinates associated with the side reference pixels; (ii) Selectively define a primary reference pixel within the encoding region, the primary reference pixel having primary x and primary y coordinates associated with the primary reference pixel; (iii) Selectively determine the main weight value associated with the main reference pixel; (iv) Selectively determine the side weight values ​​associated with the side reference pixels; (v) Generate a prediction CU for the coding unit based at least in part on at least one of the main reference pixel combined with the sovereign weight value and the side reference pixel combined with the side weight value; (vi) wherein the predicted CU is filtered based on a post-generative filter using position-related prediction combination (PDPC) after the predicted CU is generated for the coding unit, and is subsequently delivered for entropy decoding; (c) Determine if the coding unit will not be decoded using the weighted angle prediction for the specific angle used for the intra-frame prediction, then (i) Generate a prediction CU for the coding unit based at least in part on another prediction; (ii) wherein the generation based at least in part on the other prediction is not based on at least one of the primary reference pixel weighted by the sovereign weight and the side reference pixel weighted by the side weight; (iii) wherein the predicted CU is not filtered based on the post-generation filtering using position-related prediction combination (PDPC) after the predicted CU is generated for the coding unit; (iv) wherein the predicted CU, which is not filtered based on the generated post-filter using the location-related prediction combination (PDPC), is subsequently delivered for entropy decoding.

Images