Histogram of gradient generation

Abstract

A system and technique for processing video data are described. For example, the apparatus can determine gradient bar chart (HoG) information based on samples of a first block of video data, using at least one sample from a second block adjacent to the first block. Based on the HoG information, the apparatus can determine an angle associated with the sample and the gradient direction from at least one sample from the second block adjacent to the first block. The apparatus can also compare the angle with one or more predefined values ​​and determine an index associated with the angle based on the comparison. The apparatus can then determine an in-frame prediction mode for encoding and decoding the first block of video data based on the index.

Description

Technical Field

[0001] This case pertains to video decoding (e.g., encoding and / or decoding of video data). For example, the various forms in this case relate to improving procedures for generating gradient bar graphs (HoG) for video decoding (e.g., video encoding and / or decoding), such as 45 degrees, 0 degrees, or other angles. These forms can be used in enhanced compression models (ECM) that go beyond VVC and / or other video decoding standards and / or formats. Prior Technology

[0002] Many devices and systems allow for the processing and output of video data for consumption. Digital video data generally comprises large amounts of data to meet the needs of video consumers and providers. For example, video consumers expect high-quality, high-fidelity, high-resolution, and high-speed video playback. Therefore, the large amounts of video data required to meet these needs place a burden on the communication networks and equipment that process and store this data.

[0003] Various video decoding techniques can be used to compress video data. Video decoding techniques can be implemented according to one or more video decoding standards. For example, video decoding standards include High Efficiency Video Decoding (HEVC), Advanced Video Decoding (AVC), Moving Picture Experts Group (MPEG) Part 2 Decoding, VP9, ​​Open Media Consortium (AOMedia) Video 1 (AV1), and Basic Video Decoding (EVC). Video decoding generally uses prediction methods that utilize redundancy present in video images or sequences (e.g., inter-frame prediction, intra-frame prediction, etc.). A key goal of video decoding technology is to compress video data into a form using a lower bit rate while avoiding or minimizing video quality degradation. As increasingly sophisticated video services become available, there is a need for coding techniques that improve decoding accuracy or efficiency. Summary of the Invention

[0004] This paper describes systems and techniques for improved video processing (e.g., video encoding and / or decoding). For example, the system can generate gradient bar graphs (HoG) for video decoding (e.g., video encoding and / or decoding), for example, at 45-degree angles, 0-degree angles, or other angles. These patterns can be used for enhanced compression models (ECM) that go beyond VVC and / or other video decoding standards and / or formats.

[0005] According to at least one example, an apparatus for processing video data is provided. The apparatus includes: at least one memory; and at least one processor (e.g., configured in a circuit) coupled to the at least one memory. The at least one processor is configured to: for sampling a first block of video data, determine gradient bar chart (HoG) information based on at least one sample from a second block adjacent to the first block; based on the HoG information, determine an angle associated with the gradient direction of the sample and the at least one sample from the second block adjacent to the first block; compare the angle with one or more predefined values; based on the comparison of the angle with one or more predefined values, determine an index associated with the angle; and based on the index, determine an in-frame prediction mode for decoding the first block of video data.

[0006] In another example, a method for processing video data is provided. The method includes: for sampling a first block of video data, determining gradient bar graph (HoG) information based on at least one sample from a second block adjacent to the first block; based on the HoG information, determining an angle associated with the gradient direction of the sample and at least one sample from the second block adjacent to the first block; comparing the angle with one or more predefined values; determining an index associated with the angle based on the comparison with the one or more predefined values; and determining an in-frame prediction mode for decoding the first block of video data based on the index.

[0007] In another example, a non-transitory computer-readable medium having instructions thereon, which, when executed by one or more processors, cause the processors to: for sampling a first block of video data, determine gradient bar graph (HoG) information based on at least one sample from a second block adjacent to the first block; based on the HoG information, determine an angle associated with the gradient direction of the sample and at least one sample from the second block adjacent to the first block; compare the angle with one or more predefined values; based on the comparison of the angle with one or more predefined values, determine an index associated with the angle; and based on the index, determine an in-frame prediction mode for decoding the first block of video data.

[0008] In another example, an apparatus for processing video data is provided. The apparatus includes: components for sampling a first block of video data and determining gradient bar graph (HoG) information based on at least one sample from a second block adjacent to the first block; components for determining an angle associated with the gradient direction of the sample and at least one sample from the second block adjacent to the first block based on the HoG information; components for comparing the angle with one or more predefined values; components for determining an index associated with the angle based on the comparison with one or more predefined values; and components for determining an in-frame prediction mode for decoding the first block of video data based on the index.

[0009] In some embodiments, one or more of the devices described herein may include, may be, or may be a subset of: mobile devices (e.g., mobile phones or so-called "smartphones," tablets, or other types of mobile devices), network-connected wearable devices, extended reality devices (e.g., virtual reality (VR) devices, augmented reality (AR) devices, or mixed reality (MR) devices), personal computers, laptops, server computers (e.g., video servers or other server devices), televisions, vehicles (or computing devices or systems of vehicles), cameras (e.g., digital cameras, Internet Protocol (IP) cameras, etc.), multi-camera systems, robotic devices or systems, aviation devices or systems, or other devices. In some embodiments, the system includes at least one camera for capturing one or more images or video frames. For example, the system may include a camera (e.g., an RGB camera) or multiple cameras for capturing one or more images and / or one or more videos including video frames. In some embodiments, the system includes a display for displaying one or more images, videos, notifications, or other displayable data. In some embodiments, the system includes a transmitter configured to transmit one or more video frames and / or syntax data to at least one device via a transmission medium. In some embodiments, the system may include one or more sensors. In some embodiments, the processor includes a neural processing unit (NPU), a central processing unit (CPU), a graphics processing unit (GPU), and / or other processing devices or elements.

[0010] The above, along with other features and examples, will become more apparent when the following description, requests, and drawings are consulted. Simple Explanation of the Diagram

[0011] The following diagrams illustrate various implementation methods in detail:

[0012] Figure 1 is a block diagram illustrating encoding and decoding devices according to some examples;

[0013] Figure 2 is a diagram illustrating an example of gradient estimation for each state according to this case;

[0014] Figure 3 is a diagram illustrating an example of mapping gradient values ​​to prediction directions according to various states in this case;

[0015] Figure 4 is a diagram showing the predicted direction selection based on the various states of this case;

[0016] Figure 5 is a graph showing the weighted in-frame predictions for each state in this case;

[0017] Figure 6 is a diagram illustrating an example of the gradient at the sampling points according to the example described herein;

[0018] Figure 7 is a flowchart illustrating the techniques for generating gradient bar graphs (HoG) according to various conditions of this case;

[0019] Figure 8 is a block diagram illustrating a video encoding device according to some examples; and

[0020] Figure 9 is a block diagram illustrating a video decoding device according to some examples. Implementation

[0021] The following provides some patterns and examples of this case. Some of these patterns and examples can be applied independently, and some can be applied in combination, as will be apparent to those skilled in the art. In the following description, specific details are set forth for illustrative purposes in order to provide a thorough understanding of the examples of this case. However, it will be clear that various examples can be practiced without these specific details. These figures and descriptions are not restrictive.

[0022] The following description is for illustrative purposes only and is not intended to limit the scope, applicability, or configuration of this application. Rather, the subsequent description of various examples will provide enabling descriptions for implementing particular examples to those skilled in the art. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope of the application set forth in the appended claims.

[0023] Video decoding equipment implements video compression technology to efficiently encode and decode video data. For example, Multifunctional Video Decoding (VVC) is the latest video decoding standard developed by the Joint Video Experts Group (JVET) of ITU-T and ISO / IEC, designed to achieve powerful compression capabilities beyond HEVC for a wider range of applications. The VVC specification was finalized in July 2020 and published by ITU-T and ISO / IEC. The VVC specification defines the bitstream and picture formats, High-Level Syntax (HLS) and decoding unit-level syntax, as well as parsing and decoding procedures. VVC also specifies profile / layer / level (PTL) constraints, byte stream formats, hypothetical reference decoders, and supplementary enhancement information (SEI) in its annex. Recently, JVET has been developing Enhanced Compression Model (ECM) software to enhance compression capabilities beyond VVC. The decoding toolset in the ECM software includes all functional blocks in the hybrid video decoding framework, including intra-frame prediction, inter-frame prediction, transform and coefficient decoding, intra-loop filtering, and entropy decoding.

[0024] Video compression techniques can include applying different prediction modes, including spatial prediction (e.g., intra-frame prediction or intra-prediction), temporal prediction (e.g., inter-frame prediction or inter-prediction), inter-layer prediction (across different layers of video data), and / or other prediction techniques, to reduce or remove inherent redundancy in a video sequence. A video encoder can partition each image of the original video sequence into multiple rectangular regions called video blocks or decoding units (described in more detail below). These video blocks can be encoded using specific prediction modes.

[0025] Video blocks can be divided into one or more smaller blocks in one or more ways. Blocks may include decode tree blocks, prediction blocks, transform blocks, and / or other suitable blocks. Unless otherwise specified, the general reference to "block" may refer to such video blocks (e.g., decode tree blocks, decode blocks, prediction blocks, transform blocks, or other suitable blocks or sub-blocks, as understood by those generally skilled in the art). Furthermore, each of these blocks may be interchangeably referred to herein as a "unit" (e.g., decode tree unit (CTU), decode unit, prediction unit (PU), transform unit (TU), etc.). In some cases, a unit may refer to a decoded logic unit encoded in a bitstream, while a block may refer to a portion of the video frame buffer to which processing is targeted.

[0026] For inter-frame prediction, the video encoder searches for blocks similar to the block being encoded in a frame (or image) located at another time position (called a reference frame or reference image). The video encoder can limit the search to a specific spatial displacement from the block to be encoded. A two-dimensional (2D) motion vector, including horizontal and vertical displacement components, can be used to locate the best match. For intra-frame prediction, the video encoder uses spatial prediction techniques to form prediction blocks based on data from previously encoded adjacent blocks within the same image.

[0027] A video encoder can determine the prediction error. For example, the prediction can be determined as the difference between the pixel values ​​in the block being encoded and the predicted block. The prediction error can also be called the residual. The video encoder can also apply a transform to the prediction error using transform decoding (e.g., using the form of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), or other suitable transforms) to produce transform coefficients. After the transform, the video encoder can quantize the transform coefficients. The quantized transform coefficients and motion vectors can be represented using syntax elements and, together with control information, form the decoded representation of the video sequence. In some cases, the video encoder can perform entropy decoding on the syntax elements, further reducing the number of bits required to represent them.

[0028] A video decoder can use the syntax elements and control information discussed above to construct prediction data (e.g., prediction blocks) for decoding the current frame. For example, a video decoder can add prediction blocks and compressed prediction errors. A video decoder can determine the compressed prediction error by weighting the transform basis function using quantization coefficients. The difference between the reconstructed frame and the original frame is called the reconstruction error.

[0029] As described in more detail below, this document describes systems, apparatus, methods (also referred to as programs), and computer-readable media (collectively, the "Systems and Techniques") for improving procedures for generating gradient bar graphs (HoGs) for video decoding (e.g., video encoding and / or decoding). For example, the Systems and Techniques may include determining a HoG used to determine a decoding mode for encoding and / or decoding video data. In some cases, the Systems and Techniques may be used to determine HoG features at 45-degree angles, 0-degree angles, or other angles. The Systems and Techniques may be used for enhanced compression models (ECMs) beyond VVC and / or other video decoding standards and / or formats. Further details relating to the various aspects of this document are described below.

[0030] The systems and techniques described herein can be applied to one or more of a variety of block-based video decoding techniques in which video is reconstructed on a block-by-block basis. For example, the systems and techniques described herein can be applied to any existing video transcoder (e.g., VVC, HEVC, AVC, or other suitable existing video transcoders), and / or can be efficient decoding tools for any video decoding standard under development and / or future video decoding standards. For example, the examples described herein can be performed using video transcoders such as VVC, HEVC, AVC, and / or their extensions. However, the techniques and systems described herein can also be applied to other decoding standards, transcoders, or formats, such as MPEG, JPEG (or other decoding standards for still images), VP9, ​​AV1, their extensions, or other suitable decoding standards that are already available or not yet available or under development. For example, in some examples, the systems and techniques can operate according to proprietary video transcoders / formats (e.g., AV1, AVI extensions, and / or subsequent versions of AV1 (e.g., AV2), or other proprietary formats or industry standards). Therefore, although the techniques and systems described herein may be described with reference to specific video decoding standards, those familiar with the technology will understand that the description should not be interpreted as applicable only to that specific standard.

[0031] This paper will discuss various aspects of the systems and technologies described herein with reference to the accompanying drawings.

[0032] Figure 1 is a block diagram illustrating an example of a system 100 that can perform one or more of the techniques described herein. System 100 includes an encoding device 104 and a decoding device 112. The encoding device 104 may be part of a source device, while the decoding device 112 may be part of a receiving device. The source device and / or receiving device may include electronic devices such as mobile or landline handsets (e.g., smartphones, cellular phones, etc.), desktop computers, laptops or notebook computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, Internet Protocol (IP) cameras, or any other suitable electronic devices. In some examples, the source device and receiving device may include one or more wireless transceivers for wireless communication. The decoding techniques described herein are applicable to video decoding in a variety of multimedia applications, including streaming video transmission (e.g., via the Internet), television broadcasting or transmission, encoding of digital video stored on data storage media, decoding of digital video stored on data storage media, or other applications. As used herein, the term "decoding" can refer to encoding and / or decoding. In some examples, system 100 may support one-way or two-way video transmission to support applications such as video conferencing, video streaming, video replay, video broadcasting, gaming, and / or video telephony.

[0033] Encoding device 104 (or encoder) can be used to encode video data using video decoding standards, formats, encoders, or protocols to produce an encoded video bitstream. Examples of video decoding standards and formats / encoders include ITU-T H.261, ISO / IEC MPEG-1 Visual, ITU-T H.262 or ISO / IEC MPEG-2 Visual, ITU-T H.263, ISO / IEC MPEG-4 Visual, ITU-T H.264 (also known as ISO / IEC MPEG-4 AVC), including its Scalable Video Decoding (SVC) and Multi-View Video Decoding (MVC) extensions, High-Efficiency Video Decoding (HEVC) or ITU-T H.265, and Multi-Functional Video Decoding (VVC) or ITU-T H.266. Several HEVC extensions exist for handling multi-layer video decoding, including range and screen content decoding extensions, 3D video decoding (3D-HEVC), multi-view extensions (MV-HEVC), and scalable extensions (SHVC). HEVC and its extensions were developed by the ITU-T Video Decoding Experts Group (VCEG) and the Joint Coordinating Group on Video Decoding (JCT-VC) and the Joint Coordinating Group on the Development of 3D Video Decoding Extensions (JCT-3V) of the ISO / IEC Moving Picture Experts Group (MPEG). VP9, ​​AOMedia Video 1 (AV1), and Basic Video Decoding (EVC), developed by the Alliance for Open Media (AOMedia), are other video decoding standards to which the technologies described herein can be applied.

[0034] Referring to Figure 1, video source 102 can provide video data to encoding device 104. Video source 102 can be part of a source device or part of a device other than a source device. Video source 102 may include video capturing devices (e.g., cameras, camera phones, video phones, etc.), video archives containing stored video, video servers or content providers providing video data, video feed interfaces receiving video from video servers or content providers, computer graphics systems for generating computer graphics video data, combinations of these sources, or any other suitable video source.

[0035] Video data from video source 102 may include one or more input pictures or frames. A picture or frame is a still image that, in some cases, is part of the video. In some examples, the data from video source 102 may be a still image that is not part of the video. In HEVC, VVC, and other video decoding specifications, a video sequence may include a series of pictures. A picture may include three sampling arrays, denoted as SL, SCb, and SCr. SL is a two-dimensional array of luminance samples, SCb is a two-dimensional array of Cb chrominance samples, and SCr is a two-dimensional array of Cr chrominance samples. Chrominance sampling may also be referred to herein as "chroma" sampling. A pixel may refer to all three components (luminance and chrominance samples) at a given location in the picture array. In other instances, the picture may be monochrome and may consist only of an array of luminance samples; in this case, the terms "pixel" and "sample" may be used interchangeably. The same technique, which represents a single sample for illustrative purposes as described herein, can be applied to pixels (e.g., all three sampled components at a given location in an image array).

[0036] The encoder engine 106 (or encoder) of the encoding device 104 encodes the video data to produce an encoded video bitstream. In some examples, the encoded video bitstream (or "video bitstream" or "bitstream") is a series of one or more decoded video sequences. A decoded video sequence (CVS) comprises a series of access units (AUs) starting with an AU that has a random access point image in the base layer and possesses certain attributes, up to and excluding the next AU that has a random access point image in the base layer and possesses certain attributes. For example, certain attributes of the random access point image that initiates the CVS may include a RASL flag equal to 1 (e.g., NoRaslOutputFlag). Otherwise, a random access point image (RASL flag equal to 0) will not initiate the CVS. An access unit (AU) includes one or more decoded images and control information corresponding to decoded images sharing the same output time. The decoded stripe of an image is encapsulated in a data unit called a Network Abstraction Layer (NAL) unit at the bitstream level. For example, an HEVC video bitstream may include one or more CVSs that include NAL units. Each NAL unit has a NAL unit header. In one example, the header for H.264 / AVC is one byte (except for multi-layer extensions), while the header for HEVC is two bytes. The syntax elements in the NAL unit header use specified bits, making them visible to all types of systems and transport layers (e.g., transport streams, Real-Time Transport (RTP) protocols, file formats, etc.).

[0037] The HEVC standard contains two types of NAL units: Video Decoding Layer (VCL) NAL units and non-VCL NAL units. VCL NAL units comprise a strip or segment of decoded image data (described below), while non-VCL NAL units contain control information related to one or more decoded images. In some cases, NAL units may be referred to as packets. HEVC AUs include VCL NAL units containing decoded image data and non-VCL NAL units (if any) corresponding to the decoded image data.

[0038] NAL units can contain bit sequences that form a decoded representation of video data (e.g., an encoded video bitstream, a CVS of a bitstream, etc.), such as a decoded representation of an image in a video. Encoder engine 106 generates a decoded representation of an image by partitioning each image into multiple stripes. Stripes are independent of other stripes, such that information in a strip is decoded without relying on data from other stripes in the same image. A stripe includes one or more strip segments, which include independent strip segments and, if present, one or more dependent strip segments that depend on previous strip segments. The stripe is partitioned into luminance-sampled and chrominance-sampled decode tree blocks (CTBs). The luminance-sampled CTB and one or more chrominance-sampled CTBs, along with the sampling syntax, are called decode tree units (CTUs). CTUs can also be called "tree blocks" or "maximum decoder units" (LCUs). A CTU is the basic processing unit for HEVC encoding. A CTU can be subdivided into multiple decoder units (CUs) of different sizes. The CU contains luminance and chrominance sampling arrays called decoding blocks (CBs).

[0039] The luma and chroma blocks (CBs) can be further subdivided into prediction blocks (PBs). A PB is a sampling block of the luma or chroma component that uses the same motion parameters for inter-frame prediction or intra-block copy prediction (when available or enabled). The luma PB and one or more chroma PBs, together with the associated syntax, form a prediction unit (PU). For inter-frame prediction, a set of motion parameters (e.g., one or more motion vectors, reference indices, etc.) is signaled in the bitstream of each PU and used for inter-frame prediction of the luma PB and one or more chroma PBs. Motion parameters can also be referred to as motion information. CBs can also be partitioned into one or more transform blocks (TBs). A TB represents a square block of color component samples to which a residual transform (e.g., in some cases the same two-dimensional transform) is applied to decode the prediction residual signal. A transform unit (TU) represents the TB of luma and chroma sampling and the corresponding syntax elements.

[0040] The size of the CU corresponds to the size of the decoding mode and can be square. For example, the size of the CU can be 8×8 sampling, 16×16 sampling, 32×32 sampling, 64×64 sampling, or any other suitable size, up to the size of the corresponding CTU. The phrase "N × N" is used herein to refer to the pixel dimensions of the video block in both the vertical and horizontal dimensions (e.g., 8 pixels × 8 pixels). Pixels in the block can be arranged in rows and columns. In some examples, the number of pixels in the horizontal direction of the block may differ from the number of pixels in the vertical direction. The syntax associated with the CU may describe, for example, partitioning the CU into one or more PUs. The partitioning mode can differ between in-frame predictive mode coding and inter-frame predictive mode coding of the CU. PUs can be partitioned into non-square shapes. The syntax associated with the CU may also describe, for example, partitioning the CU into one or more TUs according to the CTU. TUs can be square or non-square shapes.

[0041] According to the HEVC standard, transform units (TUs) can be used to perform transforms. TUs may differ for different core units (CUs). The size of a TU can be determined based on the size of a physical unit (PU) within a given CU. A TU may be the same size as or smaller than a PU. In some examples, the residual samples corresponding to the CU can be subdivided into smaller units using a quadtree structure called a residual quadtree (RQT). The leaf nodes of the RQT can correspond to TUs. The pixel differences associated with the TU can be transformed to produce transform coefficients. The transform coefficients can be quantized by the encoder engine 106.

[0042] Once the video data is partitioned into Units (CUs), encoder engine 106 uses a prediction mode to predict each Unit (PU). Prediction units or blocks are subtracted from the original video data to obtain residuals (described below). For each CU, the prediction mode can be signaled within the bitstream using syntax data. Prediction modes can include intra-frame prediction (or intra-picture prediction) or inter-frame prediction (or inter-picture prediction). Intra-frame prediction utilizes the correlation between spatially adjacent samples within a picture. For example, using intra-frame prediction, each PU can be predicted from adjacent image data in the same picture using, for example, DC prediction, to find the average value of the PU; planar prediction can be used to fit a planar surface to the PU; directional prediction can be used to extrapolate from adjacent data; or any other suitable type of prediction. Inter-frame prediction uses the temporal correlation between pictures to output motion-compensated predictions for image sampling blocks. For example, using inter-frame prediction, each PU can be predicted using motion-compensated prediction based on image data in one or more reference pictures (in output order before or after the current picture). For example, at the CU level, a decision can be made about whether to use inter-image prediction or intra-image prediction to decode image regions.

[0043] Encoder engine 106 and decoder engine 116 (described in more detail below) can be configured to operate according to VVC. According to VVC, the video decoder (e.g., encoder engine 106 and / or decoder engine 116) partitions the image into a plurality of decoder tree units (CTUs) (where the CTB of luminance sampling and one or more CTBs of chrominance sampling, along with the sampling syntax, are referred to as CTUs). The video decoder can partition the CTUs according to a tree structure, such as a quadtree-binary tree (QTBT) structure or a multi-type tree (MTT) structure. The QTBT structure eliminates the concept of multiple partition types, such as the separation between CUs, PUs, and TUs in HEVC. The QTBT structure includes two levels: a first level partitioned according to the quadtree partition and a second level partitioned according to the binary tree partition. The root node of the QTBT structure corresponds to a CTU. The leaf nodes of the binary tree correspond to decoder units (CUs).

[0044] In the MTT partitioning structure, blocks can be partitioned using quadtree partitioning, binary tree partitioning, and one or more types of ternary tree partitioning. A ternary tree partition is a partition in which a block is split into three sub-blocks. In some examples, a ternary tree partition divides a block into three sub-blocks without splitting the original block through a center. Partition types in MTT (e.g., quadtree, binary tree, and ternary tree) can be symmetric or asymmetric.

[0045] When operating according to the AV1 transcoder, the encoding device 104 and the decoding device 112 can be configured to decode video data in blocks. In AV1, the largest decoded block that can be processed is called a superblock. In AV1, a superblock can be 128×128 luminance samples or 64×64 luminance samples. However, in subsequent video decoding formats (e.g., AV2), superblocks can be defined by different (e.g., larger) luminance sample sizes. In some examples, the superblock is the top level of a block quadtree. The encoding device 104 can further partition the superblock into smaller decoded blocks. The encoding device 104 can use square or non-square partitions to partition the superblock and other decoded blocks into multiple smaller blocks. Non-square blocks can include N / 2×N, N×N / 2, N / 4×N, and N×N / 4 blocks. The encoding device 104 and the decoding device 112 can perform separate prediction and transformation procedures for each decoded block.

[0046] AV1 also defines video data tiles. A tile is a rectangular array of superblocks that can be decoded independently of other tiles. That is, the encoding device 104 and the decoding device 112 can encode and decode the decoding blocks within a tile separately, without using video data from other tiles. However, the encoding device 104 and the decoding device 112 can perform filtering across tile boundaries. The tile size can be uniform or non-uniform. Tile-based decoding enables parallel processing and / or multithreading for encoder and decoder implementations.

[0047] In some examples, encoding device 104 and decoding device 112 may use a single QTBT or MTT structure to represent each of the luminance and chrominance components, while in other examples, the video decoder may use two or more QTBT or MTT structures, for example, one QTBT or MTT structure for the luminance component and another QTBT or MTT structure for the two chrominance components (or two QTBT and / or MTT structures for the respective chrominance components).

[0048] Encoding device 104 and decoding device 112 can be configured to use HEVC quadtree partitioning, QTBT partitioning, MTT partitioning or other partitioning structures.

[0049] In some examples, one or more stripes of an image are assigned stripe types. Stripe types include I-strips, P-strips, and B-strips. An I-strip (intra-frame, independently decodable) is a stripe of an image that is decoded solely by in-frame prediction and is therefore independently decodable because an I-strip only requires in-frame data to predict any prediction unit or prediction block of the stripe. A P-strip (one-way prediction frame) is a stripe of an image that can be decoded using both in-frame prediction and one-way inter-frame prediction. Each prediction unit or prediction block within a P-strip is decoded using either in-frame prediction or inter-frame prediction. When inter-frame prediction is applied, the prediction unit or prediction block is predicted by only one reference image, and therefore the reference sample comes from only one reference region of a frame. A B-strip (two-way prediction frame) is a stripe of an image that can be decoded using both in-frame prediction and inter-frame prediction (e.g., two-way or one-way prediction). Prediction units or blocks of B-bands can be bidirectionally predicted from two reference images, where each image contributes a reference region, and the sample sets of the two reference regions are weighted (e.g., with equal weights or with different weights) to generate the prediction signal for the bidirectional prediction block. As explained above, the stripes of an image are decoded independently. In some cases, an image can be decoded into only one stripe.

[0050] As noted above, intra-frame prediction utilizes the correlation between spatially adjacent samples within an image. There are multiple intra-frame prediction modes (also called "intra-frame patterns"). In some examples, intra-frame prediction for a luma block includes 35 modes, including planar modes, DC modes, and 33 corner modes (e.g., diagonal intra-frame prediction modes and corner modes adjacent to the diagonal intra-frame prediction modes). The 35 intra-frame prediction modes are indexed as shown in Table 1 below. In other examples, more intra-frame modes can be defined, including prediction corners that may not yet be represented by the 33 corner modes. In other examples, the prediction corners associated with corner modes may differ from the prediction corners used in HEVC. Table 1 Specifications of In-Frame Prediction Modes and Associated Names In-frame prediction mode Related names 0 INTRA_PLANAR 1 INTRA_DC 2..34 INTRA_ANGULAR2..INTRA_ANGULAR34

[0051] Inter-image prediction uses the temporal correlation between images to generate motion-compensated predictions for sampled blocks of images. Using a translational motion model, the position of a block in a previously decoded image (reference image) is indicated by a motion vector (Δx, Δy), where Δx specifies the horizontal displacement of the reference block relative to the current block's position, and Δy specifies the vertical displacement of the reference block relative to the current block's position. In some cases, the motion vector (Δx, Δy) can be in integer sample precision (also known as integer precision), in which case the motion vector points to an integer pixel grid (or integer pixel sampling grid) of the reference frame. In other cases, the motion vector (Δx, Δy) can have fractional sample precision (also known as fractional pixel precision or non-integer precision) to more accurately capture the motion of underlying objects, not limited to an integer pixel grid of the reference frame. The precision of the motion vector can be represented by its quantization level. For example, the quantization level can be integer precision (e.g., 1 pixel) or fractional pixel precision (e.g., 1 / 4 pixel, 1 / 2 pixel, or other sub-pixel values). When the corresponding motion vector has fractional sampling precision, the reference image is interpolated to produce the predicted signal. For example, samples available at integer positions can be filtered (e.g., using one or more interpolation filters) to estimate values ​​at fractional positions. The previously decoded reference image is indicated by a reference index (refIdx) pointing to a list of reference images. The motion vector and reference index can be referred to as motion parameters. Two types of inter-image prediction can be performed, including one-way prediction and two-way prediction.

[0052] By using bidirectional prediction for inter-frame prediction (also known as bidirectional inter-frame prediction), two sets of motion parameters are used ( and Two motion-compensated predictions (from the same reference image or possibly from different reference images) are generated. For example, with bidirectional prediction, each prediction block uses two motion-compensated prediction signals and generates B prediction units. These two motion-compensated predictions are combined to obtain the final motion-compensated prediction. For example, the two motion-compensated predictions can be combined by averaging. In another example, weighted prediction can be used, in which case different weights can be applied to each motion-compensated prediction. The reference images that can be used for bidirectional prediction are stored in two separate lists, denoted as list 0 and list 1. Motion parameters can be exported at the encoder using a motion estimation procedure.

[0053] By using inter-frame prediction with unidirectional prediction (also known as one-way inter-frame prediction), a set of motion parameters is used ( Motion-compensated predictions are generated from a reference image. For example, by using unidirectional prediction, each prediction block uses at most one motion-compensated prediction signal and generates P prediction units.

[0054] The PU may include information related to the prediction process (e.g., motion parameters or other suitable information). For example, when encoding the PU using intra-frame prediction, the PU may include information describing the intra-frame prediction mode of the PU. As another example, when encoding the PU using inter-frame prediction, the PU may include information defining the motion vectors of the PU. The information defining the motion vectors of the PU may describe, for example, the horizontal component (Δx) of the motion vector, the vertical component (Δy) of the motion vector, the resolution of the motion vector (e.g., integer precision, quarter-pixel precision, or eighth-pixel precision), the reference picture to which the motion vector points, the reference index, a list of reference pictures for the motion vector (e.g., list 0, list 1, or list C), or any combination thereof.

[0055] AV1 includes two common techniques for encoding and decoding decoded blocks of video data. The two common techniques are intra-frame prediction (e.g., intra-frame prediction or spatial prediction) and inter-frame prediction (e.g., inter-frame prediction or temporal prediction). In the context of AV1, when predicting blocks of the current frame of video data using an intra-frame prediction mode, the encoding device 104 and the decoding device 112 do not use video data from other frames of the video data. For most intra-frame prediction modes, the video encoding device 104 encodes the blocks of the current frame based on the difference between the sampled values ​​in the current block and the predicted values ​​generated from a reference sample in the same frame. The video encoding device 104 determines the predicted values ​​generated from the reference sample based on the intra-frame prediction mode.

[0056] After performing prediction using intra-frame and / or inter-frame prediction, encoding device 104 can perform transformation and quantization. For example, after prediction, encoder engine 106 can calculate a residual value corresponding to the PU. The residual value can include the pixel difference between the current block (PU) of the decoded pixel and the prediction block used to predict the current block (e.g., a prediction version of the current block). For example, after generating a prediction block (e.g., issuing inter-frame or intra-frame prediction), encoder engine 106 can generate a residual block by subtracting the prediction block generated by the prediction unit from the current block. The residual block includes a set of pixel difference values ​​that quantize the difference between the pixel values ​​of the current block and the pixel values ​​of the prediction block. In some examples, the residual block can be represented in a two-dimensional block format (e.g., a two-dimensional matrix or an array of pixel values). In such examples, the residual block is a two-dimensional representation of the pixel values.

[0057] Any residual data that may remain after prediction is performed is transformed using a block transform, which can be based on the discrete cosine transform, discrete sine transform, integer transform, wavelet transform, other suitable transform functions, or any combination thereof. In some cases, one or more block transforms (e.g., sizes of 32×32, 16×16, 8×8, 4×4, or other suitable sizes) can be applied to the residual data in each CU. In some examples, TUs can be used for transform and quantization procedures implemented by encoder engine 106. A given CU with one or more PUs may also include one or more TUs. As described further in detail below, residual values ​​can be transformed into transform coefficients using block transforms, and can be quantized and scanned using TUs to produce serialized transform coefficients for entropy decoding.

[0058] In some examples, after intra-frame prediction or inter-frame prediction decoding using the PU of the CU, the encoder engine 106 can calculate residual data for the TU of the CU. The PU may include pixel data in the spatial domain (or pixel domain). The TU may include coefficients in the transform domain after applying the block transform. As previously noted, the residual data may correspond to the pixel difference between a pixel of the uncoded image and the predicted value corresponding to the PU. The encoder engine 106 can form a TU that includes the residual data for the CU, and can transform the TU to produce transform coefficients for the CU.

[0059] The encoder engine 106 can perform quantization of the transform coefficients. Quantization provides further compression by reducing the amount of data used to represent the coefficients. For example, quantization can reduce the bit depth associated with some or all of the coefficients. In one example, a coefficient with an n-bit value can be rounded down to an m-bit value during quantization, where n is greater than m.

[0060] Once quantization is performed, the decoded video bitstream includes quantized transform coefficients, prediction information (e.g., prediction patterns, motion vectors, block vectors, etc.), partitioning information, and any other suitable data, such as other syntax data. The different elements of the decoded video bitstream can be entropy-encoded by encoder engine 106. In some examples, encoder engine 106 can scan the quantized transform coefficients using a predefined scanning order to produce a serialized vector that can be entropy-encoded. In some examples, encoder engine 106 can perform adaptive scanning. After scanning the quantized transform coefficients to form a vector (e.g., a one-dimensional vector), encoder engine 106 can entropy-encode the vector. For example, encoder engine 106 can use context-adaptive variable-length decoding, context-adaptive binary arithmetic decoding, syntax-based context-adaptive binary arithmetic decoding, probabilistic interval partitioning entropy decoding, or another suitable entropy coding technique.

[0061] The output 110 of the encoding device 104 can transmit the NAL units constituting the encoded video bitstream data to the decoding device 112 of the receiving device via the communication link 120. The input 114 of the decoding device 112 can receive the NAL units. The communication link 120 may include a channel provided by a wireless network, a wired network, or a combination of wired and wireless networks. The wireless network may include any wireless interface or combination of wireless interfaces, and may include any suitable wireless network (e.g., the Internet or other wide area networks, packet-based networks, WiFi™, radio frequency (RF), UWB, WiFi-Direct, cellular, LTE, WiMax™, etc.). The wired network may include any wired interface (e.g., fiber optic, Ethernet, powerline Ethernet, Ethernet over coaxial cable, digital signal line (DSL), etc.). Wired and / or wireless networks can be implemented using various devices, such as base stations, routers, access points, bridges, gateways, switches, etc. Encoded video bitstream data can be modulated according to communication standards (e.g., wireless communication protocols) and sent to the receiving device.

[0062] In some examples, encoding device 104 may store encoded video bitstream data in storage device 108. Output 110 may retrieve the encoded video bitstream data from encoder engine 106 or from storage device 108. Storage device 108 may include any of a variety of distributed or locally accessed data storage media. For example, storage device 108 may include a hard disk, storage disk, flash memory, volatile or non-volatile memory, or any other suitable digital storage medium for storing encoded video data. Storage device 108 may also include a decoded picture buffer (DPB) for storing reference pictures for inter-frame prediction. In another example, storage device 108 may correspond to a file server or another intermediate storage device capable of storing encoded video generated by a source device. In this case, receiving device including decoding device 112 may access the stored video data from the storage device via streaming or downloading. The file server may be any type of server capable of storing encoded video data and sending such encoded video data to the receiving device. Example file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The receiving device can access the encoded video data via any standard data connection, including an internet connection, and may include wireless channels (e.g., Wi-Fi connections), wired connections (e.g., DSL, cable modems, etc.), or combinations thereof, suitable for accessing the encoded video data stored on the file server. Transmission of the encoded video data from storage device 108 can be streaming, downloading, or a combination thereof.

[0063] Input 114 of decoding device 112 receives encoded video bitstream data and can provide the video bitstream data to decoder engine 116 or storage device 118 for later use by decoder engine 116. For example, storage device 118 may include a DPB for storing reference pictures used for inter-frame prediction. A receiving device including decoding device 112 can receive the encoded video data to be decoded via storage device 108. The encoded video data can be modulated according to a communication standard such as a wireless communication protocol and transmitted to the receiving device. The communication medium used to transmit the encoded video data can include any wireless or wired communication medium, such as radio frequency (RF) spectrum or one or more physical transmission lines. The communication medium can form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. The communication medium can include routers, switches, base stations, or any other device that can help facilitate communication from the source device to the receiving device.

[0064] Decoder engine 116 can decode encoded video bitstream data by entropy decoding (e.g., using an entropy decoder) and extracting elements constituting one or more decoded video sequences of encoded video data. Decoder engine 116 can rescale the encoded video bitstream data and perform an inverse transform. Residual data is passed to the prediction stage of decoder engine 116. Decoder engine 116 predicts blocks (e.g., PU). In some examples, the prediction is added to the output of the inverse transform (residual data).

[0065] Decoding device 112 can output decoded video to video destination device 122, which may include a display or other output device for displaying the decoded video data to a consumer of the content. In some embodiments, video destination device 122 may be part of a receiving device that includes decoding device 112. In some embodiments, video destination device 122 may be part of a separate device other than a receiving device.

[0066] In some examples, video encoding device 104 and / or video decoding device 112 may be integrated with audio encoding device and audio decoding device, respectively. Video encoding device 104 and / or video decoding device 112 may also include other hardware or software necessary to implement the above-described decoding techniques, such as one or more microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), individual logic, software, hardware, firmware, or any combination thereof. Video encoding device 104 and video decoding device 112 may be integrated as part of a combined encoder / decoder (transcoder) in the respective device. An example of specific details of encoding device 104 is described below with reference to Figure 7. An example of specific details of decoding device 112 is described below with reference to Figure 8.

[0067] The example system shown in Figure 1 is an illustrative example that can be used herein. The techniques for processing video data using the methods described herein can be implemented by any digital video encoding and / or decoding device. While the techniques described herein are generally implemented by video encoding or video decoding devices, these techniques can also be implemented by a combined video encoder-decoder (commonly referred to as a "transcoder"). Furthermore, the techniques described herein can also be implemented by a video preprocessor. The source device and receiving device are merely examples of such decoding devices, where the source device generates decoded video data for transmission to the receiving device. In some examples, the source device and receiving device can operate in a substantially symmetrical manner, such that each device includes elements for video encoding and decoding. Therefore, the example system can support one-way or two-way video transmission between video devices, for example, for video streaming, video replay, video broadcasting, or video telephony.

[0068] Extensions to the HEVC standard include the Multi-View Video Decoding Extension (MV-HEVC) and the Scalable Video Decoding Extension (SHVC). MV-HEVC and SHVC extensions share the concept of layered decoding, where different layers are included in the encoded video bitstream. Each layer in the decoded video sequence is addressed by a unique layer identifier (ID). The layer ID may exist in the header of a NAL unit to identify the layer associated with that NAL unit. In MV-HEVC, different layers can represent different views of the same scene in the video bitstream. In SHVC, different scalable layers are provided, representing the video bitstream at different spatial resolutions (or picture resolutions) or different reconstruction fidelities. Scalable layers may include a base layer (where layer ID = 0) and one or more enhancement layers (where layer IDs = 1, 2, ..., n). The base layer may conform to the first version of the HEVC profile and represent the lowest available layer in the bitstream. Compared to the base layer, enhancement layers offer increased spatial resolution, temporal resolution, or playback rate and / or reconstruction fidelity (or quality). Enhancement layers are hierarchically organized and may (or may not) depend on lower layers. In some examples, a single standard encoder can be used to decode different layers (e.g., encoding all layers using HEVC, SHVC, or other decoding standards). In some examples, multi-standard transcoders can be used to decode different layers. For example, AVC can be used to decode the base layer, while SHVC and / or MV-HEVC extensions to the HEVC standard can be used to decode one or more enhancement layers.

[0069] Generally, a layer consists of a set of VCL NAL units and corresponding sets of non-VCL NAL units. NAL units are assigned specific layer ID values. Layers can be hierarchical, as one layer can depend on a lower layer. A layer set refers to a self-contained set of layers represented in a bitstream, meaning that layers in a layer set can depend on other layers in the layer set during decoding, but not on any other layers for decoding. Therefore, layers in a layer set can form independent bitstreams that can represent video content. The set of layers in a layer set can be obtained from another bitstream through the operations of a sub-bitstream extractor. A layer set can correspond to the set of layers that the decoder wants to decode when it wants to operate according to specific parameters.

[0070] As mentioned earlier, the HEVC bitstream includes a set of NAL units, including VCL NAL units and non-VCL NAL units. VCL NAL units contain decoded picture data that forms the decoded video bitstream. For example, the bit sequence forming the decoded video bitstream exists in VCL NAL form. Non-VCL NAL units may contain parameter sets with advanced information related to the encoded video bitstream, as well as other information. For example, parameter sets may include a Video Parameter Set (VPS), a Sequence Parameter Set (SPS), and a Picture Parameter Set (PPS). Examples of objectives for parameter sets include bit rate efficiency, error resilience, and providing a system-level interface. Each stripe references a single active PPS, SPS, and VPS to access information that the decoding device 112 can use to decode the stripe. Identifiers (IDs) can be decoded for each parameter set (including VPS ID, SPS ID, and PPS ID). An SPS includes an SPS ID and a VPS ID. A PPS includes a PPS ID and an SPS ID. Each strip header includes a PPS ID. Using the ID, the active parameter set can be identified for a given strip.

[0071] The PPS includes information applicable to all stripes within a given image. In some examples, all stripes in an image reference the same PPS. Stripes in different images may also reference the same PPS. The SPS includes information applicable to all images within the same decoded video sequence (CVS) or bitstream. As mentioned earlier, a decoded video sequence is a series of access units (AUs) that begins with a random access point image (e.g., a transient decode reference (IDR) image, a broken link access (BLA) image, or other suitable random access point image) in the base layer with certain properties (as described above) until, and does not include, the next AU (or the end of the bitstream) with a random access point image in the base layer and with certain properties. The information in the SPS may remain unchanged within the decoded video sequence. Images within a decoded video sequence may use the same SPS. The VPS includes information applicable to all layers within the decoded video sequence or bitstream. The VPS includes a syntax structure with syntax elements applicable to the entire decoded video sequence. In some examples, VPS, SPS, or PPS can be transmitted in-band along with the encoded bitstream. In other examples, VPS, SPS, or PPS can be transmitted out-of-band in a separate transmission from the NAL unit containing the decoded video data.

[0072] This term generally refers to "signaling" certain information, such as syntax elements. The term "signaling" generally refers to the communication of values ​​for syntax elements and / or other data used to decode encoded video data. For example, video encoding device 104 can signal the value of a syntax element in a bitstream. Generally, "signaling" means generating a value in a bitstream. As noted above, video source 102 can transmit the bitstream to video destination device 122 substantially immediately or not immediately (e.g., when syntax elements are stored in storage device 108 for later retrieval by video destination device 122).

[0073] Video bitstreams may also include Supplemental Enhancement Information (SEI) messages. For example, an SEI NAL unit may be part of the video bitstream. In some cases, SEI messages may contain information that is not needed by the decoder. For example, the information in the SEI message may not be necessary for the decoder to decode the video images in the bitstream, but the decoder can use the information to improve the display or processing of the images (e.g., the decoded output). The information in the SEI message may be embedded relay data. In an illustrative example, the information in the SEI message may be used by the decoder-side entity to improve the visibility of the content. In some cases, certain application standards may specify the presence of such SEI messages in the bitstream, allowing the quality improvements to be brought to all devices that comply with the application standard (e.g., the transmission of frame-packed SEI messages for frame-compatible planar stereoscopic 3DTV video formats, where the SEI message is transmitted to each frame of the video, the processing of restore point SEI messages, the use of panscanning rectangle SEI messages in DVB, and many other examples).

[0074] As mentioned earlier, for each block, a set of motion information (also referred to herein as motion parameters) may be available. This set of motion information may contain motion information for both the forward and backward prediction directions. Here, the forward and backward prediction directions are two prediction directions in a bidirectional prediction mode, and the terms "forward" and "backward" do not necessarily have geometric meaning. Instead, forward and backward may correspond to a reference image list 0 (RefPicList0) and a reference image list 1 (RefPicList1) for the current image, strip, or block. In some examples, when only one reference image list is available for an image, strip, or block, only RefPicList0 is available, and the motion information for each block of a strip is always forward. In some examples, RefPicList0 includes reference images that are temporally preceding the current image, while RefPicList1 includes reference images that are temporally following the current image. In some cases, motion vectors and associated reference indices can be used together in the decoding process. Such motion vectors with associated reference indices are represented as a unidirectional prediction set of motion information.

[0075] For each prediction direction, motion information can include a reference index and a motion vector. In some cases, for simplicity, the motion vector can have associated information, thus it can be assumed that the motion vector has an associated reference index. The reference index can be used to identify reference images in the current list of reference images (RefPicList0 or RefPicList1). The motion vector can have horizontal and vertical components, which provide the offset from the coordinate position in the current image to the coordinates in the reference image identified by the reference index. For example, the reference index can indicate a specific reference image that should be used for a block in the current image, and the motion vector can indicate the position of the best-matching block in the reference image (the block that best matches the current block) in the reference image.

[0076] Picture order count (POC) can be used in video decoding standards to identify the display order of pictures. Although it is possible for two pictures within a decoded video sequence to have the same POC value, this is not a frequent occurrence. When multiple decoded video sequences exist in a bitstream, pictures with the same POC value are likely to be decoded closer to each other. The POC value of a picture can be used for constructing a reference picture list, such as exporting a reference picture set in HEVC, and / or motion vector scaling, etc.

[0077] For in-frame prediction, in addition to the normal in-frame prediction modes (e.g., the planar mode, DC mode, and corner mode mentioned above), there is another in-frame prediction mode called decoder-side in-frame prediction mode. Decoder-side in-frame prediction mode is designed to output the decoding mode from the decoder side (e.g., via decoding device 112).

[0078] In some cases, the decoding mode can be output in at least four steps (e.g., by decoding device 112 when performing decoder-side in-frame prediction mode). For example, in the first step, the decoding device (e.g., encoding device 104 and / or decoding device 112) can estimate the per-sample gradient. In the second step, the decoding device can map the gradient values ​​to the prediction direction. In the third step, the decoding device can select the prediction direction; and in the fourth step, the selected direction can be used to perform in-frame prediction (e.g., weighted in-frame prediction).

[0079] Figure 2 is a diagram 200 illustrating an example of gradient estimation performed by a decoding device (e.g., encoding device 104 and / or decoding device 112) according to various modes of this case. Figure 2 can represent the actions performed in the first step for exporting the decoding mode. As shown in Figure 2, the current block 202 of the current image 204 is being processed (e.g., currently being decoded). The current image 204 also includes: a reconstructed region 206 of video data, which includes already decoded pixels; and an unusable region 208 of video data, which includes pixels that have not yet been decoded. In the illustrative example of Figure 2, gradients can be exported for each block of the reference sample set 216. For the sample set 216, the gradient in the horizontal direction is obtained by applying a 3×3 filter such as [-1-2-1; 0 0; 1 2 1] to adjacent samples in the horizontal direction, and the gradient in the vertical direction can be applied by applying a 3×3 filter such as [1 0-1; 2 0 -2; 1 0 -1] to adjacent samples in the vertical direction. For example, for sample 214, the gradient in the horizontal direction can be determined by applying a 3×3 filter to adjacent samples 220, 222, and 224, as well as adjacent samples 228, 230, and 232. Similarly, for sample 214, the gradient in the vertical direction can be determined by applying a 3×3 filter to adjacent samples 220, 234, and 232, as well as adjacent samples 224, 226, and 228.

[0080] Figure 3 is a diagram 300 illustrating an example of mapping gradient values ​​to prediction directions according to various schemes of this case. Figure 3 can represent the actions performed in the second step for exporting the decoding mode. For example, as shown in Figure 3, the decoding device (e.g., encoding device 104 and / or decoding device 112) can map the gradient value (e.g., determined according to the description above regarding Figure 2) to the nearest prediction direction (e.g., within the prediction direction in the range [2, 66]). As shown in Figure 3, the angle 302 for estimating the gradient can be located between two predetermined angles (angle N 304 and angle N+1 306), based on the distance 308 between angle 302 and the predetermined angles N 304 and N+1 306, to determine which predetermined angle is closest to angle 302. The decoding device can determine that angle N 304 is closest to the angle 302 of the gradient.

[0081] Figure 4 is a diagram 400 illustrating the selection of prediction directions according to various patterns in this case. Figure 4 can represent the actions performed in the third step of the output decoding mode. For example, for each prediction direction (e.g., angle), the decoding device can sum all prediction gradients (e.g., absolute gradients Gx and Gy) of neighboring pixels (pixels adjacent to the current block, e.g., neighboring samples 220 to 232 of the current block 202 in Figure 2) falling within the corresponding prediction direction to determine the first N (e.g., the first two) common directions (e.g., angles) of the prediction gradients of the current block (e.g., the current block 202 in Figure 2). In the example of Figure 4, the most common direction is M1 402, and the second most common direction is M2 404.

[0082] Figure 5 is a diagram illustrating the weighted in-frame prediction 500 according to the various samples in this case. Figure 5 can represent the actions performed in the fourth step of the export decoding mode. In some cases, the gradient of the reference sample 402 (e.g., reference sample set 216 in Figure 2) is weighted by the first two common directions, which here are M1 prediction 502 and M2 prediction 504, and the planar prediction 506 for the sampling of the current block (e.g., sample 210 of the current block 202 in Figure 2). The planar prediction 506 (e.g., INTRA_PLANAR mode 0 from Table 1 above) can be used to generate a prediction surface without discontinuities on the block boundaries, for example, by averaging the horizontal and vertical linear predictions on a sampling basis. The weighted gradients are combined to produce the final prediction value 508.

[0083] In some cases, the decoding pattern can be exported using a gradient long bar graph (HoG) (e.g., by decoding device 112 when performing decoder-side in-frame prediction patterns). For example, HoG can be applied in the second step of exporting the decoding pattern. In some cases, HoG can be based on the gradient estimated for the current block 202 based on sample 210. For example, HoG can be or can include a vector (e.g., describing or representing the gradient) of a specific length (e.g., a length of 67, such as a vector with 67 values), where each element represents the magnitude in the corresponding direction. HoG can be used to establish or as a cue for possible in-frame prediction angle patterns. In some examples, for the current block (e.g., the current decoding unit (CU) of the current block 202 in Figure 2), decoding device 112 (or in some cases encoding device 104) can compute HoG using reconstructed samples from the upper reconstructed neighbor, the left reconstructed neighbor, and the upper left corner neighbor, as shown in Figure 2.

[0084] In some cases, for each sample of a block (e.g., the current CU), its horizontal gradient will be used. and the gradient in the vertical direction We can calculate tan(θ), which determines the angle θ. The angle θ represents the direction of the gradient of this sample, as shown in Figure 6.

[0085] Of course, since determining tan(θ) may result in non-integer values, tan(θ) can be modified to avoid using non-integer values ​​(e.g., floating-point values), which can simplify the computational burden on decoding devices (e.g., encoding device 104 and / or decoding device 112).

[0086] For example, in ECM, Then tan(θ) is modified by multiplying by 65536. The result can then be compared with the values ​​in a table, for example, an angTable of length 17 (including 17 values): {0, 2048, 4096, 6144, 8192, 12288, 16384, 20480, 24576, 28672, 32768, 36864, 40960, 47104, 53248, 59392, 65536}.

[0087] The index in the table closest to the modified tan(θ) value will be selected as the direction of the gradient. For example, in one implementation of ECM, this is done by iterating through the table, as shown below: for( int i = 0; i < 17; i++ ) { if( modified tan(θ) < angTable[i] ) { idx = modified tan(θ) - angTable[i - 1] < angTable[i] - modified tan(θ) ? i-1 : i; break } } Where idx is the selected index.

[0088] In some cases, the HoG feature generation procedure may have issues, for example, when θ equals 45 degrees. For instance, when θ = 45, tan(θ) = 1, and the modified tan(θ) is 65536. In such examples, the condition check inside the loop will never be true, and the correct idx will not be selected. In another example, when θ = 0, the condition check may also encounter the problem of index i-1 being out of range of the angTable.

[0089] As noted above, systems and techniques for improving procedures for generating gradient bar graphs (HoG) for video decoding (e.g., video encoding and / or decoding) are described. For example, these systems and techniques may include determining the HoG to determine the decoding mode for encoding and / or decoding video data. In some cases, these systems and techniques may be used to determine HoG features for 45-degree angles or other angles. For example, the systems and techniques described herein can solve the aforementioned problems, such as those related to θ equal to 45 degrees, and those related to θ equal to 0 degrees and / or other angles. The various forms or examples described herein may be used individually or in any combination.

[0090] In an illustrative example, to identify the correct angle and thus the index, the obtained angle θ is compared with a predefined internal orientation angle to determine if θ is less than or equal to the predefined angle. For example, in one example implementation, the condition check within the loop above could be modified as follows: if( modified tan(θ) <= angTable[i] ) { idx = modified tan(θ) - angTable[i - 1] < angTable[i] - modified tan(θ) ? i-1 : i; break }

[0091] Furthermore, or alternatively, as noted above, there may be issues when exporting indices in tangent 0 mode (θ equals 0 degrees). For example, in the case of using the "<=" condition, when i equals 0, an index of i-1 will be needed, and the table will be accessed using the i-1 index; however, -1 is undefined (based on 0-1). To avoid this problem when θ equals 0 degrees, the starting index of the loop can be changed to the value 1. For example, in an example implementation, the condition check within the loop described above can be modified as follows (where i=1 instead of i=0 as shown in the ECM example implementation above): for( int i = 1; i < 17; i++ ) { if( modified tan(θ) <= angTable[i] ) { idx = modified tan(θ) - angTable[i - 1] < angTable[i] - modified tan(θ) ? i-1 : i; break } }

[0092] Alternatively, in another illustrative example, the predefined angle table can be expanded to include one or more additional angles after 45 degrees (tan(45)), such as {0, 2048, 4096, 6144, 8192, 12288, 16384, 20480, 24576, 28672, 32768, 36864, 40960, 47104, 53248, 59392, 65536, 67584}. For example, to make the table's size 18, in this way, comparing the exported angles with the predefined angles will perform more than one step beyond 45 degrees and can identify the correct index. In one example implementation, the loop structure is changed accordingly as follows: for( int i = 1; i < 18; i++ ) { if( modified tan(θ) < angTable[i] ) { idx = modified tan(θ) - angTable[i - 1] < angTable[i] - modified tan(θ) ? i-1 : i; break } }

[0093] Figure 7 is a flowchart illustrating examples of programs for processing video data according to various embodiments of the present invention. Program 700 may be executed by a computing device (or apparatus) or an element of a computing device (e.g., chipset, encoder, etc.). The computing device may be a mobile device (e.g., a mobile phone), a network-connected wearable device such as a watch, an extended reality (XR) device such as a virtual reality (VR) device or an augmented reality (AR) device, a vehicle or a vehicle component or system, or other types of computing devices. In some cases, the computing device may be or may include a decoding device, such as an encoding device 104, a decoding device 112, or a combined encoding device (or transcoder). Operation of program 700 may be implemented as a software element that executes and runs on one or more processors.

[0094] At block 702, a computing device (or a component thereof) can, for sampling a first block of video data, determine gradient bar chart (HoG) information based on at least one sample from a second block adjacent to the first block. In an illustrative example, the HoG information includes vectors (e.g., vectors of length 67 corresponding to prediction directions within 67 frames, as noted above). In some cases, at least one sample from the second block adjacent to the first block includes one or more of the following: a reconstructed sample from the block above the first block, a reconstructed sample from the block to the left of the first block, or a reconstructed sample from the block to the top left of the first block. For example, at least one sample from the second block adjacent to the first block may include a reconstructed sample from the block above the first block, a reconstructed sample from the block to the left of the first block, and a reconstructed sample from the block to the top left of the first block.

[0095] At block 704, the computing device (or its components) can determine, based on HoG information, the angle associated with the gradient direction of the sample and at least one sample from a second block adjacent to the first block.

[0096] At block 706, the computing device (or a component thereof) can compare the angle with one or more predefined values. For example, as described herein, in order to compare the angle with one or more predefined values, the computing device (or a component thereof) can determine whether the angle is less than or equal to one or more predefined values ​​(e.g., using the "if( modified tan(θ) <= angTable[i] )" notation described above). In this case, in order to determine the index associated with the angle, the computing device (or a component thereof) can determine the index based on whether the angle is less than or equal to one or more predefined values.

[0097] At block 708, the computing device (or its components) can determine the index associated with the angle based on a comparison of the angle with one or more predefined values. In some cases, the computing device (or its components) can determine the tangent of the angle. The computing device (or its components) can modify the tangent of the angle to produce a modified tangent, in which case one or more predefined values ​​represent the modified tangent values. The computing device (or its components) can select the index of a value in a table that is closest to the modified tangent of the angle compared to one or more predefined values. In some cases, to select an index, the computing device (or its components) can compare the value associated with the index with the modified tangent of the angle, where the index starts with the value 1 (e.g., using the "for(int i = 1; i < 17; i++)" notation described earlier). The computing device (or its components) can then increment the index (e.g., by comparing the value with all index values). In some cases, the values ​​in the table are associated with the predicted angle within the frame. In some examples, the table may include 18 values, where the values ​​of the 18 values ​​are associated with the angle after 45 degrees tan(45) (e.g., the value 67584 in the predefined angle table {0, 2048, 4096, 6144, 8192, 12288, 16384, 20480, 24576, 28672, 32768, 36864, 40960, 47104, 53248, 59392, 65536, 67584}).

[0098] At block 710, the computing device (or its components) can determine the intra-frame prediction mode for decoding the first block of video data based on an index. In some cases, the computing device (or its components) can encode the first block based on the intra-frame prediction mode. In some cases, the computing device (or its components) can transmit a bitstream including the encoded first block, or can store the bitstream. Alternatively or concurrently, in some cases, the computing device (or its components) can decode the first block based on the intra-frame prediction mode.

[0099] In some examples, the techniques or programs described herein can be executed by a computing device or apparatus, such as encoding device 104, decoding device 112, and / or any other computing device. In some cases, the computing device or apparatus may include a processor, microprocessor, microcomputer, or other elements of a device configured to perform the steps of the program described herein. In some examples, the computing device or apparatus may include a camera configured to capture video data (e.g., a video sequence) including video frames. For example, the computing device may include a camera device that may or may not include a video transcoder. As another example, the computing device may include a mobile device with a camera (e.g., a camera device such as a digital camera, IP camera, etc., a mobile phone or tablet computer including a camera, or other types of devices with a camera). In some cases, the computing device may include a display for displaying images. In some examples, the camera or other capturing device for capturing video data is separate from the computing device, in which case the computing device receives the captured video data. The computing device may also include a network interface, transceiver, and / or transmitter configured to transmit video data. Network interfaces, transceivers, and / or transmitters can be configured to transmit Internet Protocol (IP) based data or other network data.

[0100] The program described herein can be implemented using hardware, computer instructions, or a combination thereof. In the context of computer instructions, an action represents computer-executable instructions stored on one or more computer-readable storage media, which, when executed by one or more processors, perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific data type. The order of the described operations is not intended to be construed as limiting, but any number of the described operations can be combined in any order and / or in parallel to implement the program.

[0101] Furthermore, the program described herein can be executed under the control of one or more computer systems configured with executable instructions, and can be hardware-implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that execute uniformly on one or more processors, or a combination thereof. As noted above, the code can be stored, for example, in the form of a computer program comprising multiple instructions executable by one or more processors on a computer-readable or machine-readable storage medium. The computer-readable or machine-readable storage medium can be non-transitory.

[0102] The decoding techniques discussed herein can be implemented in an example video encoding and decoding system (e.g., System 100). In some examples, the system includes a source device that provides encoded video data to be decoded later by a destination device. Specifically, the source device provides the video data to the destination device via computer-readable media. The source and destination devices can include any of a variety of devices, including desktop computers, laptops, tablets, set-top boxes, mobile phones (e.g., so-called "smartphones"), so-called "smart" boards, televisions, cameras, display devices, digital media players, video game consoles, video streaming devices, etc. In some cases, the source and destination devices can be equipped for wireless communication.

[0103] The destination device can receive encoded video data to be decoded via computer-readable media. Computer-readable media can include any type of media or device capable of moving encoded video data from the source device to the destination device. In one example, computer-readable media can include communication media that enables the source device to directly and instantly transmit encoded video data to the destination device. The encoded video data can be modulated according to a communication standard such as a wireless communication protocol and transmitted to the destination device. Communication media can include any wireless or wired communication media, such as radio frequency (RF) spectrum or one or more physical transmission lines. Communication media can form part of a packet-based network, such as a local area network, a wide area network, or a global network such as the Internet. Communication media can include routers, switches, base stations, or any other devices that can facilitate communication from the source device to the destination device.

[0104] In some examples, encoded data can be output from an output interface to a storage device. Similarly, encoded data can be accessed from a storage device via an input interface. The storage device can include any of a variety of distributed or locally accessible data storage media, such as hard drives, Blu-ray discs, DVDs, CD-ROMs, flash memory, volatile or non-volatile memory, or any other suitable digital storage media for storing encoded video data. In further examples, the storage device can correspond to a file server or another intermediate storage device capable of storing encoded video generated by a source device. The destination device can access the stored video data from the storage device via streaming or downloading. The file server can be any type of server capable of storing encoded video data and sending that encoded video data to the destination device. Example file servers include web servers (e.g., for websites), FTP servers, network attached storage (NAS) devices, or local disk drives. The destination device can access the encoded video data via any standard data connection, including an internet connection. The connection may include a wireless channel (e.g., Wi-Fi connection), a wired connection (e.g., DSL, cable modem, etc.), or a combination of both, suitable for accessing encoded video data stored on a file server. The transfer of encoded video data from the storage device may be streaming, downloading, or a combination thereof.

[0105] The technology described in this application is not necessarily limited to wireless applications or setups. It can be applied to video decoding to support any of a variety of multimedia applications, such as over-the-air television broadcasting, cable television transmission, satellite television transmission, internet streaming video transmission (e.g., Dynamic Adaptive Streaming (DASH) via HTTP), digital video encoded onto data storage media, decoding digital video stored on data storage media, or other applications. In some examples, the system can be configured to support one-way or two-way video transmission to support applications such as video streaming, video replay, video broadcasting, and / or video telephony.

[0106] In one example, the source device includes a video source, a video encoder, and an output interface. The destination device may include an input interface, a video decoder, and a display device. The video encoder of the source device may be configured to apply the techniques disclosed herein. In other examples, the source and destination devices may include other elements or arrangements. For example, the source device may receive video data from an external video source such as an external camera. Similarly, the destination device may be coupled to an external display device, rather than including an integrated display device.

[0107] The example system described above is merely an example. Techniques for processing video data in parallel can be implemented by any digital video encoding and / or decoding device. Although, in general, the technique described herein is implemented by a video encoding device, it can also be implemented by a video encoder / decoder, commonly referred to as a "transcoder." Furthermore, the technique described herein can also be implemented by a video preprocessor. The source and destination devices are merely examples of such decoding devices: in which the source device generates encoded video data for transmission to the destination device. In some examples, the source and destination devices can operate in a substantially symmetrical manner, such that each device includes elements for video encoding and decoding. Therefore, the example system can support one-way or two-way video transmission between video devices, for example, for video streaming, video replay, video broadcasting, or video telephony.

[0108] A video source may include a video capturing device, such as a camera, a video archive containing previously captured video, and / or a video feed interface for receiving video from a video content provider. As a further alternative, the video source may generate computer-graphics-based data as source video, or a combination of real-time video, archived video, and computer-generated video. In some cases, if the video source is a camera, the source and destination devices may form what is known as a camera phone or video phone. However, as mentioned above, the techniques described in this case are generally applicable to video decoding and can be applied to wireless and / or wired applications. In each case, the captured, pre-captured, or computer-generated video can be encoded by a video encoder. The encoded video information can then be output to a computer-readable medium via an output interface.

[0109] As noted, computer-readable media can include transient media such as wireless broadcasting or wired network transmissions, or storage media such as hard disks, flash memory drives, compressed optical discs, digital versatile discs, Blu-ray discs (i.e., non-transitory storage media), or other computer-readable media. In some examples, a network server (not shown) can, for example, receive encoded video data from a source device via a network transmission and provide the encoded video data to a destination device. Similarly, computing equipment in a media production facility (e.g., a disc stamping facility) can receive encoded video data from a source device and produce an optical disc containing the encoded video data. Therefore, in the various examples, computer-readable media can be understood to include one or more forms of computer-readable media.

[0110] The input interface of the destination device receives information from computer-readable media. The information in the computer-readable media may include syntax information defined by the video encoder (which is also used by the video decoder), including syntax elements describing the characteristics of blocks and other decoding units (e.g., groups of pictures (GOPs)) and / or processing of those blocks and other decoding units. The display device displays the decoded video data to the user and may include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic light-emitting diode (OLED) display, or another type of display device. Various examples of this case have been described.

[0111] Specific details of the encoding device 104 and the decoding device 112 are shown in Figures 8 and 9, respectively. Figure 8 is a block diagram illustrating an example encoding device 104 that can implement one or more of the techniques described herein. The encoding device 104 can, for example, generate the syntax structures described herein (e.g., syntax structures of VPS, SPS, PPS, or other syntax elements). The encoding device 104 can perform intra-frame prediction and inter-frame prediction decoding of video blocks within a video stripe. As mentioned above, intra-frame decoding relies at least in part on spatial prediction to reduce or remove spatial redundancy within a given video frame or picture. Inter-frame decoding relies at least in part on temporal prediction to reduce or remove temporal redundancy within adjacent or surrounding frames in a video sequence. Intra-frame mode (I-mode) can refer to any of several spatial-based compression modes. Inter-frame modes such as one-way prediction (P-mode) or two-way prediction (B-mode) can refer to any of several time-based compression modes.

[0112] Encoding device 104 includes a partitioning unit 35, a prediction processing unit 41, a filter unit 63, an image memory 64, an adder 50, a transform processing unit 52, a quantization unit 54, and an entropy coding unit 56. Prediction processing unit 41 includes a motion estimation unit 42, a motion compensation unit 44, and an in-frame prediction processing unit 46. For video block reconstruction, encoding device 104 also includes an inverse quantization unit 58, an inverse transform processing unit 60, and an adder 62. Filter unit 63 is intended to represent one or more loop filters, such as deblocking filters, adaptive loop filters (ALF), and sample adaptive offset (SAO) filters. Although filter unit 63 is shown as an in-loop filter in FIG8, in other configurations, filter unit 63 can be implemented as a post-loop filter. Post-processing device 57 can perform additional processing on the encoded video data generated by encoding device 104. In some cases, the techniques of this invention can be implemented by encoding device 104. However, in other cases, one or more techniques of this invention can be implemented by post-processing device 57.

[0113] As shown in Figure 8, encoding device 104 receives video data, and partitioning unit 35 partitions the data into video blocks. Partitioning may also include, for example, partitioning into stripes, strip segments, tiles, or other larger units, and partitioning video blocks according to a quadtree structure of LCUs and CUs. Encoding device 104 is generally shown as the element that encodes video blocks within a video strip to be encoded. A strip can be divided into multiple video blocks (and possibly into a set of video blocks called tiles). Prediction processing unit 41 can select one of several possible decoding modes for the current video block based on error results (e.g., decoding rate and distortion level), for example, one of several intra-frame prediction decoding modes or one of several inter-frame prediction decoding modes. The prediction processing unit 41 can provide the obtained in-frame or inter-frame decoded blocks to the adder 50 to generate residual block data, and provide them to the adder 62 to reconstruct the encoded blocks for use as a reference image.

[0114] The intraframe prediction processing unit 46 within the prediction processing unit 41 can perform intraframe prediction decoding of the current video block relative to one or more adjacent blocks in the same frame or stripe as the current video block to be decoded, to provide spatial compression. The motion estimation unit 42 and motion compensation unit 44 within the prediction processing unit 41 perform interframe prediction decoding of the current video block relative to one or more prediction blocks in one or more reference images, to provide temporal compression.

[0115] Motion estimation unit 42 can be configured to determine the inter-frame prediction mode for a video strip based on a predetermined pattern used for the video sequence. The predetermined pattern can designate a video strip in the sequence as a P-strip, B-strip, or GPB-strip. Motion estimation unit 42 and motion compensation unit 44 can be highly integrated, but are shown separately for conceptual purposes. Motion estimation performed by motion estimation unit 42 is a procedure that generates motion vectors that estimate the motion of the video block. The motion vectors can, for example, indicate the displacement of a prediction unit (PU) of a video block within the current video frame or image relative to a prediction block within a reference image.

[0116] The predicted block is the block found to closely match the PU of the video block to be decoded in the pixel difference, which can be determined by the sum of absolute differences (SAD), sum of squared differences (SSD), or other difference metrics. In some examples, the encoding device 104 can calculate the value of a sub-integer pixel position relative to a reference image stored in the image memory 64. For example, the encoding device 104 can interpolate the values ​​of a quarter-pixel position, an eighth-pixel position, or other fractional pixel positions of the reference image. Thus, the motion estimation unit 42 can perform motion search relative to full-pixel positions and fractional pixel positions, and output a motion vector with fractional pixel precision.

[0117] The motion estimation unit 42 calculates the motion vector for the PU by comparing the position of the PU in the video block in the inter-frame decoding strip with the position of the predicted block in the reference image. The reference image can be selected from a first list of reference images (list 0) or a second list of reference images (list 1), and the identifiers of each of these two lists are stored in the image memory 64 as one or more reference images. The motion estimation unit 42 sends the calculated motion vector to the entropy coding unit 56 and the motion compensation unit 44.

[0118] Motion compensation performed by motion compensation unit 44 may involve acquiring or generating prediction blocks based on motion vectors determined by motion estimation, possibly performing interpolation with subpixel precision. Upon receiving the motion vector of the PU for the current video block, motion compensation unit 44 can locate the prediction block pointed to by the motion vector in a list of reference images. Encoding device 104 forms a residual video block by subtracting the pixel values ​​of the prediction block from the pixel values ​​of the current video block being decoded to form a pixel difference. The pixel difference forms residual data for that block and may include both luminance difference components and chrominance difference components. Adder 50 represents the element or multiple elements that perform this subtraction operation. Motion compensation unit 44 may also generate syntax elements associated with video blocks and video stripes for use by decoding device 112 when decoding video blocks of a video strip.

[0119] Intra-frame prediction processing unit 46 can perform intra-frame prediction on the current block as an alternative to the inter-frame prediction performed by motion estimation unit 42 and motion compensation unit 44 as described above. Specifically, intra-frame prediction processing unit 46 can determine the intra-frame prediction mode to be used for encoding the current block. In some examples, intra-frame prediction processing unit 46 can use various intra-frame prediction modes to encode the current block (e.g., during a separate encoding pass), and intra-frame prediction processing unit 46 can select a suitable intra-frame prediction mode from the tested modes. For example, intra-frame prediction processing unit 46 can use rate-distortion analysis for various tested intra-frame prediction modes to calculate rate-distortion values, and can select the intra-frame prediction mode with the best rate-distortion characteristics among the tested modes. Rate-distortion analysis largely determines the amount of distortion (or error) between the encoded block and the original uncoded block that was encoded to produce the encoded block, as well as the bit rate (i.e., the number of bits) used to produce the encoded block. The in-frame prediction processing unit 46 can calculate a ratio based on the distortion and rate for various encoded blocks to determine which in-frame prediction mode exhibits the optimal rate-distortion value for that block.

[0120] In any case, after selecting an intra-frame prediction mode for a block, the intra-frame prediction processing unit 46 can provide information indicating the selected intra-frame prediction mode for the block to the entropy coding unit 56. The entropy coding unit 56 can encode the information indicating the selected intra-frame prediction mode. The encoding device 104 can include definitions of encoding contexts for various blocks, as well as indications of the most likely intra-frame prediction modes to be used for each context, an intra-frame prediction mode index table, and a modified intra-frame prediction mode index table, in the transmitted bitstream configuration data. The bitstream configuration data can include a plurality of intra-frame prediction mode index tables and a plurality of modified intra-frame prediction mode index tables (also called encoded character mapping tables).

[0121] After prediction processing unit 41 generates a prediction block for the current video block via inter-frame prediction or intra-frame prediction, encoding device 104 forms a residual video block by subtracting the prediction block from the current video block. The residual video data in the residual block may be contained in one or more TUs and applied to transform processing unit 52. Transform processing unit 52 uses a transform (e.g., discrete cosine transform (DCT) or a conceptually similar transform) to transform the residual video data into residual transform coefficients. Transform processing unit 52 can transform the residual video data from the pixel domain to the transform domain (e.g., the frequency domain).

[0122] The transform processing unit 52 can send the obtained transform coefficients to the quantization unit 54. The quantization unit 54 quantizes the transform coefficients to further reduce the bit rate. The quantization procedure can reduce the bit depth associated with some or all of the coefficients. The degree of quantization can be modified by adjusting the quantization parameters. In some examples, the quantization unit 54 can then perform a scan of the matrix including the quantized transform coefficients. Alternatively, the entropy encoding unit 56 can perform the scan.

[0123] After quantization, entropy coding unit 56 performs entropy coding on the quantized transform coefficients. For example, entropy coding unit 56 can perform context-adaptive variable-length decoding (CAVLC), context-adaptive binary arithmetic decoding (CABAC), syntax-based context-adaptive binary arithmetic decoding (SBAC), probabilistic interval partitioned entropy (PIPE) decoding, or another entropy coding technique. After entropy coding by entropy coding unit 56, the encoded bitstream can be sent to decoding device 112, archived for later transmission, or retrieved by decoding device 112. Entropy coding unit 56 can also perform entropy coding on motion vectors and other syntax elements for the current video strip being decoded.

[0124] Inverse quantization unit 58 and inverse transform processing unit 60 apply inverse quantization and inverse transform, respectively, to reconstruct residual blocks in the pixel domain for later use as reference blocks in a reference image. Motion compensation unit 44 can calculate the reference block by adding the residual block to a prediction block of one of the reference images in the reference image list. Motion compensation unit 44 can also apply one or more interpolation filters to the reconstructed residual block to compute pixel values ​​of meta-integers for use in motion estimation. Adder 62 adds the reconstructed residual block to the motion compensation prediction block generated by motion compensation unit 44 to produce a reference block for storage in image memory 64. The reference block can be used by motion estimation unit 42 and motion compensation unit 44 for inter-frame prediction of blocks in subsequent video frames or images.

[0125] Encoding device 104 can perform any of the techniques described herein. Some of the techniques in this case are described generally with respect to encoding device 104, but as mentioned above, some of the techniques in this case can also be implemented by post-processing device 57.

[0126] The encoding device 104 in Figure 8 represents an example of a video encoder configured to perform one or more of the transform decoding techniques described herein. The encoding device 104 can perform any of the techniques described herein, including the program described above with respect to Figure 9.

[0127] Figure 9 is a block diagram illustrating an example decoding device 112. Decoding device 112 includes an entropy decoding unit 80, a prediction processing unit 81, an inverse quantization unit 86, an inverse transform processing unit 88, an adder 90, a filter unit 91, and an image memory 92. The prediction processing unit 81 includes a motion compensation unit 82 and an in-frame prediction processing unit 84. In some examples, decoding device 112 can perform a decoding path that is substantially the inverse of the encoding path described with respect to encoding device 104 from Figure 8.

[0128] During the decoding process, decoding device 112 receives an encoded video bitstream sent by encoding device 104, the encoded video bitstream representing video blocks and associated syntax elements of an encoded video stripe. In some examples, decoding device 112 may receive the encoded video bitstream from encoding device 104. In some examples, decoding device 112 may receive the encoded video bitstream from network entity 79 (e.g., a server, a media-aware network element (MANE), a video editor / stitcher, or other such device configured to implement one or more of the techniques described above). Network entity 79 may or may not include encoding device 104. Some of the techniques described in this document may be implemented by network entity 79 before network entity 79 sends the encoded video bitstream to decoding device 112. In some video decoding systems, network entity 79 and decoding device 112 may be part of a separate device, while in other cases, the functions described with respect to network entity 79 may be performed by the same device that includes decoding device 112.

[0129] The entropy decoding unit 80 of the decoding device 112 performs entropy decoding on the bitstream to generate quantization coefficients, motion vectors, and other syntax elements. The entropy decoding unit 80 forwards the motion vectors and other syntax elements to the prediction processing unit 81. The decoding device 112 can receive syntax elements at the video stripe level and / or video block level. The entropy decoding unit 80 can process and parse both fixed-length and variable-length syntax elements in one or more parameter sets, such as VPS, SPS, and PPS.

[0130] When a video strip is decoded into an intra-frame decoded (I) strip, the intra-frame prediction processing unit 84 of the prediction processing unit 81 can generate prediction data for video blocks of the current video strip based on the intra-frame prediction mode notified by a signal and data from previously decoded blocks from the current frame or picture. When a video frame is decoded into an inter-frame decoded (i.e., B, P, or GPB) strip, the motion compensation unit 82 of the prediction processing unit 81 generates prediction blocks for video blocks of the current video strip based on motion vectors and other syntax elements received from the entropy decoding unit 80. The prediction blocks can be generated from one of the reference pictures in the reference picture list. The decoding device 112 can construct a reference frame list—list 0 and list 1—based on the reference pictures stored in the picture memory 92 using a preset construction technique.

[0131] The motion compensation unit 82 determines prediction information for video blocks of the current video strip by parsing motion vectors and other syntax elements, and uses this prediction information to generate prediction blocks for the current video block being decoded. For example, the motion compensation unit 82 can use one or more syntax elements in the parameter set to determine the prediction mode (e.g., intra-frame or inter-frame prediction) used to decode video blocks of the video strip, the inter-frame prediction stripe type (e.g., B stripe, P stripe, or GPB stripe), construction information for one or more reference picture lists for the stripe, motion vectors for each inter-frame encoded video block of the stripe, inter-frame prediction state for each inter-frame decoded video block of the stripe, and other information used to decode video blocks in the current video stripe.

[0132] The motion compensation unit 82 can also perform interpolation based on an interpolation filter. The motion compensation unit 82 can use an interpolation filter, such as that used by the encoding device 104 during the encoding of a video block, to calculate the interpolated values ​​for sub-integer pixels of the reference block. In the above case, the motion compensation unit 82 can determine the interpolation filter used by the encoding device 104 based on the received syntax elements, and can use the interpolation filter to generate a prediction block.

[0133] The inverse quantization unit 86 inversely quantizes or dequantizes the quantized transform coefficients provided in the bitstream and decoded by the entropy decoding unit 80. The inverse quantization procedure may include determining the degree of quantization using quantization parameters calculated by the encoding device 104 for each video block in the video strip, and similarly determining the degree of inverse quantization to be applied. The inverse transform processing unit 88 applies an inverse transform (e.g., inverse DCT or other suitable inverse transform), an inverse integer transform, or a conceptually similar inverse transform procedure to the transform coefficients to generate residual blocks in the pixel domain.

[0134] After the motion compensation unit 82 generates a prediction block for the current video block based on motion vectors and other syntax elements, the decoding device 112 forms a decoded video block by adding the residual block from the inverse transform processing unit 88 to the corresponding prediction block generated by the motion compensation unit 82. Adder 90 represents the element or multiple elements performing the addition operation. If needed, loop filters (inside or after the decoding loop) can also be used to smooth pixel transitions or otherwise improve video quality. Filter unit 91 is intended to represent one or more loop filters, such as deblocking filters, adaptive loop filters (ALF), and sample adaptive offset (SAO) filters. Although filter unit 91 is shown as an in-loop filter in FIG. 9, in other configurations, filter unit 91 can be implemented as a post-loop filter. The decoded video block in a given frame or picture is stored in picture memory 92, which stores a reference picture for subsequent motion compensation. Image memory 92 also stores decoded video for later presentation on a display device (e.g., video destination device 122 shown in FIG1).

[0135] The decoding device 112 in Figure 9 represents an example of a video decoder configured to perform one or more of the transform decoding techniques described herein. The decoding device 112 can perform any of the techniques described herein, including the procedure 700 described above with respect to Figure 7.

[0136] In the foregoing description, the various embodiments of this application have been described with reference to specific examples of this application; however, those skilled in the art will recognize that this application is not limited thereto. Therefore, although illustrative examples of this application have been described in detail herein, it should be understood that the concepts described herein may be embodied and adopted in other ways, and the appended claims are intended to be construed as including such variations, in addition to variations limited by prior art. Various features and embodiments of the subject matter described above may be used individually or collectively. Furthermore, without departing from the broader spirit and scope of this specification, the examples may be utilized in any number of environments and applications beyond those described herein. Therefore, the specification and drawings are considered illustrative rather than restrictive. For illustrative purposes, the methods are described in a particular order. It should be understood that in alternative examples, the methods may be performed in a different order than that described.

[0137] When a component is described as being "configured" to perform certain operations, such configuration can be accomplished, for example, by designing electronic circuits or other hardware to perform operations, by programming programmable electronic circuits (e.g., microprocessors or other suitable electronic circuits) to perform operations, or any combination thereof.

[0138] A request language or other language that specifies "at least one" and / or "one or more" of a set indicates that one or more members of the set (in any combination) satisfy the request. For example, a request language specifying "at least one of A and B" means A, B, or A and B. In another example, a request language specifying "at least one of A, B, and C" means A, B, C, or A and B, or A and C, or B and C, or A and B and C. Specifying "at least one" and / or "one or more" of a set does not limit the set to items listed in the set. For example, a request language specifying "at least one of A and B" could mean A, B, or A and B, and could additionally include items not listed in the set of A and B.

[0139] The various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the examples presented herein can be implemented as electronic hardware, computer software, firmware, or a combination thereof. To clearly illustrate this interchangeability between hardware and software, the various illustrative elements, blocks, modules, circuits, and steps have been described above in general terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. Those skilled in the art can implement the described functionality in alternative ways for each specific application; however, such implementation decisions should not be construed as deviating from the scope of this document.

[0140] The techniques described herein can also be implemented using electronic hardware, computer software, firmware, or any combination thereof. Such techniques can be implemented in any of a variety of devices, such as general-purpose computers, wireless communication devices, mobile phones, or integrated circuit devices with multiple uses (including applications in wireless communication devices, mobile phones, and other devices). Any feature described as a module or element can be implemented together in an integrated logic device or separately as an individual but interoperable logic device. If implemented in software, the techniques can be implemented at least in part by a computer-readable data storage medium including program code that, when executed, implements one or more of the methods described above. The computer-readable data storage medium can form part of a computer program product, which may include encapsulation material. Computer-readable media may include memory or data storage media, such as random access memory (RAM) (e.g., synchronous dynamic random access memory (SDRAM)), read-only memory (ROM), non-volatile random access memory (NVRAM), electronically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, etc. Alternatively or additionally, the technology may be implemented at least in part by computer-readable communication media (e.g., propagated signals or waves) that carry or transmit program code in the form of instructions or data structures and can be accessed, read, and / or executed by a computer.

[0141] The code can be executed by a processor, which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or individual logic circuits. Such processors can be configured to perform any of the techniques described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any general processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a combination of one or more microprocessors with a DSP core, or any other such configuration. Therefore, the term "processor" as used herein may refer to any of the foregoing structures, any combination of the foregoing structures, or any other structure or device suitable for implementing the techniques described herein. Additionally, in some cases, the functionality described herein may be provided within a dedicated software or hardware module configured for encoding and decoding, or incorporated into a combined video encoder-decoder (transcoder).

[0142] The explanatory features of this case include:

[0143] Sample 1: An apparatus for processing video data, the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor being configured to: determine gradient bar graph (HoG) information of a block of video data; determine an angle associated with the gradient direction of the sampling based on sampling of the block using the HOG; determine whether the angle is less than or equal to one or more predefined angles; and determine an index associated with the angle from a table based on whether the angle is less than or equal to one or more predefined angles.

[0144] State 2: The apparatus of State 1, wherein the gradient bar graph (HoG) information includes the HoG vector.

[0145] State 3: A device for any one of State 1 or 2, wherein at least one processor is configured to determine HOG information using reconstructed sampling from the upper neighbor of the sample, reconstructed sampling from the left neighbor of the sample, and reconstructed sampling from the upper left neighbor of the sample.

[0146] State 4: A device of any one of states 1 to 3, wherein at least one processor is configured to: determine the tangent of the angle.

[0147] State 5: A device of any one of states 1 to 4, wherein the tangent of the angle comprises the vertical gradient divided by the horizontal gradient.

[0148] State 6: A device of any one of State 4 or 5, wherein at least one processor is configured to: modify the tangent of the angle using a value.

[0149] State 7: A device for State 6, wherein at least one processor is configured to: select from the table the index of the tangent closest to the modified angle.

[0150] State 8: The apparatus of State 7, wherein the table includes 18 values.

[0151] State 9: The apparatus of State 8, wherein one of the 18 values ​​includes the angle after 45 degrees tan(45).

[0152] State 10: An apparatus of any one of states 7 to 9, wherein at least one processor is configured to: determine an in-frame prediction mode for predicting blocks of video data based on a selected index.

[0153] State 11: An apparatus of any one of states 1 to 9, wherein at least one processor is configured to: determine an in-frame prediction mode for predicting blocks of video data based on a determined angle.

[0154] State 12: A device according to any one of request items 1 to 11, wherein the starting index value is equal to 1.

[0155] State 13: A method for processing video data, including operations according to any one of states 1 to 12.

[0156] State 14: A non-transitory computer-readable storage medium including instructions stored thereon, which, when executed by one or more processors, cause one or more processors to perform an operation according to any one of states 1 to 12.

[0157] Format 15: An apparatus for processing video data, comprising one or more components for performing operations according to any one of Formats 1 to 12.

[0158] Sample 16: An apparatus for processing video data, the apparatus comprising: at least one memory; and at least one processor coupled to the at least one memory, the at least one processor being configured to: for sampling a first block of video data, determine gradient bar graph (HoG) information based on at least one sample from a second block adjacent to the first block; based on the HoG information, determine an angle associated with the sampling and the gradient direction from at least one sample from the second block adjacent to the first block; compare the angle with one or more predefined values; based on the comparison of the angle with one or more predefined values, determine an index associated with the angle; and based on the index, determine an in-frame prediction mode for decoding the first block of video data.

[0159] State 17: The apparatus of state 16, wherein HoG information includes vectors.

[0160] Sample 18: An apparatus of any one of Samples 16 or 17, wherein at least one sample from a second block adjacent to the first block includes one of the following: a reconstruction sample from the upper adjacent block of the first block, a reconstruction sample from the left adjacent block of the first block, or a reconstruction sample from the upper left adjacent block of the first block.

[0161] State 19: An apparatus of any one of states 16 to 18, wherein at least one sample from a second block adjacent to the first block includes: a reconstruction sample from an upper adjacent block of the first block, a reconstruction sample from a left adjacent block of the first block, and a reconstruction sample from a top-left adjacent block of the first block.

[0162] State 20: An apparatus of any one of states 16 to 19, wherein: in order to compare an angle with one or more predefined values, at least one processor is configured to determine whether the angle is less than or equal to one or more predefined values; and in order to determine an index associated with the angle, at least one processor is configured to determine the index based on whether the angle is less than or equal to the one or more predefined values.

[0163] State 21: An apparatus of any one of states 16 to 20, wherein at least one processor is further configured to: determine the tangent of an angle; modify the tangent of an angle to produce a modified tangent of an angle, wherein one or more predefined values ​​represent the modified tangent values; and select an index of a value based on a value in a table that is closest to the modified tangent of the angle compared to one or more predefined values.

[0164] State 22: A device for state 21, wherein the values ​​in the table are associated with the predicted angle within the frame, wherein the table includes 18 values, and wherein one of the 18 values ​​is associated with an angle after 45 degrees tan(45).

[0165] State 23: A means of any one of states 21 or 22, wherein, in order to select an index, at least one processor is further configured to: compare a value associated with the index with the tangent of a modified angle, wherein the index begins with the value 1; and increment the index.

[0166] State 24: An apparatus of any one of states 16 to 23, wherein the apparatus is an encoding device, and at least one processor is further configured to: encode a first block based on an in-frame prediction pattern; and transmit a bitstream including the encoded first block.

[0167] State 25: A device of any one of states 16 to 24, wherein: the device is a decoding device, and at least one processor is further configured to: decode the first block based on an in-frame prediction pattern.

[0168] Sample 26: A method for processing video data, comprising: for sampling a first block of video data, determining gradient bar graph (HoG) information based on at least one sample from a second block adjacent to the first block; determining an angle associated with the gradient direction of the sample and at least one sample from the second block adjacent to the first block based on the HoG information; comparing the angle with one or more predefined values; determining an index associated with the angle based on the comparison of the angle with one or more predefined values; and determining an in-frame prediction mode for decoding the first block of video data based on the index.

[0169] State 27: The method of State 26, where HoG information includes vectors.

[0170] State 28: A method of any one of State 26 or 27, wherein at least one sample from a second block adjacent to the first block includes one of the following: a reconstruction sample from the upper adjacent block of the first block, a reconstruction sample from the left adjacent block of the first block, or a reconstruction sample from the upper left adjacent block of the first block.

[0171] State 29: The method of any one of states 26 to 28, wherein at least one sample from a second block adjacent to the first block includes: a reconstruction sample from the upper adjacent block of the first block, a reconstruction sample from the left adjacent block of the first block, and a reconstruction sample from the upper left adjacent block of the first block.

[0172] Method 30: Any of the methods in methods 26 to 29, further comprising: determining whether an angle is less than or equal to one or more predefined values; and determining an index based on whether an angle is less than or equal to one or more predefined values.

[0173] Method 31: Any of the methods in methods 26 to 30, further comprising: determining the tangent of an angle; modifying the tangent of an angle to produce a modified tangent of an angle, wherein one or more predefined values ​​represent the modified tangent values; and selecting an index of a value based on a value in a table that is closest to the modified tangent of the angle compared to one or more predefined values.

[0174] State 32: The method of State 31, wherein the values ​​in the table are associated with the predicted angle within the frame, wherein the table includes 18 values, and one of the 18 values ​​is associated with an angle after 45 degrees tan(45).

[0175] Method of any one of states 31 or 32, further comprising: comparing the value associated with the index with the tangent of the modified angle, wherein the index begins with the value 1; and incrementing the index.

[0176] Method 34: Any of the methods in methods 26 to 33, further comprising: encoding a first block based on an in-frame prediction pattern; and transmitting a bitstream including the encoded first block.

[0177] Method of any one of states 26 to 34, in state 35, further includes: decoding the first block based on an in-frame prediction pattern.

[0178] Sample 36: A non-transitory computer-readable storage medium including instructions stored thereon, which, when executed by one or more processors, cause one or more processors to perform an operation according to any one of samples 26 to 35.

[0179] Format 37: An apparatus for processing video data, comprising one or more components for performing operations according to any one of formats 26 to 35.

[0180] 21:Substrate 21,22:Substrate 21~22:Substrate 79: Network Entities 80: Entropy Decoding Unit 81: Predictive Processing Unit 82: Motion Compensation Unit 84: In-frame prediction processing unit 86: Inverse quantization unit 88: Inverse Transformation Processing Unit 90: Adder 91: Filter Unit 92: Image Memory 100: System 102: Video Source 104: Encoding devices 106: Encoder Engine 108: Storage device 110: Output 112: Decoding equipment 114: Input 116: Decoder Engine 118: Storage device 120: Communication Link 122: Video Destination Equipment 200: Illustration 202: Current block 204: Current Image 206: Reconstruction Area 208: Unavailable Zone 214: Sampling 216: Reference Sample Set 220: Adjacent sampling 222: Adjacent Sampling 224: Adjacent Sampling 226: Adjacent Sampling 228: Adjacent Sampling 230: Adjacent sampling 232: Adjacent Sampling 234: Adjacent Sampling 300: Illustration 302: Angle 304: Pre-determined angle N 306:Angle N+1 308: Distance 400: Illustration 402: The most common direction M1 404: The second most common direction is M2 500: Weighted prediction within the frame 502:M1 Prediction 504:M2 Forecast 506: Plane Prediction 508: Final Predicted Value 700: Program 702: Square 704: Square 706: Square 708: Square 710: Square GH: Gradient in the horizontal direction G Ver: Gradient in the vertical direction M1: The most common direction θ: angle

[0181] Domestic storage information (please note in order of storage institution, date, and number) none Overseas storage information (please note in the order of storage country, institution, date, and number) none

Claims

1. An apparatus for processing video data, the apparatus comprising: At least one memory cell; and at least one processor coupled to the at least one memory, the at least one processor being configured to: determine gradient bar chart (HoG) information based on at least one sample from a second block adjacent to the first block, for a sample of a first block of video data; determine an angle associated with a gradient direction of the sample and the at least one sample from the second block adjacent to the first block, based on the HoG information; determine a tangent of the angle; modify the tangent of the angle to produce a modified tangent of the angle; determine whether the modified tangent of the angle is less than or equal to a first predefined value among a plurality of predefined values ​​in a table, wherein the plurality of predefined values ​​in the table correspond to the modified tangent value, wherein each predefined value among the plurality of predefined values ​​is associated with a relative index value of an index, and wherein the index begins with an index value of 1; The index value is incremented to an updated index value to compare an additional predefined value among the plurality of predefined values ​​in the table with the modified tangent of the angle, wherein the updated index value is associated with the additional predefined value in the table; a specific index value is selected based on the fact that the predefined value is closest to the modified tangent of the angle compared to other predefined values ​​among the plurality of predefined values ​​in the table, and based on the fact that the modified tangent of the angle is less than or equal to the predefined value, and a specific index value is associated with a predefined value among the plurality of predefined values ​​in the table; an intra-frame prediction mode for coding the first block of the video data is determined based on the specific index value; and the first block of the video data is decoded based on the determined intra-frame prediction mode.

2. The apparatus according to claim 1, wherein the HoG information includes a vector.

3. The apparatus according to claim 1, wherein the at least one sample from the second block adjacent to the first block includes one of the following: a reconstruction sample from an upper adjacent block of the first block, a reconstruction sample from a left adjacent block of the first block, or a reconstruction sample from an upper left adjacent block of the first block.

4. The apparatus according to claim 1, wherein the at least one sample from the second block adjacent to the first block includes: A reconstruction sample from an upper adjacent block of the first block, a reconstruction sample from a left adjacent block of the first block, and a reconstruction sample from a top-left adjacent block of the first block.

5. The apparatus according to claim 1, wherein: The at least one processor is configured to increment the index value to the updated index value based on the fact that the modified tangent of the angle is less than the first predefined value.

6. The apparatus according to claim 1, wherein the values ​​in the table are associated with a prediction angle within a frame, wherein the table includes 18 values, and wherein one of the 18 values ​​is associated with an angle greater than 45 degrees tan(45).

7. The apparatus according to claim 1, wherein the apparatus is an encoding device, and wherein, in order to decode the first block, the at least one processor is further configured to: encode the first block based on the in-frame prediction mode.

8. The apparatus according to claim 1, wherein the apparatus is a decoding device, and wherein, in order to decode the first block, the at least one processor is further configured to: decode the first block based on the in-frame prediction mode.

9. A method for processing video data, comprising: For a sample of a first block of video data, gradient bar chart (HoG) information is determined based on at least one sample from a second block adjacent to the first block; based on the HoG information, an angle associated with a gradient direction of the sample and the at least one sample from the second block adjacent to the first block is determined; a tangent of the angle is determined; the tangent of the angle is modified to produce a modified tangent of the angle; it is determined whether the modified tangent of the angle is less than or equal to a first predefined value among a plurality of predefined values ​​in a table, wherein the plurality of predefined values ​​in the table correspond to the modified tangent value, wherein each predefined value among the plurality of predefined values ​​is associated with a relative index value of an index, and wherein the index begins with an index value of 1; the index value is incremented to an updated index value to compare an additional predefined value among the plurality of predefined values ​​in the table with the modified tangent of the angle, wherein the at least one updated index value is associated with the at least one additional predefined value in the table; Based on the predefined value being closest to the modified tangent of the angle compared to other predefined values ​​among the plurality of predefined values ​​in the table, and based on the modified tangent of the angle being less than or equal to the predefined value, a specific index value associated with a predefined value among the plurality of predefined values ​​in the table is selected; an intra-frame prediction mode for decoding the first block of video data is determined based on the specific index value; and the first block of video data is decoded based on the determined intra-frame prediction mode.

10. The method according to request item 9, wherein the HoG information includes a vector.

11. The method of claim 9, wherein the at least one sample from the second block adjacent to the first block includes one of the following: a reconstructed sample from an upper adjacent block of the first block, a reconstructed sample from a left adjacent block of the first block, or a reconstructed sample from an upper left adjacent block of the first block.

12. The method according to claim 9, wherein the at least one sample from the second block adjacent to the first block includes: A reconstruction sample from an upper adjacent block of the first block, a reconstruction sample from a left adjacent block of the first block, and a reconstruction sample from a top-left adjacent block of the first block.

13. The method according to claim 9 also includes: Since the modified tangent based on the angle is less than the first predefined value, the index value is incremented to the updated index value.

14. The method of claim 9, wherein the values ​​in the table are associated with the predicted angle within the frame, wherein the table includes 18 values, and wherein one of the 18 values ​​is associated with an angle greater than 45 degrees tan(45).

15. The method according to claim 9, wherein decoding the first block of video data includes: The first block is encoded based on the prediction pattern within the frame.

16. The method according to claim 9, wherein decoding the first block of video data includes: The first block is decoded based on the prediction pattern within the frame.

17. An apparatus for processing video data, the apparatus comprising: At least one memory cell; and at least one processor coupled to the at least one memory, the at least one processor being configured to: determine gradient bar chart (HoG) information based on at least one sample from a second block adjacent to the first block, for a sample of a first block of video data; determine an angle associated with a gradient direction of the sample and the at least one sample from the second block adjacent to the first block, based on the HoG information; determine a tangent of the angle; and modify the tangent of the angle to produce a modified tangent of the angle; Compare at least one predefined value from a set of 18 predefined values ​​in a table with the modified tangent of the angle, wherein the plurality of predefined values ​​in the table correspond to the modified tangent value, and wherein the plurality of predefined values ​​are associated with the predicted angle within the frame, such that a first predefined value from the set of 18 predefined values ​​is associated with an angle greater than a 45-degree angle tan(45), wherein each predefined value from the plurality of predefined values ​​is associated with a corresponding index value of an index, and wherein the index begins with an index value of 1; increment the index value to an updated index value to compare an additional predefined value from the plurality of predefined values ​​in the table with the modified tangent of the angle, wherein the updated index value is associated with the additional predefined value in the table; select a specific index value associated with a predefined value from the plurality of predefined values ​​in the table based on the predefined value being closest to the modified tangent of the angle compared to the other predefined values ​​from the plurality of predefined values ​​in the table; Based on the specific index value, determine an in-frame prediction mode for decoding the first block of video data; and based on the determined in-frame prediction mode, decode the first block of video data.

18. The apparatus according to claim 17, wherein the HoG information includes a vector.

19. The apparatus according to claim 17, wherein the at least one sample from the second block adjacent to the first block includes one of the following: a reconstruction sample from an upper adjacent block of the first block, a reconstruction sample from a left adjacent block of the first block, or a reconstruction sample from an upper left adjacent block of the first block.

20. The apparatus according to claim 17, wherein the at least one sample from the second block adjacent to the first block comprises: A reconstruction sample from an upper adjacent block of the first block, a reconstruction sample from a left adjacent block of the first block, and a reconstruction sample from a top-left adjacent block of the first block.

21. The apparatus according to claim 17, wherein the at least one processor is configured to increment the index value to the updated index value based on the modified tangent of the angle being less than the first predefined value.

22. The apparatus according to claim 21, wherein the at least one processor is configured to select the particular index value associated with the predefined value based on the modified tangent of the angle being less than or equal to the predefined value.

23. The apparatus according to claim 17, wherein the apparatus is an encoding device, and wherein, in order to decode the first block, the at least one processor is further configured to: encode the first block based on the in-frame prediction mode.

24. The apparatus according to claim 17, wherein the apparatus is a decoding device, and wherein, in order to decode the first block, the at least one processor is further configured to decode the first block based on the in-frame prediction mode.

Images