Systems and methods for angular intra mode coding

Abstract

Various implementations described herein include methods and systems for encoding and decoding video. In one aspect, a method of video decoding includes receiving video data from a video bitstream, the video data including a first block, where the first block is predicted in an angular prediction mode. The method further includes identifying a reference sample set for a portion of the first block using a prediction angle of the angle prediction mode; and deriving a first angle predictor for the portion using at least a first subset of the reference sample set. The method further comprises deriving a second angle predictor for this portion using a weighted sum of at least a second subset of the reference sample set and the first angle predictor; and decoding the portion using the second angle prediction value.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 467,213, filed on May 17, 2023, entitled “Angular Blend Mode in Intra Mode Coding,” and this application is a continuation of and claims priority to U.S. Patent Application No. 18 / 474,532, filed on September 26, 2023, entitled “Systems and Methods for Angular Intra Mode Coding.” Technical Field

[0003] The disclosed embodiments relate generally to video coding and decoding, including but not limited to systems and methods for angle mode prediction for video coding / decoding. Background Art

[0004] Digital video is supported by various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. Electronic devices send and receive or otherwise communicate digital video data over a communication network and / or store the digital video data on a storage device. Due to the limited bandwidth capacity of a communication network and the limited memory resources of a storage device, video encoding may be used to compress the video data according to one or more video encoding standards before communicating or storing the video data.

[0005] A variety of video codec standards have been developed. For example, video coding standards include Alliance for Open Media Video 1 (AOMedia Video 1, AV1), Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), High Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264) and Moving Picture Experts Group (MPEG) coding. Video coding generally utilizes prediction methods (e.g., inter-frame prediction, intra-frame prediction, etc.) that utilize the redundancy inherent in video data. Video coding is intended to compress video data into a form that uses a lower bit rate while avoiding or minimizing degradation in video quality.

[0006] HEVC, also known as H.265, is a video compression standard designed as part of the MPEG-H project. The HEVC / H.265 standard was published by ITU-T and ISO / IEC in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC), also known as H.266, is a video compression standard intended to be the successor to HEVC. The VVC / H.266 standard was published by ITU-T and ISO / IEC in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as an alternative to HEVC. On January 8, 2019, the validated version 1.0.0 of the specification was released with Errata 1. Summary of the invention

[0007] As described in more detail below, the angular prediction mode in some systems does not take into account the uneven distribution of available reference samples (e.g., only top reference samples and left reference samples may be available). The systems and methods described herein can improve prediction accuracy by biasing the prediction process that uses top and / or left reference samples. For example, to predict samples in a current block using spatially neighboring reference samples, an angular prediction value is first derived, and then a weighted sum of the left reference samples and the angular prediction value is used to derive a final prediction value. In this example, the final prediction value can then be used to encode / decode samples in the current block.

[0008] According to some embodiments, a video decoding method is provided. The method includes: (i) receiving video data from a video code stream, the video data including a first block, wherein the first block is predicted in an angle prediction mode; (ii) identifying a reference sample set for a portion of the first block (e.g., a current sample / pixel of the first block), wherein the reference sample set is identified using a prediction angle of the angle prediction mode; (iii) deriving a first angle prediction value for the portion of the first block using at least a first subset of the reference sample set; (iv) deriving a second angle prediction value for the portion of the first block using a weighted sum of at least a second subset of the reference sample set and the first angle prediction value; and (v) decoding the portion of the first block using the second angle prediction value.

[0009] According to some embodiments, a video encoding method is provided. The method includes: (i) receiving video data, the video data including a first block, wherein the first block is to be encoded in an angular mode; (ii) identifying a reference sample set for a portion of the first block (e.g., a current sample / pixel of the first block), wherein the reference sample set is identified using a prediction angle of an angular prediction mode; (iii) deriving a first angular prediction value for the portion of the first block using at least a first subset of the reference sample set; (iv) deriving a second angular prediction value for the portion of the first block using a weighted sum of at least a second subset of the reference sample set and the first angular prediction value; and (v) encoding the portion of the first block using the second angular prediction value.

[0010] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes a control circuit and a memory storing one or more instruction sets. The one or more instruction sets include instructions for performing any of the methods described herein. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder component).

[0011] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more instruction sets executed by a computing system. The one or more instruction sets include instructions for performing any of the methods described herein.

[0012] Therefore, a device and system having methods for encoding and decoding a video are disclosed. Such methods, devices and systems can supplement or replace conventional methods, devices and systems for video encoding / decoding.

[0013] The features and advantages described in the specification are not necessarily all-inclusive, and in particular, some additional features and advantages will be apparent to those of ordinary skill in the art in view of the drawings, descriptions, and claims provided by the present disclosure. In addition, it should be noted that the language used in the specification is primarily selected for readability and instructional purposes, and is not necessarily selected to describe or limit the subject matter described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order that the present disclosure may be understood in more detail, a more specific description may be given with reference to the features of various embodiments, some of which are illustrated in the accompanying drawings. However, the accompanying drawings only illustrate the relevant features of the present disclosure and are therefore not necessarily to be considered limiting, as the description may allow for other effective features that will be understood by those skilled in the art upon reading the present disclosure.

[0015] Figure 1 is a block diagram illustrating an exemplary communication system in accordance with some embodiments.

[0016] Figure 2A is a block diagram illustrating exemplary elements of an encoder assembly according to some embodiments.

[0017] Figure 2B is a block diagram illustrating exemplary elements of a decoder component according to some embodiments.

[0018] Figure 3 is a block diagram illustrating an exemplary server system in accordance with some embodiments.

[0019] 4A to 4D An exemplary coding tree structure according to some embodiments is shown.

[0020] Figure 5A Examples of directional intra prediction mode angles are shown in accordance with some embodiments.

[0021] Figure 5B and Figure 5C Exemplary reference samples of a current block according to some embodiments are shown.

[0022] Figure 5D and Figure 5E Exemplary reference samples for samples in a current block according to some embodiments are shown.

[0023] Fig. 6A is a flow chart illustrating an exemplary method of encoding a video according to some embodiments.

[0024] Figure 6B is a flow chart illustrating an exemplary method of decoding a video according to some embodiments.

[0025] According to common practice, the various features shown in the drawings are not necessarily drawn to scale, and like reference numerals may be used to denote like features throughout the specification and drawings. DETAILED DESCRIPTION

[0026] The present disclosure describes, among other things, using top and / or left reference samples to bias angular mode prediction. For example, a refined angular prediction value for a portion (sample) of a first block may be derived using a weighted sum of the (top / left) reference samples and the unrefined angular prediction value. Biasing angular mode prediction in this manner may improve prediction accuracy because only top and left reference samples are available (e.g., other reference samples may be padded).

[0027] Exemplary Systems and Apparatus

[0028] Figure 11 is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic devices 120-1 to 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, for example, used with video-enabled applications such as video conferencing applications, digital television applications, media storage and / or distribution applications.

[0029] Source device 102 includes a video source 104 (e.g., a camera component or a media storage) and an encoder component 106. In some embodiments, video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). Encoder component 106 generates one or more encoded video streams from the video stream. The video stream from video source 104 can be high data volume compared to the encoded video stream 108 generated by encoder component 106. Because the encoded video stream 108 has a lower data volume (less data) than the video stream from the video source, the encoded video stream 108 requires less bandwidth for transmission and less storage space for storage than the video stream from video source 104. In some embodiments, source device 102 does not include encoder component 106 (e.g., configured to transmit uncompressed video to network 110).

[0030] One or more networks 110 represent any number of networks that transmit information between source device 102, server system 112, and / or electronic device 120, including, for example, wired and / or wireless communication networks. One or more networks 110 can exchange data in circuit switching channels and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet.

[0031] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is a streaming server or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from the source device 102). The server system 112 includes an encoder component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the encoder component 114 includes an encoder component and / or a decoder component. In various embodiments, the encoder component 114 is instantiated as hardware, software, or a combination of hardware and software. In some embodiments, the encoder component 114 is configured to decode the encoded video stream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings from the encoded video stream 108.

[0032] In some embodiments, the server system 112 functions as a Media-Aware Network Element (MANE). For example, the server system 112 may be configured to prune the encoded video stream 108 to customize potentially different streams for one or more electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0033] The electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode the encoded video data 116 to generate an output video stream that can be presented on a display or other type of presentation device. In some embodiments, one or more electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or include a media storage device). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access the server system 112 to obtain the encoded video data 116.

[0034] The source device and / or multiple electronic devices 120 are sometimes referred to as “end devices” or “user devices.” In some embodiments, the source device 102 and / or one or more electronic devices 120 are examples of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing devices, and / or other types of electronic devices.

[0035] In an exemplary operation of the communication system 100, the source device 102 transmits an encoded video stream 108 to a server system 112. For example, the source device 102 can encode a picture stream captured by the source device. The server system 112 receives the encoded video stream 108 and can decode and / or encode the encoded video stream 108 using an encoder component 114. For example, the server system 112 can apply encoding to the video data, which is more optimized for network transmission and / or storage. The server system 112 can send the encoded video data 116 (e.g., one or more encoded video streams) to one or more electronic devices 120. Each electronic device 120 can decode the encoded video data 116 and optionally display the video picture.

[0036] Figure 2A1 is a block diagram illustrating exemplary elements of an encoder component 106 according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., a transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives the video sequence from a remote video source (e.g., a video source that is a component of a device different from the encoder component 106). The video source 104 may provide a source video sequence in the form of a digital video sample stream, which may have any suitable bit depth (e.g., 8 bits, 10 bits, or 12 bits), any color space (e.g., BT.601 Y CrCB, or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0, or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared videos. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The picture itself can be organized into a spatial pixel array, where each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples can be easily understood by those of ordinary skill in the art. The following description focuses on samples.

[0037] The encoder component 106 is configured to encode and / or compress the pictures of the source video sequence into an encoded video sequence 216 in real time or under other time constraints required by the application. Implementing the appropriate encoding speed is a function of the controller 204. In some embodiments, the controller 204 controls and is functionally coupled to other functional units as described below. The parameters set by the controller 204 may include rate control related parameters (e.g., lambda values ​​for picture skipping, quantizer, and / or rate-distortion optimization techniques), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. Other functions of the controller 204 may be readily identified by one of ordinary skill in the art as these functions may involve the encoder component 106 optimized for a particular system design.

[0038] In some embodiments, the encoder component 106 is configured to operate in an encoding loop. In a simplified example, the encoding loop includes a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder 210. The decoder 210 reconstructs the symbols to create sample data in a manner similar to that of the (remote) decoder (when the compression between the symbols and the encoded video code stream is lossless). The reconstructed sample stream (sample data) is input to the reference picture memory 208. Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory 208 are also bit-accurate between the local encoder and the remote encoder. In this way, the prediction part of the encoder interprets the reference picture samples as the same sample values ​​as the decoder will interpret when using prediction during decoding. This reference picture synchronization principle (and the drift caused when synchronization cannot be maintained, such as due to channel errors) is known to those of ordinary skill in the art.

[0039] The operation of decoder 210 may be combined with, for example, Figure 2B The remote decoder of the decoder assembly 122 described in detail is the same. However, briefly refer to Figure 2B Since the symbols are available and the entropy encoder 214 and the parser 254 can encode / decode the symbols into the encoded video sequence losslessly, the entropy decoding part of the decoder component 122 including the buffer memory 252 and the parser 254 may not be fully implemented in the local decoder 210.

[0040] The decoder techniques described herein (except for parsing / entropy decoding) can exist in a corresponding encoder in substantially the same functional form. For this reason, the disclosed subject matter focuses on the decoder operation. The description of the encoder techniques can be simplified because the encoder techniques are inverse to the decoder techniques.

[0041] As part of its operation, source encoder 202 may perform motion compensated predictive coding, which predictively encodes an input frame by referencing one or more previously encoded frames from a video sequence designated as reference frames. In this manner, encoding engine 212 encodes the differences between pixel blocks of an input frame and pixel blocks of a reference frame that may be selected as a prediction reference for the input frame. Controller 204 may manage the encoding operations of source encoder 202, including, for example, setting parameters and subgroup parameters for encoding video data.

[0042] The decoder 210 decodes the encoded video data of the frame that can be designated as the reference frame based on the symbols created by the source encoder 202. The operation of the encoding engine 212 can advantageously be a lossy process. When the encoded video data is in the video decoder ( Figure 2A When the reconstructed video sequence is decoded at a remote video decoder (not shown), the reconstructed video sequence may be a replica of the source video sequence with some errors. The decoder 210 replicates the decoding process that may be performed by the remote video decoder on the reference frame and may cause the reconstructed reference frame to be stored in the reference picture memory 208. In this way, the encoder component 106 locally stores a copy of the reconstructed reference frame that has common content (absent transmission errors) with the reconstructed reference frame that will be obtained by the remote video decoder.

[0043] The predictor 206 may perform a prediction search for the encoding engine 212. For a new frame to be encoded, the predictor 206 may search the reference picture memory 208 for sample data (as candidate reference pixel blocks) or certain metadata, such as reference picture motion vectors, block shapes, etc., that may be used as appropriate prediction references for the new picture. The predictor 206 may operate on a pixel block by pixel block basis to find a suitable prediction reference. In some cases, as determined by the search results obtained by the predictor 206, the input picture may have prediction references taken from multiple reference pictures stored in the reference picture memory 208.

[0044] The outputs of all the above functional units may be entropy encoded in the entropy encoder 214. The entropy encoder 214 losslessly compresses the symbols generated by the various functional units according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, and / or arithmetic coding) to convert the symbols into a coded video sequence.

[0045] In some embodiments, the output of the entropy encoder 214 is coupled to a transmitter. The transmitter may be configured to buffer the encoded video sequence created by the entropy encoder 214, thereby preparing for transmission through a communication channel 218, which may be a hardware / software link to a storage device that can store the encoded video data. The transmitter may be configured to merge the encoded video data from the source encoder 202 with other data to be transmitted, such as encoded audio data and / or an auxiliary data stream (source not shown). In some embodiments, the transmitter may transmit additional data when transmitting the encoded video. The source encoder 202 may include such data as part of the encoded video sequence. The additional data may include time / space / signal-to-noise ratio (SNR) enhancement layers, other forms of redundant data such as redundant pictures and slices, auxiliary enhancement information (SEI) messages, video usability information (VUI) parameter set fragments, etc.

[0046] The controller 204 may manage the operation of the encoder component 106. During encoding, the controller 204 may assign a certain encoded picture type to each encoded picture, but this may affect the encoding technique applied to the corresponding picture. For example, a picture may be assigned as an intra picture (I picture), a predictive picture (P picture), or a bidirectional predictive picture (B picture). An intra picture may be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, Independent Decoder Refresh (IDR) pictures. Those of ordinary skill in the art are aware of these variants of I pictures and their corresponding applications and features, so they are not discussed here. Predictive pictures may be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block. Bidirectional predictive pictures may be encoded and decoded using intra prediction or inter prediction, which uses at most two motion vectors and reference indexes to predict the sample values ​​of each block. Similarly, multiple predictive pictures may use more than two reference pictures and associated metadata for reconstructing a single block.

[0047] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples) and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined by the coding allocation applied to the corresponding picture of the block. For example, blocks of an I picture may be non-predictively coded, or blocks of an I picture may be predictively coded (spatial prediction or intra-frame prediction) with reference to already coded blocks of the same picture. Blocks of pixels of a P picture may be non-predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. Blocks of a B picture may be non-predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0048] Video may be captured as multiple source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often shortened to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In one example, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0049] The encoder component 106 may perform encoding operations according to any predetermined video encoding technique or standard such as described herein. In operation, the encoder component 106 may perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in the input video sequence. Thus, the encoded video data may conform to the syntax specified by the video encoding technique or standard used.

[0050] Figure 2B is a block diagram illustrating exemplary elements of decoder component 122 according to some embodiments. Figure 2B The decoder component 122 in is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to transmit data to the display 124 (eg, via a wired or wireless connection).

[0051] In some embodiments, the decoder component 122 includes a receiver coupled to the channel 218 and configured to receive data from the channel 218 (e.g., via a wired or wireless connection). The receiver may be configured to receive one or more encoded video sequences to be decoded by the decoder component 122. In some embodiments, the decoding of each encoded video sequence is independent of the decoding of other encoded video sequences. Each encoded video sequence may be received from the channel 218, which may be a hardware / software link leading to a storage device storing the encoded video data. The receiver may receive encoded video data and other data that may be forwarded to their respective use entities (not depicted), such as encoded audio data and / or auxiliary data streams. The receiver may separate the encoded video sequence from other data. In some embodiments, the receiver receives additional (redundant) data when receiving the encoded video. The additional data may be included as part of the encoded video sequence. The additional data may be used by the decoder component 122 to decode the data and / or more accurately reconstruct the original video data. The additional data may take the form of, for example, time, space or SNR enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0052] According to some embodiments, the decoder component 122 includes a buffer memory 252, a parser 254 (sometimes also referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra picture prediction unit 262, a motion compensated prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, the decoder component 122 is implemented as one or more integrated circuits and / or other electronic circuits. In some embodiments, the decoder component 122 is at least partially implemented in software.

[0053] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to prevent network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, in addition to buffer memory 252 located inside decoder component 122 (e.g., configured to handle playout timing), a separate buffer memory is provided outside decoder component 122 (e.g., to prevent network jitter). When receiving data from a store / forward device with sufficient bandwidth and controllability or from an isochronous network, buffer memory 252 may not be required, or buffer memory 252 may be smaller. For use on a traffic packet network such as the Internet, buffer memory 252 may be required, buffer memory 252 may be relatively large, advantageously may have an adaptive size, and may be implemented at least in part in an operating system or similar element (not depicted) external to decoder component 122.

[0054] The parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. These symbols may include, for example, information for managing the operation of the decoder component 122, and / or information for controlling a rendering device such as the display 124. The control information for the rendering device may be in the form of, for example, a Supplementary Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not depicted). The parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be performed according to a video coding technique or standard, and may follow various principles known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 254 may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. The subgroup may include a group of pictures (GOP), a picture, a tile, a slice, a macroblock, a coding unit (CU), a block, a transform unit (TU), a prediction unit (PU), etc. The parser 254 may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, etc.

[0055] Depending on the type of the coded video picture or a portion of the coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol 270 may involve multiple different units. Which units are involved and how they are involved can be controlled by the parser 254 through subgroup control information parsed from the coded video sequence. For clarity, such subgroup control information flow between the parser 254 and the multiple units below is not depicted.

[0056] The decoder component 122 may be conceptually subdivided into multiple functional units, which in some implementations closely interact with each other and may be at least partially integrated with each other. However, for the sake of clarity, the conceptual subdivision into multiple functional units is maintained hereinafter.

[0057] The sealer / inverse transform unit 258 receives the quantized transform coefficients and control information (e.g., which transform to use, block size, quantization factor, and / or quantization scaling matrix) as symbols 270 from the parser 254. The sealer / inverse transform unit 258 may output a block including sample values, which may be input into the aggregator 268.

[0058] In some cases, the output samples of the sealer / inverse transform unit 258 belong to intra-coded blocks, i.e., blocks that do not use prediction information from previously reconstructed pictures, but can use prediction information from previously reconstructed portions of the current picture. Such prediction information may be provided by the intra picture prediction unit 262. The intra picture prediction unit 262 may use surrounding reconstructed information extracted from the current (partially reconstructed) picture from the current picture memory 264 to generate a block of the same size and shape as the block being reconstructed. The aggregator 268 may add the prediction information generated by the intra picture prediction unit 262 to the output sample information provided by the sealer / inverse transform unit 258 on a per-sample basis.

[0059] In other cases, the output samples of the scaler / inverse transform unit 258 belong to a block that is inter-coded and potentially motion compensated. In this case, the motion compensated prediction unit 260 may access the reference picture memory 266 to extract samples for prediction. After the extracted samples are motion compensated according to the symbol 270 belonging to the block, these samples may be added to the output of the scaler / inverse transform unit 258 (in this case, referred to as residual samples or residual signals) by the aggregator 268, thereby generating output sample information. The extraction of prediction samples by the motion compensated prediction unit 260 from the address within the reference picture memory 266 may be controlled by a motion vector. The motion vector may be provided to the motion compensated prediction unit 260 for use in the form of a symbol 270, which may have, for example, an X component, a Y component, and a reference picture component. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory 266 when using sub-sample accurate motion vectors, motion vector prediction mechanisms, etc.

[0060] The output samples of aggregator 268 may be subjected to various loop filtering techniques in loop filter unit 256. The video compression techniques may include in-loop filter techniques that are controlled by parameters included in the coded video bitstream and available to loop filter unit 256 as symbols 270 from parser 254, and may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of a coded picture or coded video sequence, and to previously reconstructed and loop filtered sample values.

[0061] The output of the loop filter unit 256 may be a sample stream that may be output to a rendering device (eg, the display 124 ) as well as stored in the reference picture memory 266 for future inter-picture prediction.

[0062] Once reconstructed, certain coded pictures may be used as reference pictures for future prediction. Once a coded picture is reconstructed, and the coded picture is identified as a reference picture (e.g., by parser 254), the current reference picture may become part of reference picture memory 266, and a new current picture memory may be reallocated before starting reconstruction of a subsequent coded picture.

[0063] The decoder component 122 may perform decoding operations according to a predetermined video compression technique that may be recorded in a standard (e.g., any of the standards described herein). In the sense that the encoded video sequence follows the syntax of the video compression technique or standard, the encoded video sequence may conform to the syntax specified by the video compression technique or standard used (as specified in the video compression technology document or standard, particularly in the profile of the video compression technology or standard). In addition, in order to comply with some video compression techniques or standards, the complexity of the encoded video sequence may be within the range defined by the hierarchy of the video compression technology or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the hypothetical reference decoder (HRD) specification and metadata of the HRD buffer management signaled in the encoded video sequence.

[0064] Figure 3 306, and a plurality of communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., CPUs, GPUs, and / or DPUs). In some embodiments, the control circuit includes one or more field programmable gate arrays (FPGAs), hardware accelerators, and / or one or more integrated circuits (e.g., application specific integrated circuits).

[0065] The network interface 304 may be configured to connect to one or more communication networks (e.g., wireless networks, wired networks, and / or optical networks). The communication network may be a local network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a delay-tolerant network, etc. Examples of communication networks include local area networks such as Ethernet, wireless LANs, cellular networks including GSM, 3G, 4G, 5G, LTE, etc., television wired or wireless wide area digital networks including cable TV, satellite TV, and terrestrial broadcast television, vehicle and industrial networks including CANBus, etc. Such communications may be one-way receiving only (e.g., broadcast television), one-way sending only (e.g., CANBus connected to certain CANBus devices), or bidirectional (e.g., using a local area network or wide area network digital network to connect to other computer systems). Such communications may include communications to one or more cloud computing networks.

[0066] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input device 310 may include one or more of a keyboard, a mouse, a trackpad, a touch screen, a data glove, a joystick, a microphone, a scanner, a camera, etc. The output device 308 may include one or more of an audio output device (e.g., a speaker), a visual output device (e.g., a display or a display), etc.

[0067] The memory 314 may include high-speed random access memory (e.g., DRAM, SRAM, DDR RAM, and / or other solid-state random access memory devices) and / or non-volatile memory (e.g., one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and / or other non-volatile solid-state storage devices). Optionally, the memory 314 includes one or more storage devices arranged away from the control circuit 302. The memory 314, or alternatively, the non-volatile solid-state memory device within the memory 314, includes a non-transitory computer-readable storage medium. In some embodiments, the memory 314 or the non-transitory computer-readable storage medium of the memory 314 stores the following programs, modules, instructions, and data structures, or a subset or superset thereof:

[0068] An operating system 316, which includes programs for handling various basic system services and for performing hardware-related tasks;

[0069] A network communication module 318 for connecting the server system 112 to other computing devices via one or more network interfaces 304 (eg, via wired and / or wireless connections);

[0070] An encoding module 320 for performing various functions related to encoding and / or decoding data (e.g., video data). In some embodiments, the encoding module 320 is an instance of the encoder component 114. The encoding module 320 includes, but is not limited to, one or more of the following:

[0071] o a decoding module 322 for performing various functions associated with decoding encoded data, such as those previously described with respect to the decoder component 122; and

[0072] o an encoding module 340 for performing various functions associated with encoding data, such as those previously described with respect to the encoder component 106; and

[0073] Picture memory 352 for storing pictures and picture data, e.g., for use with encoding module 320. In some embodiments, picture memory 352 includes one or more of reference picture memory 208, buffer memory 252, current picture memory 264, and reference picture memory 266.

[0074] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform the various functions previously described for the parser 254), a transform module 326 (e.g., configured to perform the various functions previously described for the scaler / inverse transform unit 258), a prediction module 328 (e.g., configured to perform the various functions previously described for the motion compensated prediction unit 260 and / or the intra-picture prediction unit 262), and a filter module 330 (e.g., configured to perform the various functions previously described for the loop filter 256).

[0075] In some embodiments, the encoding module 340 includes a code module 342 (e.g., configured to perform the various functions previously described for the source encoder 202 and / or the encoding engine 212) and a prediction module 344 (e.g., configured to perform the various functions previously described for the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes Figure 3 For example, the decoding module 322 and the encoding module 340 both use a shared prediction module.

[0076] Each module in the modules identified above stored in the memory 314 corresponds to an instruction set for performing the functions described herein. The modules identified above (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules, so each subset of these modules can be combined or otherwise rearranged in various embodiments. For example, optionally, the encoding module 320 does not include a separate decoding module and an encoding module, but uses the same set of modules to perform these two groups of functions. In some embodiments, the memory 314 stores a subset of the modules and data structures identified above. In some embodiments, the memory 314 stores additional modules and data structures not described above, such as an audio processing module.

[0077] Although Figure 3 A server system 112 is shown according to some embodiments, but Figure 3 The present invention is intended more as a functional description of various features that may exist in one or more server systems rather than as a schematic diagram of the structures of the embodiments described herein. In practice, as recognized by those of ordinary skill in the art, the items shown separately may be combined, and some items may be separated. For example, Figure 3 Some of the items shown individually in the figure may be implemented on a single server, and a single item may be implemented by one or more servers. The actual number of servers used to implement server system 112, and how features are distributed among the servers, will vary from implementation to implementation, optionally depending in part on the amount of data traffic that the server system handles during peak usage periods and during average usage periods.

[0078] Exemplary Encoding Processes and Techniques

[0079] The encoding processes and techniques described below may be performed at the devices and systems described above (eg, source device 102, server system 112, and / or electronic device 120). 4A to 4D An exemplary coding tree structure according to some embodiments is shown. Figure 4A As shown in the first coding tree structure (400) in , some coding methods (e.g., VP9) use a 4-way partition tree starting from the 64×64 level and reducing to the 4×4 level, which has some additional restrictions on 8×8 blocks. Figure 4A The partitioning specified as R in can be called recursive partitioning because the same partition tree is repeated in smaller sizes until a minimum 4×4 level is reached.

[0080] like Figure 4B As shown in the second coding tree structure (402) in , some coding methods (e.g., AV1) expand the partition tree to a 10-way structure and increase the maximum size (e.g., superblock in VP9 / AV1 parlance) to start at 128×128. The second coding tree structure includes 4:1 / 1:4 rectangular partitions that are not in the first coding tree structure. Figure 4B The partition type with 3 sub-partitions in the second row of is called T-type partition. In addition to the coding block size, a coding tree depth may also be defined to indicate the partition depth from the root node.

[0081] As an example, for example in HEVC, a CTU can be divided into CUs using a quadtree structure represented as a coding tree to accommodate various local characteristics. In some embodiments, a decision as to whether to use inter-picture (temporal) prediction or intra-picture (spatial) prediction to encode a picture area is made at the CU level. Depending on the PU partition type, each CU can be further divided into one, two, or four PUs. Within a PU, the same prediction process is applied, and relevant information is transmitted to the decoder based on the PU. After obtaining the residual block by applying a prediction process based on the PU partition type, the CU can be divided into TUs according to another quadtree structure (similar to the coding tree for the CU).

[0082] For example, in VVC, a quadtree with a nested multi-type tree can replace the concept of multiple partition unit types using a binary and ternary partitioning structure. For example, a quadtree removes the separation of the concepts of CU, PU, ​​and TU (except for the need for CUs that are too large for the maximum transform length), and a quadtree supports greater flexibility in the shape of CU partitions. In the coding tree structure, a CU can have a square or rectangular shape. CTUs are first partitioned by a four-level tree (also called a quadtree) structure. The leaf nodes of the four-level tree can be further partitioned by a multi-type tree structure. Figure 4C As shown in the third coding tree structure (404) in FIG. 4 , the multi-type tree structure includes four types of partitioning. The leaf nodes of the multi-type tree are called CUs, and unless the CU is too large for the maximum transform length, the segmentation is used for prediction and transform processing without any further partitioning. In most cases, CUs, PUs, and TUs have the same block size in a quadtree with a nested multi-type tree coding block structure. An example of a block partition of a CTU (406) is shown in FIG. Figure 4D As shown, Figure 4D An exemplary quadtree with a nested multi-type tree coding block structure is shown.

[0083] A fixed set of intra prediction angles can be used to remove spatial redundancy in video signals. For directional intra prediction, some methods support 8 directional modes corresponding to angles from 45 degrees to 207 degrees. In order to exploit a wider variety of spatial redundancy in directional textures, directional intra modes can be extended to a set of angles with finer granularity. For example, 8 angles can be represented as nominal angles. Figure 5A 8 nominal angles named V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED and D67_PRED are shown. Figure 5A As shown, for each nominal angle, there may be 7 finer angles, for a total of 56 directional angles. The predicted angle may be described by the nominal intra angle plus an angle increment (e.g., -3 to 3 times a step size of 3 degrees). In order to implement the directional prediction modes in a general manner, the 56 directional intra prediction modes may be implemented using a unified directional predictor that projects each pixel to a reference sub-pixel position and interpolates the reference pixel through a 2-tap bilinear filter. This directional intra prediction is sometimes also referred to as unidirectional intra prediction.

[0084] In some methods, a lookup table is used to map each intra prediction angle to a horizontal offset and a vertical offset between each pixel in the current block and a reference sample. For each intra prediction angle, the offset in the lookup table can be an integer value of the tangent of the angle multiplied by 64. For example, 64 is equal to the tangent of (45°) multiplied by 64, where the tangent of (45°) is equal to 1. For example, for an intra prediction angle of 45 degrees, the associated offset is 64, and the horizontal offset between each pixel in the current block and the reference pixel increases by 1 pixel as the row number of the pixel increases by 1.

[0085] In some methods, there are also non-directional smooth intra prediction modes, such as DC, PAETH, SMOOTH, SMOOTH-V, and SMOOTH-H. For DC prediction, the average of the left neighboring samples and the upper neighboring samples can be used as the predictor for the block to be predicted. For PAETH prediction, the top reference sample, the left reference sample, and the upper left reference sample can be extracted, and the value closest to the given value (e.g., top + left - upper left) is set as the predictor for the pixel to be predicted. Figure 5B The positions of the top sample 514, the left sample 518, and the upper left sample 516 of the current pixel (or small block or sub-block) in the current block are shown. For SMOOTH mode, SMOOTH-V mode, and SMOOTH-H mode, quadratic interpolation in the vertical direction or the horizontal direction or the average of the two directions can be used to predict the block.

[0086] In order to capture the attenuated spatial correlation using references on edges, filter intra modes can be used for luma blocks. In some methods, five filter intra modes are defined, where each filter intra mode is represented by a set of eight 7-tap filters, which reflects the correlation between a pixel in a 4×2 small block (subblock) and its 7 neighbors. In this way, the weighting factors of the 7-tap filters depend on the position. For example, Figure 5C As shown in , the 8×8 block can be divided into 8 4×2 small blocks. Figure 5C In FIG. 1 , the blocks are indicated by B0, B1, B2, B3, B4, B5, B6 and B7. For each block, 7 neighbors (indicated by R0 to R7) are used to predict the pixels in the current block. For block B0, all neighbors have been reconstructed. However, for other blocks, some neighbors may not be reconstructed. In this case, the predicted values ​​of the direct neighbors can be used as a reference. For example, all neighbors of block B7 may not be reconstructed, so instead, the predicted samples of neighbors (such as B5 and B6) are used.

[0087] The directional intra prediction process may include as input the following items: a variable plane specifying which plane is being predicted, variables x and y specifying the position of the upper left sample in the CurrFrame[plane] array of the current transform block, a variable haveLeft (if there is a valid sample to the left of the transform block, the variable haveLeft is equal to 1), a variable haveAbove (if there is a valid sample above the transform block, the variable haveAbove is equal to 1), a variable mode specifying the type of intra prediction to be applied, a variable w specifying the width of the area to be predicted, a variable h specifying the height of the area to be predicted, a variable maxX specifying the maximum valid x-coordinate of the current plane, and a variable maxY specifying the maximum valid y-coordinate of the current plane.

[0088] The output of the directional intra prediction process can be the angular prediction value P A The angle prediction value can be based on the prediction angle pAngle, the row above AboveRow, or the column on the left LeftCol. For example, if the prediction angle is less than 180 degrees, the angle prediction value can be derived from the AboveRow sample, and if the prediction angle is greater than 180 degrees, the angle prediction value can be derived from the LeftCol sample.

[0089] In some systems, angular (directional) prediction modes do not take into account the uneven distribution of available reference samples (e.g., only top and left reference samples may be available), and therefore, the prediction accuracy of these angular modes may be reduced compared to modes that weight top and left reference samples more heavily. The systems and methods described below may improve prediction accuracy and precision compared to angular prediction modes. The methods and processes described below may be used alone or in combination in any order. In the description below, a mode is referred to as a directional mode if it generates prediction samples according to a given prediction direction.

[0090] As used herein, a left reference sample refers to a reference sample whose vertical coordinate value is within the minimum vertical coordinate value and the maximum vertical coordinate value of a current block (eg, current block 522). Figure 5D The top reference sample refers to a reference sample whose horizontal coordinate value is within the minimum horizontal coordinate value and the maximum horizontal coordinate value of the current block, as shown in the left part 528 of FIG. Figure 5D The lower left reference sample refers to a reference sample whose vertical coordinate value is greater than the maximum vertical coordinate value of the current block, such as Figure 5D The upper right reference sample refers to a reference sample whose horizontal coordinate value is greater than the maximum horizontal coordinate value of the current block, such as Figure 5D The upper left reference sample refers to a reference sample whose horizontal coordinate value is less than the minimum horizontal coordinate value of the current block and whose vertical coordinate value is less than the minimum vertical coordinate value of the current block, as shown in the upper right part 526 of FIG. Figure 5D As shown in the upper left portion 532 of FIG.

[0091] Figure 5D Also shown is a predicted angle 538 for sample 536 of current block 522. Predicted angle 538 leads to top sample 542 in upper right portion 526 and bottom sample 540 in lower left portion 530. Figure 5E Predicted angle 550 is shown for sample 552. Predicted angle 550 leads to top sample 556 and bottom sample 554. In some embodiments, predicted angles 538 and 550 are based on an angle pattern (eg, a particular angle pattern indicates a corresponding predicted angle).

[0092] Fig. 6A6 is a flow chart illustrating a method 600 of encoding a video according to some embodiments. The method 600 may be performed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having a control circuit and a memory storing instructions executed by the control circuit. In some embodiments, the method 600 is performed by executing instructions stored in a memory (e.g., memory 314) of the computing system.

[0093] The system receives (602) video data, the video data including a plurality of blocks, the plurality of blocks including a first block, wherein the first block is encoded in an angular mode (e.g., a directional intra prediction mode). The system identifies (604) a reference sample set (e.g., top samples 542 and / or bottom samples 540) for a portion of the first block, wherein the reference sample set is identified using a prediction angle of the angular prediction mode. The system derives (606) a first angular prediction value for the portion of the first block using at least a first subset of the reference sample set. The system derives (608) a second angular prediction value for the portion of the first block using a weighted sum of at least a second subset of the reference sample set and the first angular prediction value. The system encodes (610) the portion of the first block using the second angular prediction value. For example, the system reconstructs the first block based on the second angular prediction value, evaluates the angular prediction mode based on the reconstructed first block, and selects the angular prediction mode for encoding the first block based on the evaluation (e.g., the angular prediction mode having the smallest error).

[0094] Figure 6B 6 is a flow chart illustrating a method 650 of decoding a video according to some embodiments. The method 650 may be performed at a computing system (e.g., server system 112, source device 102, or electronic device 120) having a control circuit and a memory storing instructions executed by the control circuit. In some embodiments, the method 650 is performed by executing instructions stored in a memory (e.g., memory 314) of the computing system.

[0095] The system receives (652) video data from a video code stream, the video data including a plurality of blocks, the plurality of blocks including a first block, wherein the first block is predicted in an angular prediction mode (e.g., a directional intra prediction mode). The system identifies (654) a reference sample set (e.g., top samples 542 and / or bottom samples 540) for a portion of the first block, wherein the reference sample set is identified using a prediction angle of the angular prediction mode. The system derives (656) a first angular prediction value for the portion of the first block using at least a first subset of the reference sample set. The system derives (658) a second angular prediction value for the portion of the first block using a weighted sum of at least a second subset of the reference sample set and the first angular prediction value. The system decodes (660) the portion of the first block using the second angular prediction value.

[0096] Although Fig. 6A and Figure 6B The various logical stages are shown in a particular order, but stages that are not order-dependent may be reordered, and other stages may be combined or decomposed. Some reordering or other groupings not specifically mentioned will be apparent to one of ordinary skill in the art, and thus the ordering and groupings presented herein are not exhaustive. Furthermore, it should be appreciated that the various stages may be implemented in hardware, firmware, software, or any combination thereof.

[0097] In some embodiments, in order to use spatially adjacent reference samples to predict samples in the current block, the angular prediction value P is first derived. A , then use the left reference sample and P A The final prediction value P' is derived by taking the weighted sum of A (eg, a horizontal blend mode corresponding to an angle mode). In some embodiments, the angle prediction value P is derived along a given prediction angle using reference samples from the top neighbor and / or the top right neighbor of the current block. A In some embodiments, when generating P A The reference samples are filtered before the values ​​are generated. A Before filtering the values, one or more smoothing filters (e.g., a Gaussian filter or a bilateral filter) are used to filter the left reference samples, the top reference samples, the upper left reference samples, the lower left reference samples, and the upper right reference samples, wherein the coefficients of the smoothing filters are all non-negative integers. For example, the disclosed method may identify a reference sample set for a first block (e.g., a portion of the first block), wherein the reference sample set is identified using a prediction angle of an angular prediction mode. Then, the disclosed method may derive a first angular prediction value for the portion of the first block using at least a first subset of the reference sample set. Subsequently, the disclosed method may derive a second angular prediction value for the portion of the first block using a weighted sum of (i) at least a second subset of the reference sample set and (ii) the derived first angular prediction value.

[0098] In some embodiments, P' is derived using the following equation 1: A .

[0099] P' A =(W L ·L+(NW L )·P A |+r) / N

[0100] Equation 1 - Refined angular prediction

[0101] Among them, the horizontal coordinate value of the current sample is used to derive w L, N is a predetermined value (e.g., an integer that is a power of 2, such as 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024), and r is a rounding offset (e.g., equal to 0.5·N). L is the left / bottom-left sample and can be obtained by extending the direction from the top reference to the left / bottom-left sample of the predicted value, such as Figure 5D shown.

[0102] In some embodiments, w L Derived as K>>((x<<1))>>s, where K is a predetermined value (e.g., 16, 32, or 64), x is the horizontal coordinate of the current sample to be predicted, and s is a scaling factor based on the block size. For example, s may be equal to (log2(W)+log2(H)+2)>>2, where W and H are the width and height of the block, respectively.

[0103] In some embodiments, if L is outside the lower left reference sample, L is thinned by some predetermined values ​​in the left or lower left portion. In some embodiments, when L is outside the lower left reference sample, a predetermined sample in the left or lower left portion is used. In some embodiments, L is filled using the nearest available sample in the lower left region, the left region, the upper left region, the top region, and / or the upper right region.

[0104] In some embodiments, for the refined (final) predictor, P is calculated A The division operation at the time of the prediction sample is moved to the last step. For example, during internal calculations, the parameters can be multiplied by a value so that these parameters have the same divisor.

[0105] In some embodiments, when the angle mode is in Figure 5A When the system can find the left sample by extending the given angular direction to the left reference sample, the refined predictor P' of Equation 1 is used. A (For example, applying a Horizontal blend mode).

[0106] In some embodiments, in order to use spatially adjacent reference samples to predict samples in the current block, the angular prediction value P is first derived. A , then use the top reference sample and P A The final prediction value P' is derived by taking the weighted sum of A (For example, a vertical blend mode corresponding to an angle mode.) In some embodiments, the angle prediction value P is derived along a given prediction angle using reference samples from the left neighboring position and / or the bottom left neighboring position of the current block. A In some embodiments, when generating P A The reference samples are filtered before the values ​​are generated. ABefore the values ​​are obtained, one or more smoothing filters (eg, Gaussian filters or bilateral filters) are used to filter the left reference samples, the top reference samples, the upper left reference samples, the lower left reference samples, and the upper right reference samples, wherein the coefficients of the smoothing filters are all non-negative integers.

[0107] In some embodiments, P' is derived using Equation 2 below A .

[0108] P' A =(W T ·T+((NW T )·P A +r) / N

[0109] Equation 2 - Refined angular prediction

[0110] Among them, the vertical coordinate value of the current sample is used to derive w T , N is a predetermined value (e.g., an integer that is a power of 2, such as 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024), and r is a rounding offset (e.g., equal to 0.5·N). T is the top / upper-right sample and may be obtained by extending the direction from the top reference to the left / lower-left sample of the predicted value, as Figure 5D shown.

[0111] In some embodiments, w T Derived as K>>((y<<1))>>s, where K is a predetermined value (e.g., 16, 32, or 64), y is the vertical coordinate of the current sample to be predicted, and s is a scaling factor based on the block size. For example, s may be equal to (log2(W)+log2(H)+2)>>2, where W and H are the width and height of the block, respectively.

[0112] In some embodiments, if T is outside the top / upper right reference sample, a predetermined sample in the top / upper right region is assigned to T. In some embodiments, T is filled with the nearest available sample in the upper right region, the top region, the upper left region, the left region, and / or the lower left region.

[0113] In some embodiments, for the refined (final) predictor, P is calculated A The division operation at the time of the prediction sample is moved to the last step. For example, during internal calculations, the parameters can be multiplied by a value so that these parameters have the same divisor.

[0114] In some embodiments, when the angle mode is in Figure 5A When the value of H_PRED is below , the refined predictor P' of Equation 2 is used A (For example, applying a vertical blend mode).

[0115] In some embodiments, to predict samples in the current block using spatially neighboring reference samples, the top (or left) reference samples (e.g., Figure 5E ) and the bottom (or right) reference sample (e.g., Figure 5E The angle prediction value P is derived by interpolating the bottom sample 554 in A , and then use the top (or left) reference sample (e.g. Figure 5E The top sample 556) and P A The weighted sum of A (eg, an angle blending mode corresponding to an angle mode).

[0116] In some embodiments, the angular prediction value P is derived along a given prediction angle using reference samples from the top reference sample, the left reference sample, the bottom reference sample, and / or the right reference sample of the current block. A In some embodiments, top (and / or left) reference samples (e.g., top sample 556) are derived using interpolation of multiple top (and / or left) reference samples at integer positions as input. In some embodiments, right (or bottom) reference samples (e.g., bottom sample 554) are derived using reference samples from the top or left reference. For example, the derivation of the right (or bottom) reference samples (e.g., bottom sample 554) can be the same as in the smooth mode, the hybrid mode, or the planar mode. In some embodiments, the top (and / or left) reference samples (e.g., top sample 556) are derived using interpolation of multiple top (and / or left) reference samples at integer positions as input. In some embodiments, the right (or bottom) reference samples (e.g., bottom sample 554) are derived using reference samples from the top or left reference. For example, the derivation of the right (or bottom) reference samples (e.g., bottom sample 554) can be the same as A The reference samples are filtered before the values ​​are generated. A Before the values ​​are obtained, one or more smoothing filters (eg, Gaussian filters or bilateral filters) are used to filter the left reference samples, the top reference samples, the upper left reference samples, the lower left reference samples, and the upper right reference samples, wherein the coefficients of the smoothing filters are all non-negative integers.

[0117] In some embodiments, when the top reference sample is used to generate P A When P' is derived using the following equation 3 A For example, by bilinear interpolation of T and B, P is generated. A .

[0118] P' A =(W·T+((NW)·P A +r) / N

[0119] Equation 3 - Refined angular prediction

[0120] Wherein, w is derived using the vertical coordinate value of the current sample, N is a predetermined value (e.g., an integer as a power of 2, such as 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024), and r is a rounding offset (e.g., equal to 0.5·N). T is the top / upper right sample and can be obtained by extending from the left sample along a predetermined angle to the top or lower right, such as Figure 5D shown.

[0121] In some embodiments, w is derived as K>>((y<<1))>>s, where K is a predetermined value (e.g., 16, 32, or 64), y is the vertical coordinate of the current sample to be predicted, and s is a scaling factor based on the block size. For example, s may be equal to (log2(W)+log2(H)+2)>>2, where W and H are the width and height of the block, respectively.

[0122] In some embodiments, if T is outside the top / upper right reference sample, a predetermined sample in the top / upper right region is assigned to T. In some embodiments, T is filled with the nearest available sample in the upper right region, the top region, the upper left region, the left region, and / or the lower left region.

[0123] In some embodiments, when the left reference sample is used to generate P A (e.g., by replacing T with the left reference sample), P' is derived using Equation 3 A .

[0124] In some embodiments, for the refined (final) predictor, P is calculated A The division operation at the time of the prediction sample is moved to the last step. For example, during internal calculations, the parameters can be multiplied by a value so that these parameters have the same divisor.

[0125] In some embodiments, the angle pattern is started from the top sample or the left sample along the angle direction, and when mixed to obtain P' A When , use the bottom sample B. For example, use the following equation 4 to derive P' A .

[0126] P' A =(w·B+((Nw)·P A +r) / N

[0127] Equation 3 - Refined angular prediction

[0128] In some embodiments, whether to use a mixed mode (eg, whether to derive P' A And use P' ATo predict samples) is signaled at the sequence header, slice header, super block or coding block level. In some embodiments, if the block is in intra mode, a hybrid mode is used (e.g., deriving P' A And use P' A For example, when a block is in intra-inter mode, a conventional intra mode is used, such as smooth mode, smooth-v mode, or smooth-h mode in AV1.

[0129] (A1) In one aspect, some embodiments include a video encoding method (e.g., method 600). In some embodiments, the method is performed at a computing system (e.g., server system 112) having a memory and a control circuit. In some embodiments, the method is performed at an encoding module (e.g., encoding module 320). In some embodiments, the method is performed at a source encoding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving video data, the video data including a plurality of blocks, the plurality of blocks including a first block, wherein the first block is encoded in a smooth mode; (ii) identifying a reference sample set for the first block; (iii) deriving a first prediction value for the first block, wherein the first prediction value is derived by a linear interpolation function involving a first reference sample in a reference sample set and a height and a width of the first block; and (iv) encoding the first block based on a first angle prediction value.

[0130] (A2) In some embodiments of A1, the second subset of reference samples includes left samples located to the left of the portion of the first block. In some embodiments, the left samples are identified by extending a direction from the top reference through the portion to the left samples. In some embodiments, the left samples include refined samples that are refined using one or more predetermined values. In some embodiments, the first angular prediction value is derived using interpolation of left samples and right samples of the second subset of reference samples, the right samples being located to the right of the portion of the first block.

[0131] (A3) In some embodiments of A1, the second subset of reference samples includes a top sample located above the portion of the first block. In some embodiments, the top sample is identified by extending a direction from a left reference through the portion to the top sample. In some embodiments, the top sample includes a refined sample that is refined using one or more predetermined values. In some embodiments, the first angular prediction value is derived using an interpolation of the top sample and a bottom sample of the second subset of reference samples, the bottom sample being located below the portion of the first block.

[0132] (A4) In some embodiments of any one of A1 to A3, the method further comprises: filtering the reference sample set before using at least a first subset of the reference sample set to derive the first angular prediction value for the portion of the first block.

[0133] (A5) In some embodiments of any one of A1 to A4, deriving the second angle prediction value includes: deriving a weighting factor based on the coordinates of the portion of the first block.

[0134] (A6) In some embodiments of any one of A1 to A5, the first angle prediction value is derived without performing a division operation. In some embodiments, the first angle prediction value is derived using interpolation of two or more reference samples. In some embodiments, the first angle prediction value is derived using a top reference sample, a left reference sample, a bottom reference sample, and a right reference sample.

[0135] (A7) In some embodiments of any one of A1 to A6, based on determining that the prediction angle of the angle prediction mode is within a predetermined angle range, the portion of the first block is encoded using a second angle prediction value.

[0136] (A8) In some embodiments of any one of A1 to A7, the second subset of reference samples includes bottom samples located below the portion of the first block.

[0137] (A9) In some embodiments of any one of A1 to A8, the method further comprises transmitting the encoded first block via a video bitstream. In some embodiments, the method further comprises signaling the use of the second angle prediction value (e.g., signaling that the angle blending mode is enabled).

[0138] (B1) In another aspect, some embodiments include a video decoding method (e.g., method 650). In some embodiments, the method is performed at a computing system (e.g., server system 112) having a memory and a control circuit. In some embodiments, the method is performed at an encoding module (e.g., encoding module 320). In some embodiments, the method is performed at a parser (e.g., parser 254), a motion prediction component (e.g., motion compensation prediction unit 260), and / or an intra-frame prediction component (e.g., intra-frame picture prediction unit 262). The method includes: (i) receiving video data (e.g., an encoded video sequence) from a video code stream, the video data including a plurality of blocks, the plurality of blocks including a first block, wherein the first block is predicted in an angular prediction mode; (ii) identifying a reference sample set for a portion of the first block, wherein the reference sample set is identified using a prediction angle of the angular prediction mode; (iii) deriving a first angular prediction value for the portion of the first block using at least a first subset of the reference sample set; (iv) deriving a second angular prediction value for the portion of the first block using a weighted sum of at least a second subset of the reference sample set and the first angular prediction value; and (v) decoding the portion of the first block using the second angular prediction value. For example, the angular prediction mode is an intra-frame prediction mode, and the angular prediction value P is derived along a given prediction angle using reference samples from a top neighboring position and / or an upper right neighboring position of the current block. A .

[0139] (B2) In some embodiments of B1, the second subset of reference samples includes left samples located to the left of the portion of the first block. For example, in order to use spatially adjacent reference samples to predict samples in the current block, the angle prediction value P is first derived. A , then use the left reference sample and P A The final prediction value P' is derived by taking the weighted sum of A In some embodiments, P' is derived using Equation 1 above. A .

[0140] (B3) In some embodiments of B2, the left sample is identified by extending the direction from the top reference through this portion to the left sample. For example, L is obtained by extending the direction from the top reference to the left / lower left sample of the predicted value.

[0141] (B4) In some embodiments of B2 or B3, the left samples include refined samples using one or more predetermined values. For example, if L is outside the lower left reference sample, L may be refined by some predetermined values ​​located in the left and / or lower left regions. As an example, when L is outside the lower left reference sample, predetermined samples located in the left or lower left regions may be used. As another example, the left samples are filled using the nearest available samples located in the lower left region, the left region, the upper left region, the upper region, and / or the upper right region.

[0142] (B5) In some embodiments of any one of B2 to B4, the first angular prediction value is derived using interpolation of left and right samples of the second subset of reference samples, where the right samples are located to the right of the portion of the first block. For example, P is generated by bilinear interpolation of top samples T and bottom samples B. A In some embodiments, P' is derived using Equation 3 above. A .

[0143] (B6) In some embodiments of B1, the second subset of reference samples includes top samples located above the portion of the first block. For example, in order to use spatially adjacent reference samples to predict samples in the current block, first derive an angular prediction value P A , then use the top reference sample and P A The final prediction value P' is derived by taking the weighted sum of A In some embodiments, P' is derived using Equation 2 above. A .

[0144] (B7) In some embodiments of B6, the top sample is identified by extending the direction from the left reference through this portion to the top sample. For example, the top sample T is obtained by extending the left sample along the predicted angle to the top / upper right portion.

[0145] (B8) In some embodiments of B6 or B7, the top samples include refined samples that are refined using one or more predetermined values. For example, if T is outside the top reference samples or the upper right reference samples, T may be refined by some predetermined samples located in the top region and / or the upper right region. As an example, T is filled with the nearest available samples located in the upper right region, the upper region, the upper left region, the left region, and / or the lower left region. In some embodiments, if T is outside the top / upper right reference samples, T is assigned predetermined samples located in the top region or the upper right region.

[0146] (B9) In some embodiments of any one of B6 to B8, the first angular prediction value is derived using interpolation of top samples and bottom samples of the second subset of reference samples, the bottom samples being located below the portion of the first block. For example, P is generated by bilinear interpolation of top samples T and bottom samples B. A In some embodiments, P' is derived using Equation 3 above. A .

[0147] (B10) In some embodiments of any one of B1 to B9, the method further comprises: filtering the reference sample set before using at least a first subset of the reference sample set to derive the first angular prediction value for the portion of the first block. A Before the values ​​are obtained, a smoothing filter (eg, a Gaussian filter or a bilateral filter) is used to filter the left reference sample, the upper reference sample, the upper left reference sample, the lower left reference sample and / or the upper right reference sample, wherein coefficients of the smoothing filter are all non-negative integers.

[0148] (B11) In some embodiments of any one of B1 to B10, deriving the second angular prediction value includes: deriving a weighting factor based on the coordinates of the portion of the first block. For example, w L It can be derived as K>>((x<<1))>>s, where K is a predetermined value, x is the horizontal coordinate of the current sample to be predicted, and s is a scaling factor based on the block size. For example, s may be equal to (log2(W)+log2(H)+2)>>2, where W and H are the width and height of the block, respectively.

[0149] (B12) In some embodiments of any one of B1 to B11, the first angle prediction value is derived without performing a division operation. For example, for the final predictor, P is calculated. A The division of the prediction samples is moved to the last step, and during internal calculations, the parameters may be multiplied by a value such that they have the same divisor.

[0150] (B13) In some embodiments of any one of B1 to B12, the first angular prediction value is derived using interpolation of two or more reference samples. For example, to predict samples in the current block using spatially adjacent reference samples, the angular prediction value P is first derived using interpolation of the top (or left) reference sample and the bottom (or right) reference sample. A , then use the top (or left) reference sample and P A The weighted sum of AAs an example, the top (or left) reference samples can be derived using interpolated values ​​of multiple top (or left) reference samples (e.g., located at integer positions) as input. As another example, the right (or bottom) reference samples can be derived using reference samples from the top or left reference. For example, the derivation of the right (or bottom) reference samples can be the same as the smooth mode, the hybrid mode, and / or the planar mode.

[0151] (B14) In some embodiments of B13, the first angle prediction value is derived using the top reference sample, the left reference sample, the bottom reference sample, and the right reference sample. For example, the angle prediction value P is derived along a given prediction angle using reference samples from the top reference sample, the left reference sample, the bottom reference sample, and the right reference sample of the current block. A .

[0152] (B15) In some embodiments of any one of B1 to B14, based on determining that the prediction angle of the angular prediction mode is within a predetermined angle range, the portion of the first block is decoded using the second angular prediction value. For example, when the prediction angle is to the right of V_PRED (e.g., when the decoder is able to find the left sample by extending the prediction angle to the left reference sample), the horizontal hybrid mode may be applied. As another example, when the prediction angle is below H_PRED, the vertical hybrid mode may be applied.

[0153] (B16) In some embodiments of any of B1 to B15, the second subset of reference samples includes bottom samples located below the portion of the first block. In some embodiments, P' is derived using Equation 3 A .

[0154] (B17) In some embodiments of any one of B1 to B15, the second angle prediction value is derived in response to a hybrid mode signaled in the video bitstream. For example, whether to use the hybrid mode is signaled at the sequence header, slice header, super block, or coding block level. In some embodiments, if the block is encoded in intra mode, the hybrid mode is used. In some embodiments, if the block is encoded in intra-inter mode, the hybrid mode is not applied. For example, for a block encoded in intra-inter mode, the intra mode is applied without hybrid, for example, using smooth mode, smooth-v mode, or smooth-h mode.

[0155] On the other hand, some embodiments include a computing system (e.g., server system 112) that includes a control circuit (e.g., control circuit 302) and a memory (e.g., memory 314) coupled to the control circuit, the memory storing one or more instruction sets configured to be executed by the control circuit, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1 to A9 and B1 to B17 above).

[0156] On the other hand, some embodiments include a non-transitory computer-readable storage medium that stores one or more instruction sets executed by control circuitry of a computing system, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1 to A9 and B1 to B17 above).

[0157] It should be understood that, although the terms "first", "second" etc. can be used in this article to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. The terms used herein are only used to describe specific embodiments and are not intended to limit claims. As used in the description of the embodiment and the attached claims, the singular forms "one", "one" and "the" are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "and / or" used herein refer to any and all possible combinations of one or more related listed items, and include any and all possible combinations of one or more related listed items. It should also be further understood that when used in this specification, the terms "include" and / or "comprising" specify the existence of stated features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups.

[0158] As used herein, the term "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "upon determining" or "in response to detecting" the stated antecedent is true, depending on the context. Similarly, the phrase "if it is determined that [the stated antecedent is true]" or "if [the stated antecedent is true]" or "when [the stated antecedent is true]" may be interpreted to mean "upon determining" or "upon determining" or "upon determining" or "upon detecting" or "in response to detecting" the stated antecedent is true, depending on the context.

[0159] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the illustrative discussion above is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in view of the above teachings. The embodiments are chosen and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art to implement.

Claims

1. A video decoding method, executed on a computing system having a memory and one or more processors, the method comprising: Receiving video data from a video code stream, the video data comprising a plurality of blocks, the plurality of blocks comprising a first block, wherein the first block is predicted in an angular prediction mode; identifying a reference sample set for a portion of the first block, wherein the reference sample set is identified using a prediction angle of the angular prediction mode; deriving a first angular prediction value for the portion of the first block using at least a first subset of the set of reference samples; deriving a second angular prediction value for the portion of the first block using (i) at least a second subset of the reference sample set and (ii) a weighted sum of the derived first angular prediction values; and The portion of the first block is decoded using the second angular prediction value.

2. The method according to claim 1, wherein: The second angular prediction value corresponds to a horizontal mix mode, and wherein the second subset of the reference samples includes left samples, the left samples being located to the left of the portion of the first block.

3. The method according to claim 2, wherein: The left sample is identified by extending a direction from a top reference through the portion to the left sample.

4. The method according to claim 2, wherein: The left samples include refined samples refined using one or more predetermined values.

5. The method according to claim 2, wherein: The first angular prediction value is derived using interpolation of the left and right samples of the second subset of the reference samples, the right samples being located to the right of the portion of the first block.

6. The method according to claim 1, wherein: The second angular prediction value corresponds to a vertical blend mode, and wherein the second subset of the reference samples includes top samples, the top samples being located above the portion of the first block.

7. The method according to claim 6, wherein: The top sample is identified by extending a direction from a left reference through the portion to the top sample.

8. The method according to claim 6, wherein: The top samples include refined samples refined using one or more predetermined values.

9. The method according to claim 6, wherein: The first angular prediction value is derived using interpolation of the top sample and a bottom sample of the second subset of the reference samples, the bottom sample being located below the portion of the first block.

10. The method according to claim 1, further comprising: The set of reference samples is filtered before using at least the first subset of the set of reference samples to derive the first angular prediction value for the portion of the first block.

11. The method according to claim 1, wherein: Deriving the second angular prediction value includes deriving a weighting factor based on the coordinates of the portion of the first block.

12. The method according to claim 1, wherein: The first angle prediction value is derived without performing a division operation.

13. The method according to claim 1, wherein: The first angle prediction value is derived using interpolation of two or more reference samples.

14. The method according to claim 13, wherein: The first angle prediction value is derived using a top reference sample, a left reference sample, a bottom reference sample, and a right reference sample.

15. The method according to claim 1, wherein: The portion of the first block is decoded using the second angular prediction value based on determining that the prediction angle of the angular prediction mode is within a predetermined angular range.

16. The method according to claim 1, wherein: The second subset of the reference samples includes bottom samples, the bottom samples being located below the portion of the first block.

17. The method according to claim 1, wherein: The second angle prediction value is derived in response to a hybrid mode signaled in the video stream.

18. A computing system comprising: Control circuit; Memory; as well as one or more instruction sets stored in the memory and configured to be executed by the control circuit, the one or more instruction sets comprising instructions for: Receiving video data from a video code stream, the video data comprising a plurality of blocks, the plurality of blocks comprising a first block, wherein the first block is predicted in an angular prediction mode; identifying a reference sample set for a portion of the first block, wherein the reference sample set is identified using a prediction angle of the angular prediction mode; deriving a first angular prediction value for the portion of the first block using at least a first subset of the set of reference samples; deriving a second angular prediction value for the portion of the first block using (i) at least a second subset of the reference sample set and (ii) a weighted sum of the derived first angular prediction values; and The portion of the first block is decoded using the second angular prediction value.

19. The computing system of claim 18, wherein: The second subset of the reference samples includes left samples, the left samples being located to the left of the portion of the first block.

20. A non-transitory computer-readable storage medium storing one or more instruction sets configured to be executed by a computing device having control circuitry and memory, the one or more instruction sets comprising instructions for: Video data is received from a video code stream, the video data comprising a plurality of blocks, the plurality of blocks comprising a first block, wherein, predicting the first block in an angular prediction mode; identifying a reference sample set for a portion of the first block, wherein the reference sample set is identified using a prediction angle of the angular prediction mode; deriving a first angular prediction value for the portion of the first block using at least a first subset of the set of reference samples; deriving a second angular prediction value for the portion of the first block using (i) at least a second subset of the reference sample set and (ii) a weighted sum of the derived first angular prediction values; and The portion of the first block is decoded using the second angular prediction value.

Images