Systems and methods for hybrid intra mode coding

Abstract

Various embodiments described herein include methods and systems for encoding and decoding video. In one aspect, a video decoding method includes receiving video data including a plurality of blocks from a video bitstream, the plurality of blocks including a first block, where the first block is over-encoded using one of a plurality of smoothing modes. The method further includes identifying a set of reference samples for the first block and deriving a first prediction value for the first block, where the first prediction value is derived from a second prediction value, the second prediction value is derived without using a division operation or a shift operation, and the second prediction value is derived from the first block. The last step of deriving the first predicted value includes a division operation or a shift operation. The method also includes decoding the first block using the first prediction value.

Description

Cross-references

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 465,451, filed on May 10, 2023, entitled “HYBRID MODES IN INTRA-MODE CODING,” and is a continuation of, and claims priority to, U.S. Patent Application No. 18 / 474,523, filed on September 26, 2023, entitled “SYSTEMS AND METHODS FOR HYBRID INTRA-MODE CODING.” Technical Field

[0002] The disclosed embodiments relate generally to video coding, including but not limited to systems and methods for hybrid-mode prediction for video encoding / decoding. Background Art

[0003] Digital video is supported by various electronic devices, such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. Electronic devices send and receive or otherwise transmit digital video data over a communication network, and / or store the digital video data on a storage device. Due to the limited bandwidth capacity of the communication network and the limited storage resources of the storage device, video encoding may be used to compress the video data according to one or more video encoding standards before transmitting or storing the video data.

[0004] A variety of video codec standards have been developed. For example, video coding standards include AOMedia Video 1 (AV1), Versatile Video Coding (VVC), Joint Exploration test Model (JEM), High-Efficiency Video Coding (HEVC / H.265), Advanced Video Coding (AVC / H.264), and Moving Picture Expert Group (MPEG) coding. Video coding typically utilizes a prediction method (e.g., inter-frame prediction, intra-frame prediction, etc.) that exploits the redundancy inherent in video data. Video coding aims to compress video data into a form that uses a lower bit rate while avoiding or minimizing the degradation of video quality.

[0005] HEVC, also known as H.265, is a video compression standard designed as part of the MPEG-H project. ITU-T and ISO / IEC published the HEVC / H.265 standard in 2013 (version 1), 2014 (version 2), 2015 (version 3), and 2016 (version 4). Versatile Video Coding (VVC), also known as H.266, is a video compression standard designed as the successor to HEVC. ITU-T and ISO / IEC published the VVC / H.266 standard in 2020 (version 1) and 2022 (version 2). AV1 is an open video coding format designed as a replacement for HEVC. On January 8, 2019, Verified Version 1.0.0 with Errata 1 of the specification was released. Summary of the invention

[0006] As described in more detail below, smooth prediction modes in current systems (e.g., SMOOTH-B, SMOOTH-H, and SMOOTH-V in AV1) do not take into account the uneven distribution of available reference samples (e.g., only top reference samples and left reference samples are available). The systems and methods described in the present application improve prediction accuracy by biasing (weighting) the prediction using top reference samples and left reference samples. In addition, the prediction derivation described in the present application can eliminate intermediate division operations (e.g., division is only performed when deriving the final prediction value). Moving the division operation to the back end of the process can improve the accuracy of the calculation process and simplify the hardware. In addition, as described below, the prediction derivation can be combined with a right shift operation, which can further improve the accuracy of the calculation.

[0007] According to some embodiments, a video decoding method is provided. The method includes: (i) receiving video data including a plurality of blocks from a video code stream, the plurality of blocks including a first block, wherein the first block is encoded using one of a plurality of smoothing modes (e.g., SMOOTH-B, SMOOTH-H, and SMOOTH-V modes); (ii) identifying a reference sample set of the first block; (iii) deriving a first prediction value of the first block, wherein the first prediction value is derived via a linear interpolation function, the linear interpolation function involving a first reference sample in the reference sample set and a height and a width of the first block; and (iv) decoding the first block based on the first prediction value.

[0008] According to some embodiments, a video encoding method is provided. The method comprises: (i) receiving video data comprising a plurality of blocks, the plurality of blocks comprising a first block, wherein the first block is to be encoded using one of a plurality of smoothing modes; (ii) identifying a reference sample set of the first block; (iii) deriving a first prediction value of the first block, wherein the first prediction value is derived via a linear interpolation function, the linear interpolation function involving a first reference sample in the reference sample set and a height and a width of the first block; and (iv) encoding the first block based on the first prediction value.

[0009] According to some embodiments, a computing system is provided, such as a streaming system, a server system, a personal computer system, or other electronic device. The computing system includes a control circuit and a memory, and the memory stores one or more instruction sets. The one or more instruction sets include instructions for performing any method described in the present application. In some embodiments, the computing system includes an encoder component and a decoder component (e.g., a transcoder component).

[0010] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores one or more sets of instructions executed by a computing system. The one or more sets of instructions include instructions for executing any method described in the present application.

[0011] Therefore, methods, devices and systems for encoding and decoding video are disclosed. Such methods, devices and systems can supplement or replace traditional methods, devices and systems for video encoding / decoding.

[0012] The features and advantages described in the specification are not necessarily all-inclusive, and in particular, some additional features and advantages will be apparent to one of ordinary skill in the art in view of the drawings, description, and claims provided in this disclosure. In addition, it should be noted that the language used in the specification is primarily selected for readability and instructional purposes and is not necessarily selected to describe or limit the subject matter described in this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to understand the present disclosure in more detail, a more specific description may be made by reference to the features of various embodiments, some of which are shown in the accompanying drawings. However, the attached drawings only illustrate the relevant features of the present disclosure and are therefore not necessarily considered to be limiting, as those skilled in the art will understand when reading the present disclosure that the description may admit other effective features.

[0014] Figure 1 is a block diagram illustrating an exemplary communication system in accordance with some embodiments.

[0015] Figure 2A is a block diagram illustrating exemplary elements of an encoder assembly according to some embodiments.

[0016] Figure 2B is a block diagram illustrating exemplary elements of a decoder component according to some embodiments.

[0017] Figure 3 is a block diagram illustrating an exemplary server system in accordance with some embodiments.

[0018] FIG. 4A to FIG. 4D An exemplary coding tree structure according to some embodiments is shown.

[0019] Figure 5A Examples of directional intra prediction mode angles are shown in accordance with some embodiments.

[0020] FIG. 5B to FIG. 5C Exemplary reference samples for a current block according to some embodiments are shown.

[0021] Figure 5D Exemplary weighting of smoothing modes according to some embodiments is shown.

[0022] Figure 5E Exemplary reference samples for a current block according to some embodiments are shown.

[0023] Fig. 6A is a flow chart illustrating an exemplary method of video encoding according to some embodiments.

[0024] Figure 6B is a flow chart illustrating an exemplary method of decoding video according to some embodiments.

[0025] As is common practice, the various features shown in the drawings are not necessarily drawn to scale, and like reference numerals may be used to denote like features throughout the specification and drawings. DETAILED DESCRIPTION

[0026] Among other things, the present disclosure describes a new smooth prediction mode that can improve prediction accuracy (compared to previous smooth modes) by biasing the prediction using top reference samples and / or left reference samples. For example, to predict samples in the current block using spatially neighboring reference samples, a horizontal prediction value P may be derived. H , then use the left reference sample and P H The weighted sum of H As another example, the vertical prediction value P is derived V , then use the top reference sample and P V The weighted sum ofV '. This can improve encoding accuracy (e.g., reduce artifacts in decoded video) compared to other smooth prediction modes. The present disclosure also describes that when calculating prediction samples for a new smooth (hybrid) mode, all division operations are applied in the last step of the derivation (e.g., division is only performed when deriving refined / combined prediction values). Moving the division operations to the end of the process can improve the accuracy of the calculations and can also simplify the hardware. Exemplary Systems and Apparatus

[0027] Figure 1 1 is a block diagram illustrating a communication system 100 according to some embodiments. The communication system 100 includes a source device 102 and a plurality of electronic devices 120 (e.g., electronic device 120-1 to electronic device 120-m) communicatively coupled to each other via one or more networks. In some embodiments, the communication system 100 is a streaming system, such as for video-enabled applications, such as video conferencing applications, digital television applications, and media storage and / or distribution applications.

[0028] Source device 102 includes a video source 104 (e.g., a camera component or a media storage) and an encoder component 106. In some embodiments, video source 104 is a digital camera (e.g., configured to create an uncompressed video sample stream). Encoder component 106 generates one or more encoded video streams from the video stream. The video stream from video source 104 can be high data volume compared to the encoded video stream 108 generated by encoder component 106. Because the encoded video stream 108 is a lower data volume (less data) than the video stream from the video source, the encoded video stream 108 requires less bandwidth for transmission and requires less storage space for storage compared to the video stream from video source 104. In some embodiments, source device 102 does not include encoder component 106 (e.g., configured to transmit uncompressed video to network 110).

[0029] The one or more networks 110 represent any number of networks that transmit information between the source device 102, the server system (112), and / or the electronic device 120, including, for example, wired (wired) and / or wireless communication networks. The one or more networks 110 can exchange data in circuit switching and / or packet switching channels. Representative networks include telecommunication networks, local area networks, wide area networks, and / or the Internet.

[0030] One or more networks 110 include a server system 112 (e.g., a distributed / cloud computing system). In some embodiments, the server system 112 is or includes a streaming server (e.g., configured to store and / or distribute video content, such as an encoded video stream from a source device 102). The server system 112 includes a codec component 114 (e.g., configured to encode and / or decode video data). In some embodiments, the codec component 114 includes an encoder component and / or a decoder component. In various embodiments, the codec component 114 is instantiated as hardware, software, or a combination thereof. In some embodiments, the codec component 114 is configured to decode the encoded video stream 108 and re-encode the video data using different encoding standards and / or methods to generate encoded video data 116. In some embodiments, the server system 112 is configured to generate multiple video formats and / or encodings from the encoded video stream 108.

[0031] In some embodiments, the server system 112 functions as a Media-Aware Network Element (MANE). For example, the server system 112 may be configured to prune the encoded video stream 108 to tailor potentially different streams to one or more electronic devices 120. In some embodiments, the MANE is provided separately from the server system 112.

[0032] The electronic device 120-1 includes a decoder component 122 and a display 124. In some embodiments, the decoder component 122 is configured to decode the encoded video data 116 to generate an output video stream that can be presented on a display or other type of presentation device. In some embodiments, one or more electronic devices 120 do not include a display component (e.g., communicatively coupled to an external display device and / or include a media storage device). In some embodiments, the electronic device 120 is a streaming client. In some embodiments, the electronic device 120 is configured to access the server system 112 to obtain the encoded video data 116.

[0033] The source device and / or multiple electronic devices 120 are sometimes referred to as “end devices” or “user devices.” In some embodiments, the source device 102 and / or one or more electronic devices 120 are examples of server systems, personal computers, portable devices (e.g., smartphones, tablets, or laptops), wearable devices, video conferencing devices, and / or other types of electronic devices.

[0034] In an exemplary operation of the communication system 100, the source device 102 transmits the encoded video code stream 108 to the server system 112. For example, the source device 102 can encode a stream of pictures collected by the source device. The server system 112 receives the encoded video code stream 108 and can decode and / or encode the encoded video code stream 108 using a codec component 114. For example, the server system 112 can apply encoding to the video data, which is more optimal for network transmission and / or storage. The server system 112 can transmit the encoded video data 116 (e.g., one or more encoded video code streams) to one or more electronic devices 120. Each electronic device 120 can decode the encoded video data 116 to recover and optionally display the video picture.

[0035] Figure 2A 1 is a block diagram illustrating example elements of an encoder component 106 according to some embodiments. The encoder component 106 receives a source video sequence from a video source 104. In some embodiments, the encoder component includes a receiver (e.g., a transceiver) component configured to receive the source video sequence. In some embodiments, the encoder component 106 receives a video sequence from a remote video source (e.g., a video source of a component of a device different from the encoder component 106). The video source 104 may provide the source video sequence in the form of a digital video sample stream, which may be any suitable bit depth (e.g., 8, 10, or 12 bits), any color space (e.g., BT.601 Y CrCB or RGB), and any suitable sampling structure (e.g., Y CrCb 4:2:0 or Y CrCb 4:4:4). In some embodiments, the video source 104 is a storage device that stores previously captured / prepared video. In some embodiments, the video source 104 is a camera that captures local image information as a video sequence. The video data may be provided as a plurality of separate pictures that are given motion when viewed sequentially. The image itself can be constructed as a spatial pixel array, where each pixel may include one or more samples depending on the sampling structure, color space, etc. used. The relationship between pixels and samples can be easily understood by those skilled in the art. The following description focuses on samples.

[0036] The encoder component 106 is configured to encode and / or compress the pictures of the source video sequence into an encoded video sequence 216 in real time or under any other time constraints required by the application. Implementing the appropriate encoding speed is a function of the controller 204. In some embodiments, the controller 204 controls other functional units as described below and is functionally coupled to these units. The parameters set by the controller 204 may include rate control related parameters (picture skipping, quantizer, lambda value of rate distortion optimization techniques, etc.), picture size, picture group (Group Of Pictures, GOP) layout, maximum motion vector search range, etc. Other functions of the controller 204 can be easily identified by those of ordinary skill in the art because these other functions involve the video encoder component 106 optimized for a certain system design.

[0037] In some embodiments, the encoder component 106 is configured to operate in an encoding loop. In a simplified example, the encoding loop may include a source encoder 202 (e.g., responsible for creating symbols, such as a symbol stream, based on an input picture to be encoded and a reference picture) and a (local) decoder 210. The decoder 210 reconstructs the symbols to create sample data in a manner similar to how the (remote) decoder creates sample data (when any compression between the symbols and the encoded video code stream is lossless). The reconstructed sample stream (sample data) is input to a reference picture memory 208. Since the decoding of the symbol stream produces bit-accurate results that are independent of the decoder location (local or remote), the contents of the reference picture memory 208 are also bit-accurate between the local encoder and the remote encoder. In this way, the reference picture samples interpreted by the prediction part of the encoder are exactly the same as the sample values ​​that the decoder will interpret when using the prediction during decoding. This principle of reference picture synchronization (and the drift that occurs when synchronization cannot be maintained, such as due to channel errors) is known to those of ordinary skill in the art.

[0038] The operation of decoder 210 may be combined with, for example, Figure 2B The remote decoder described in detail, such as decoder assembly 122, is the same. However, briefly referring to Figure 2B , when symbols are available and the entropy encoder 214 and the parser 254 are capable of losslessly encoding / decoding the symbols into an encoded video sequence, the entropy decoding portion of the decoder component 122, including the buffer memory 252 and the parser 254, may not be fully implemented in the local decoder 210.

[0039] Except parsing / entropy decoding, the decoder technology described in this application can be present in the corresponding encoder in the form of substantially the same function. For this reason, this application focuses on decoder operation. The description of encoder technology can be simplified because encoder technology is mutually inverse to the decoder technology described comprehensively. Only in some areas, a more detailed description is needed, and it is provided below.

[0040] As part of the operation, the source encoder 202 may perform motion compensated predictive coding that predictively encodes an input frame with reference to one or more previously encoded frames from a video sequence designated as "reference frames." In this manner, the encoding engine 212 encodes the differences between pixel blocks of an input frame and pixel blocks of a reference frame that may be selected as a prediction reference for the input frame. The controller 204 may manage the encoding operations of the source encoder 202, including, for example, the setting of parameters and subgroup parameters for encoding video data.

[0041] The decoder 210 may decode the encoded video data of the frame that may be designated as the reference frame based on the symbol created by the source encoder 202. The operation of the encoding engine 212 may advantageously be a lossy process. Figure 2A When decoded at a remote video decoder (not shown), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. Decoder 210 replicates the decoding process that may be performed by the video decoder on the reference frame and may cause the reconstructed reference frame to be stored in reference picture cache 208. In this way, encoder component 106 may locally store a copy of the reconstructed reference frame that has common content (absent transmission errors) with the reconstructed reference frame that will be obtained by the remote video decoder.

[0042] The predictor 206 may perform a prediction search for the encoding engine 212. That is, for a new frame to be encoded, the predictor 206 may search the reference picture memory 208 for sample data (as a candidate reference pixel block) or certain metadata, such as a reference picture motion vector, block shape, etc., that may serve as a suitable prediction reference for the new picture. The predictor 206 may operate pixel-by-pixel based on sample blocks to find a suitable prediction reference. In some cases, based on the search results obtained by the predictor 206, it may be determined that the input picture may have a prediction reference taken from a plurality of reference pictures stored in the reference picture memory 208.

[0043] The outputs of all the above functional units may be entropy encoded in the entropy encoder 214. The entropy encoder 214 performs lossless compression on the symbols generated by the various functional units according to techniques known to those skilled in the art (e.g., Huffman coding, variable length coding, arithmetic coding), thereby converting the symbols into a coded video sequence.

[0044] In some embodiments, the output of the entropy encoder 214 is coupled to a transmitter. The transmitter can be configured to buffer the encoded video sequence created by the entropy encoder 214, thereby preparing for transmission through a communication channel 218, which can be a hardware / software link to a storage device that will store the encoded video data. The transmitter can be configured to merge the encoded video data from the source encoder 202 with other data to be transmitted, such as encoded audio data and / or auxiliary data streams (source not shown). In some embodiments, the transmitter can transmit additional data when transmitting the encoded video. The source encoder 202 can use such data as part of the encoded video sequence. Additional data may include redundant data in other forms such as time / space / SNR enhancement layers, redundant pictures and slices, auxiliary enhancement information (Supplementary Enhancement Information, SEI) messages, visual usability information (Visual Usability Information, VUI) parameter set fragments, etc.

[0045] The controller 204 can manage the operation of the encoder component 106. During encoding, the controller 204 can assign a certain encoded picture type to each encoded picture, but this may affect the encoding technology that can be applied to the corresponding picture. For example, a picture can generally be assigned as an intra picture (I picture), a predictive picture (P picture), or a bidirectional predictive picture (B picture). An intra picture (I picture) can be encoded and decoded without using any other picture in the sequence as a prediction source. Some video codecs allow different types of intra pictures, including, for example, independent decoder refresh (Independent DecoderRefresh, "IDR") pictures. Those skilled in the art understand the variants of I pictures and their corresponding applications and features, so this application will not repeat them. Predictive pictures P pictures can be encoded and decoded using intra prediction or inter prediction, which uses at most one motion vector and reference index to predict the sample values ​​of each block. Bidirectional predictive pictures (B pictures) can be encoded and decoded using intra prediction or inter prediction, which uses at most two motion vectors and reference indexes to predict the sample values ​​of each block. Similarly, multiple predictive pictures may use more than two reference pictures and associated metadata for reconstructing a single block.

[0046] The source picture may typically be spatially subdivided into blocks of samples (e.g., blocks of 4×4, 8×8, 4×8, or 16×16 samples), and coded block by block. These blocks may be predictively coded with reference to other (already coded) blocks, which are determined according to the coding allocation applied to the block's corresponding picture. For example, a block of an I picture may be non-predictively coded, or the block may be predictively coded (spatial prediction or intra-frame prediction) with reference to an already coded block of the same picture. A block of a P picture may be non-predictively coded by spatial prediction with reference to one previously coded reference picture or by temporal prediction. A block of a B picture may be non-predictively coded by spatial prediction with reference to one or two previously coded reference pictures or by temporal prediction.

[0047] The captured video may be taken as a plurality of source pictures (video pictures) in a temporal sequence. Intra-picture prediction (often simplified to intra-prediction) exploits spatial correlations in a given picture, while inter-picture prediction exploits (temporal or other) correlations between pictures. In an embodiment, a particular picture being encoded / decoded is divided into blocks, and the particular picture being encoded / decoded is referred to as the current picture. When a block in the current picture is similar to a reference block in a reference picture that was previously encoded in the video and is still buffered, the block in the current picture may be encoded by a vector called a motion vector. The motion vector points to a reference block in a reference picture, and in the case where multiple reference pictures are used, the motion vector may have a third dimension that identifies the reference picture.

[0048] The encoder component 106 can perform encoding operations according to a predetermined video encoding technique or standard, such as any of the techniques or standards described herein. In operation, the encoder component 106 can perform various compression operations, including predictive encoding operations that exploit temporal and spatial redundancy in an input video sequence. Thus, the encoded video data can conform to the syntax specified by the video encoding technique or standard used.

[0049] Figure 2B is a block diagram illustrating exemplary elements of decoder component 122 according to some embodiments. Figure 2B The decoder component 122 in is coupled to the channel 218 and the display 124. In some embodiments, the decoder component 122 includes a transmitter coupled to the loop filter 256 and configured to transmit data to the display 124 (eg, via a wired or wireless connection).

[0050] In some embodiments, the decoder component 122 includes a receiver coupled to the channel 218 and configured to receive data from the channel 218 (e.g., via a wired or wireless connection). The receiver can be configured to receive one or more encoded video sequences for decoding by the decoder component 122. In some embodiments, the decoding of each encoded video sequence is independent of other encoded video sequences. Each encoded video sequence can be received from the channel 218, which can be a hardware / software link to a storage device storing the encoded video data. The receiver can receive the encoded video data as well as other data, such as encoded audio data and / or auxiliary data streams that can be forwarded to their respective use entities (not shown). The receiver can separate the encoded video sequence from the other data. In some embodiments, the receiver can receive additional (redundant) data along with the encoded video. The additional data is included as part of the encoded video sequence. The additional data can be used by the video decoder 122 to decode the data and / or more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0051] According to some embodiments, the decoder component 122 includes: a buffer memory 252, a parser 254 (sometimes also referred to as an entropy decoder), a scaler / inverse transform unit 258, an intra picture prediction unit 262, a motion compensated prediction unit 260, an aggregator 268, a loop filter unit 256, a reference picture memory 266, and a current picture memory 264. In some embodiments, the decoder component 122 is implemented as an integrated circuit, a series of integrated circuits, and / or other electronic circuits. In some embodiments, the decoder component 122 is at least partially implemented in software.

[0052] Buffer memory 252 is coupled between channel 218 and parser 254 (e.g., to prevent network jitter). In some embodiments, buffer memory 252 is separate from decoder component 122. In some embodiments, a separate buffer memory is provided between the output of channel 218 and decoder component 122. In some embodiments, in addition to buffer memory 252 internal to decoder component 122 (e.g., which is configured to handle playout timing), a separate buffer memory is provided external to decoder component 122 (e.g., to prevent network jitter). Buffer memory 252 may not be required or may be made smaller when data is received from a store / forward device with sufficient bandwidth and controllability or from an isochronous network. Buffer memory 252 may be necessary for use on a traffic packet network such as the Internet, and may be relatively large and may have an adaptive size and may be implemented at least in part in an operating system or similar element (not shown) external to decoder component 122.

[0053] The parser 254 is configured to reconstruct symbols 270 from the encoded video sequence. The symbols may include, for example, information for managing the operation of the decoder component 122, and / or information for controlling a display device such as the display 124. The control information for the display device may be in the form of, for example, auxiliary enhancement information (SEI message) or video usability information (VUI) parameter set fragments (not shown). The parser 254 parses (entropy decodes) the encoded video sequence. The encoding of the encoded video sequence may be performed according to a video coding technique or standard, and may follow principles well known to those skilled in the art, including variable length coding, Huffman coding, arithmetic coding with or without context sensitivity, etc. The parser 254 may extract a subgroup parameter set for at least one of the subgroups of pixels in the video decoder from the encoded video sequence based on at least one parameter corresponding to the group. Subgroups may include group of pictures (GOP), pictures, tiles, slices, macroblocks, coding units (CU), blocks, transform units (TU), prediction units (PU), etc. The parser 254 may also extract information such as transform coefficients, quantizer parameter values, motion vectors, etc. from the encoded video sequence.

[0054] Depending on the type of the coded video picture or a portion of the coded video picture (e.g., inter-frame and intra-frame pictures, inter-frame blocks and intra-frame blocks) and other factors, the reconstruction of the symbol 270 may involve multiple different units. Which units are involved and how they are involved may be controlled by subgroup control information parsed from the coded video sequence by the parser 254. For the sake of brevity, such subgroup control information flow between the parser 254 and the multiple units is not described.

[0055] The decoder component 122 may be conceptually subdivided into several functional units. And in some embodiments, these units closely interact with each other and may be at least partially integrated with each other. However, for the sake of clarity, this conceptual subdivision into functional units is maintained.

[0056] The sealer / inverse transform unit 258 receives the quantized transform coefficients and control information (e.g., which transform method to use, block size, quantization factor, and / or quantization scaling matrix) as symbols 270 from the parser 254. The sealer / inverse transform unit 258 may output a block including sample values, which may be input into the aggregator 268.

[0057] In some cases, the output samples of the sealer / inverse transform unit 258 belong to intra-coded blocks; that is, blocks that do not use predictive information from previously reconstructed pictures, but can use predictive information from previously reconstructed portions of the current picture. Such predictive information may be provided by the intra-picture prediction unit 262. The intra-picture prediction unit 262 uses the reconstructed information obtained from the current picture memory 264 to generate surrounding blocks of the same size and shape as the block being reconstructed. The aggregator 268 adds the prediction information generated by the intra-prediction unit 262 to the output sample information provided by the sealer / inverse transform unit 258 on a per-sample basis.

[0058] In other cases, the output samples of the sealer / inverse transform unit 258 may belong to an inter-frame coded block and a potential motion compensated block. In this case, the motion compensated prediction unit 260 may access the reference picture memory 266 to extract samples for prediction. After the extracted samples are motion compensated according to the symbol 270 associated with the block, these samples may be added to the output of the sealer / inverse transform unit 268 (in this case, referred to as residual samples or residual signals) by the aggregator 268 to generate output sample information. The extraction of prediction samples by the motion compensated prediction unit 260 from the address in the reference picture memory 266 may be controlled by a motion vector, and the motion vector is provided to the motion compensated prediction unit 260 in the form of the symbol 270, which includes, for example, X, Y and reference picture components. Motion compensation may also include interpolation of sample values ​​extracted from the reference picture memory 266 when using sub-sample accurate motion vectors, motion vector prediction mechanisms, and the like.

[0059] The output samples of aggregator 268 may be employed by various loop filtering techniques in loop filter unit 256. The video compression techniques may include in-loop filter techniques controlled by parameters included in the encoded video bitstream and available to loop filter unit 256 as symbols 270 from parser 254. However, the video compression techniques may also be responsive to meta-information obtained during decoding of a previous (in decoding order) portion of an encoded picture or encoded video sequence, as well as to previously reconstructed and loop filtered sample values.

[0060] The output of the loop filter unit 256 may be a sample stream that may be output to a display device such as the display 124 , as well as stored in the reference picture memory 266 for subsequent inter-picture prediction.

[0061] Once fully reconstructed, certain coded pictures may be used as reference pictures for future prediction. Once a coded picture is fully reconstructed, and the coded picture is identified as a reference picture (e.g., by parser 254), the current reference picture may become part of reference picture memory 457, and a new current picture memory may be reallocated before starting reconstruction of a subsequent coded picture.

[0062] The decoder component 122 may perform decoding operations according to a predetermined video compression technique. The technique may be recorded in a standard, such as any standard described in this application. In the sense that the encoded video sequence follows the syntax of the video compression technique or standard and the profile recorded in the video compression technique or standard, the encoded video sequence may conform to the syntax specified by the video compression technique or standard used. In addition, in order to conform to some video compression techniques or standards, the complexity of the encoded video sequence is within the range defined by the hierarchy of the video compression technique or standard. In some cases, the hierarchy limits the maximum picture size, the maximum frame rate, the maximum reconstruction sampling rate (measured in, for example, mega samples per second), the maximum reference picture size, etc. In some cases, the limits set by the hierarchy may be further defined by the Hypothetical Reference Decoder (HRD) specification and the metadata of the HRD buffer management represented by the signal in the encoded video sequence.

[0063] Figure 3306, and a plurality of communication buses 312 for interconnecting these components. In some embodiments, the control circuit 302 includes one or more processors (e.g., a central processing unit (CPU), a graphics processing unit (GPU), and / or a data processing unit (DPU)). In some embodiments, the control circuit includes one or more field programmable gate arrays (FPGA), hardware accelerators, and / or one or more integrated circuits (e.g., application specific integrated circuits).

[0064] The network interface 304 can be configured to interface with one or more communication networks (e.g., wireless networks, wired networks, and / or optical networks). The communication network can be a local network, a wide area network, a metropolitan area network, a vehicle and industrial network, a real-time network, a delay-tolerant network, and the like. Examples of communication networks include local area networks such as Ethernet, wireless LANs, cellular networks including Global System for Mobile communications (GSM), 3G, 4G, 5G, Long-Term Evolution (LTE), etc., television wired or wireless wide area digital networks including cable television, satellite television, and terrestrial broadcast television, vehicle and industrial networks including CANBus, and the like. Such communications can be one-way reception only (e.g., broadcast television), one-way transmission only (e.g., CANbus to certain CANbus devices), or bidirectional, for example, using a local area network or a wide area network digital network to connect to other computer systems. Such communications can include communications to one or more cloud computing networks.

[0065] The user interface 306 includes one or more output devices 308 and / or one or more input devices 310. The input device 310 may include one or more of the following: a keyboard, a mouse, a touch pad, a touch screen, a data glove, a joystick, a microphone, a scanner, a camera. The output device 308 may include one or more of the following: an audio output device (e.g., a speaker), a visual output device (e.g., a display or monitor), etc.

[0066] The memory 314 may include high-speed random access memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM, double data rate (DDR) RAM and / or other random access solid-state memory devices) and / or non-volatile memory (e.g., one or more magnetic disk storage devices, optical disk storage devices, flash memory devices and / or other non-volatile solid-state storage devices). The memory 314 may optionally include one or more storage devices remote from the control circuit 302. The memory 314, or alternatively, the non-volatile solid-state memory device within the memory 314, includes a non-transitory computer-readable storage medium. In some embodiments, the memory 314 or the non-transitory computer-readable storage medium of the memory 314 stores the following programs, modules, instructions and data structures, or a subset or superset thereof: Operating system 316, which includes procedures for handling various basic system services and for performing hardware-related tasks; A network communication module 318 for connecting the server system 112 to other computing devices via one or more network interfaces 304 (e.g., via wired and / or wireless connections); A codec module 320 for performing various functions related to encoding and / or decoding data (eg, video data). In some embodiments, the codec module 320 is an example of the codec component 114. The codec module 320 includes but is not limited to: o a decoding module 322 for performing various functions related to decoding encoded data, such as those previously described with respect to the decoder component 122; and o an encoding module 340 for performing various functions related to encoding data, such as those previously described with respect to the encoder component 106; and Picture memory 352, for storing pictures and picture data, for example, for use with the codec module 320. In some embodiments, the picture memory 352 includes one or more of the following: reference picture memory 208, buffer memory 252, current picture memory 264, and reference picture memory 266.

[0067] In some embodiments, the decoding module 322 includes a parsing module 324 (e.g., configured to perform the various functions previously described with respect to the parser 254), a transform module 326 (e.g., configured to perform the various functions previously described with respect to the scaler / inverse transform unit 258), a prediction module 328 (e.g., configured to perform the various functions previously described with respect to the motion compensated prediction unit 260 and / or the intra-picture prediction unit 262), and a filter module 330 (e.g., configured to perform the various functions previously described with respect to the loop filter 256).

[0068] In some embodiments, the encoding module 340 includes a sub-encoding module 342 (e.g., configured to perform various functions previously described with respect to the source encoder 202 and / or the encoding engine 212) and a prediction module 344 (e.g., configured to perform various functions previously described with respect to the predictor 206). In some embodiments, the decoding module 322 and / or the encoding module 340 includes Figure 3 A subset of the modules shown. For example, both the decoding module 322 and the encoding module 340 use a shared prediction module.

[0069] Each of the identified modules stored in the memory 314 corresponds to an instruction set for performing the functions described in the present application. The identified modules (e.g., instruction sets) do not need to be implemented as separate software programs, processes, or modules, so in various embodiments, various subsets of these modules can be combined or otherwise rearranged. For example, the codec module 320 optionally does not include a separate decoding module and encoding module, but uses the same set of modules to perform two sets of functions. In some embodiments, the memory 314 stores subsets and data structures of the identified modules. In some embodiments, the memory 314 stores additional modules and data structures not described above, such as audio processing modules.

[0070] although Figure 3 The server system 112 is shown according to some embodiments, but Figure 3 The present invention is intended more as a functional description of various features that may be present in one or more server systems rather than as a block diagram of the embodiments described herein. In practice, as will be appreciated by one of ordinary skill in the art, items shown separately may be combined and some items may be separated. For example, Figure 3Some of the items shown individually in the figure may be implemented on separate servers, and a single item may be implemented by one or more servers. The actual number of servers used to implement server system 112 and the manner in which features are distributed among them will vary from implementation to implementation, and may depend in part on the amount of data traffic that the server system handles during peak usage and during average usage. Exemplary Encoding and Decoding Processes and Techniques

[0071] The encoding processes and techniques described below may be performed on the devices and systems described above (eg, source device 102, server system 112, and / or electronic device 120). FIG. 4A to FIG. 4D An exemplary coding tree structure according to some embodiments is shown. Figure 4A As shown in the first coding tree structure (400) in FIG, some coding methods (e.g., VP9) use a 4-way partition tree from the 64×64 level down to the 4×4 level, with some additional restrictions for 8×8 blocks. Figure 4A The partitioning designated as R in can be called recursive because the same partitioning tree is repeated at a lower scale until the lowest 4×4 level is reached.

[0072] like Figure 4B As shown in the second coding tree structure (402) in FIG. 4 , some coding methods (e.g., AV1) extend the partition tree to a 10-way structure and increase the maximum size (e.g., superblock in VP9 / AV1 parlance) to start at 128×128. The second coding tree structure includes 4:1 / 1:4 rectangular partitions that are not present in the first coding tree structure. Figure 4B The partition type with 3 sub-partitions in the second row of is called T-type partition. In addition to the coding block size, a coding tree depth can be defined to indicate the partition depth starting from the root node.

[0073] For example, a Coding Tree Unit (CTU) can be divided into Coding Units (CU) using a quadtree structure represented as a coding tree to adapt to various local characteristics in, for example, HEVC. In some embodiments, the decision on whether to encode a picture area using inter (temporal) prediction or intra (spatial) prediction is made at the CU level. Depending on the partition type of the PU, each CU can be further split into one PU, two PUs, or four PUs. Within a PU, the same prediction process is applied, and the relevant information is transmitted to the decoder PU by PU. After obtaining the residual block by applying the prediction process based on the PU partition type, the CU can be divided into TUs according to another quadtree structure similar to the coding tree for the CU.

[0074] For example, in VVC, a quadtree with nested multi-type trees using a binary and ternarysplits segmentation structure can replace the concept of multiple partition unit types. For example, this type of quadtree removes the concept of distinguishing between CU, PU, ​​and TU, unless the size of the CU is too large for the maximum transform length and the distinction is required, and this type of quadtree supports greater flexibility in the shape of CU partitions. In the coding tree structure, the CU can have a square or rectangular shape. The CTU is first divided by a quadtree (also called a quadtree) structure. The quadtree leaf nodes can be further divided by a multi-type tree structure. Figure 4C As shown in the third coding tree structure (404) in FIG, the multi-type tree structure includes four types of partitioning. The multi-type tree leaf node is called a CU, and this partitioning is used for prediction and transform processing without any further division unless the CU is too large for the maximum transform length. This means that in most cases, in a quadtree with a nested multi-type tree coding block structure, CU, PU, ​​and TU have the same block size. Figure 4D An example of block partitioning for one CTU (406) is shown, showing an example quadtree with a nested multi-type tree coding block structure.

[0075] For directional intra prediction, some methods support 8 directional modes corresponding to angles from 45 degrees to 207 degrees. In order to exploit a wider variety of spatial redundancies in directional texture, directional intra modes can be extended to a set of angles with finer granularity. For example, 8 angles can be represented as nominal angles. Figure 5A 8 nominal angles are shown in FIG. 1 , which are named V_PRED, H_PRED, D45_PRED, D135_PRED, D113_PRED, D157_PRED, D203_PRED and D67_PRED. Figure 5A As shown, for each nominal angle, there can be 7 finer angles, for a total of 56 directional angles. The predicted angle can be described by the nominal intra angle plus an angle increment (e.g., -3 to 3 times a step size of 3 degrees). In order to implement directional prediction modes in a general way, the 56 directional intra-frame prediction modes can be implemented with a unified directional predictor that projects each pixel to a reference sub-pixel position and interpolates the reference pixel through a 2-tap bilinear filter. This directional intra-frame prediction is sometimes also called unidirectional intra-frame prediction.

[0076] In some methods, a lookup table is used to map each intra prediction angle to a horizontal offset and a vertical offset between each pixel in the current block and a reference sample. The offset in the lookup table can be an integer value of the tangent value of the angle of each intra prediction angle multiplied by 64. For example, 64 is equal to the tangent (45°) value of 45 degrees multiplied by 64, where the tangent value of 45 degrees is equal to 1. For example, for a 45-degree intra prediction angle, the associated offset is 64, and the horizontal offset between each pixel in the current block and the reference pixel increases by 1 pixel as the row number of the pixel increases by 1.

[0077] In some methods, there are also non-directional smooth intra prediction modes, such as DC, PAETH, SMOOTH-B (sometimes expressed as SMOOTH (smooth)), SMOOTH-V, and SMOOTH-H. For DC prediction, the average of the left neighboring samples and the upper neighboring samples can be used as the prediction value of the block to be predicted. For PAETH prediction, the top reference sample, the left reference sample, and the upper left reference sample can be taken, and the value closest to the given value (e.g., top + left - upper left) is set as the prediction value of the pixel to be predicted. Figure 5B The positions of the top sample 514, the left sample 518, and the upper left sample 516 of the current pixel (or partition or sub-block) in the current block are shown. For SMOOTH-B, SMOOTH-V, and SMOOTH-H modes, blocks can be predicted using quadratic interpolation in the vertical or horizontal direction or an average in both directions.

[0078] In order to capture the attenuated spatial correlation with the reference on the edge, filter intra modes can be used for luma blocks. In some methods, five filter intra modes are defined, each represented by a set of eight 7-tap filters that reflect the correlation between a pixel in a 4×2 block (subblock) and its 7 adjacent neighbors. In this way, the weight factors of the 7-tap filters are position-dependent. For example, Figure 5C As shown in , the 8×8 block can be divided into 8 4×2 blocks. Figure 5C In , these blocks are represented by B0, B1, B2, B3, B4, B5, B6 and B7. For each block, its 7 neighbors (represented by R0-R7) are used to predict the pixels in the current block. For block B0, all neighbors have been reconstructed. But for other blocks, some neighbors may not be reconstructed. In this case, the predicted values ​​of the immediate neighbors can be used as a reference. For example, instead of reconstructing all the neighbors of block B7, the predicted samples of the neighbors (for example, B5 and B6) can be used.

[0079] In some methods, the SMOOTH-V mode and the SMOOTH-H mode generate prediction values ​​by using interpolation in the vertical direction and interpolation in the horizontal direction, respectively, while the SMOOTH-B mode generates prediction values ​​using the average of the interpolation results in both directions. The top adjacent, left adjacent, upper right adjacent, and lower left adjacent reconstructed sample values ​​may be represented as T, L, TR, and BL, respectively. In the three modes, the prediction sample values ​​may be calculated as follows, P H (x,y)=w(x)×L+(1-w(x))×TR Equation 1 - SMOOTH-H Mode P V (x,y)=w(y)×T+(1-w(y))×BL Equation 2 - SMOOTH-V Mode P(x,y)=(PH(x,y)+PV(x,y)) / 2 Equation 3 - SMOOTH-B Mode Where x represents the horizontal coordinate of the prediction sample in the prediction block, y represents the vertical coordinate of the prediction sample in the prediction block, w(x) and w(y) are weight values ​​derived based on the coordinate values, and P H , P V and P represent the prediction block of SMOOTH-H mode, the prediction block of SMOOTH-V mode and the prediction block of SMOOTH-B mode respectively. Figure 5D The prediction process of the SMOOTH-V mode and the prediction process of the SMOOTH-H mode are shown, wherein w(y) and w(x) are nonlinear weight values ​​derived based on the vertical coordinates and horizontal coordinates in the SMOOTH-V mode and the SMOOTH-H mode, respectively. The weights may be stored in a lookup table.

[0080] In some systems, smooth prediction modes (e.g., the SMOOTH-B, SMOOTH-H, and SMOOTH-V modes described above) do not take into account the uneven distribution of available reference samples (e.g., only top reference samples and left reference samples are available) or the nonlinear weight distribution of the available reference samples, and therefore, the prediction accuracy of these smooth prediction modes may be reduced compared to modes that give greater weight to top reference samples and left reference samples. The systems and methods described below can improve prediction accuracy and precision compared to the smooth prediction modes shown in Equations 1 to 3 above. The methods and processes described below can be used alone or in combination in any order. In the description below, if a mode generates prediction samples according to a given prediction direction, the mode is referred to as a directional mode.

[0081] As used in this application, Figure 5EAs shown in the left part 528 in FIG. 1 , the left reference sample refers to a reference sample whose vertical coordinate value is within the minimum vertical coordinate value and the maximum vertical coordinate value of the current block (eg, the current block 522). Figure 5E As shown in the top portion 524 of FIG. 1 , the top reference sample refers to a reference sample whose horizontal coordinate value is within the minimum horizontal coordinate value and the maximum horizontal coordinate value of the current block. Figure 5E As shown in the lower left portion 530 of FIG. 1 , the lower left reference sample refers to a reference sample whose vertical coordinate value is greater than the maximum vertical coordinate value of the current block. Figure 5E As shown in the upper right portion 526 of FIG. 1 , the upper right reference sample refers to a reference sample whose horizontal coordinate value is greater than the maximum horizontal coordinate value of the current block. Figure 5E As shown in the upper left part 532 in FIG. 1 , the upper left reference sample refers to a reference sample whose horizontal coordinate value is smaller than the minimum horizontal coordinate value of the current block and whose vertical coordinate value is smaller than the minimum vertical coordinate value of the current block.

[0082] Fig. 6A 6 is a flow chart illustrating a video encoding method 600 according to some embodiments. The method 600 may be performed on a computing system (e.g., server system 112, source device 102, or electronic device 120) having control circuitry and a memory storing instructions executed by the control circuitry. In some embodiments, the method 600 is performed by executing instructions stored in a memory (e.g., memory 314) of the computing system.

[0083] A system receives (602) video data including a plurality of blocks, the plurality of blocks including a first block, wherein the first block is to be encoded using one of a plurality of smoothing modes (e.g., SMOOTH-B, SMOOTH-V, or SMOOTH-H). The system identifies (604) a set of reference samples for the first block. The system derives (606) a first prediction value (e.g., a combined prediction value) for the first block, wherein (i) the first prediction value is derived from a second prediction value (e.g., a horizontal prediction value or a vertical prediction value), (ii) the second prediction value is derived without using a division operation or a shift operation, and (iii) a final step of deriving the first prediction value includes a division operation or a shift operation (e.g., using Equation 13, Equation 18, or Equation 19 described below). The system encodes (608) the first block based on the first prediction value. For example, the system reconstructs the first block based on the first prediction value, evaluates a smoothing mode based on the reconstructed first block, and selects a smoothing mode for encoding the first block (e.g., a smoothing mode with a minimum error) based on the evaluation.

[0084] Figure 6B6 is a flow chart illustrating a method 650 of decoding video according to some embodiments. The method 650 may be performed on a computing system (e.g., server system 112, source device 102, or electronic device 120) having control circuitry and a memory storing instructions executed by the control circuitry. In some embodiments, the method 650 is performed by executing instructions stored in a memory (e.g., memory 314) of the computing system.

[0085] The system receives (652) video data including a plurality of blocks from a video code stream, the plurality of blocks including a first block, wherein the first block is encoded using one of a plurality of smoothing modes (e.g., SMOOTH-B, SMOOTH-V, and SMOOTH-H modes). The system identifies (654) a reference sample set for the first block. The system derives (656) a first prediction value (e.g., a combined prediction value) for the first block, wherein (i) the first prediction value is derived from a second prediction value (e.g., a horizontal prediction value or a vertical prediction value), (ii) the second prediction value is derived without using a division operation or a shift operation, and (iii) the last step of deriving the first prediction value includes a division operation or a shift operation (e.g., using Equation 13, Equation 18, or Equation 19 described below). The system decodes (658) the first block based on the first prediction value.

[0086] Although Fig. 6A and 6B The multiple logical stages are shown in a specific order, but the stages that do not depend on the order can be reordered, and other stages can be combined or decomposed. For those of ordinary skill in the art, some reordering or other groupings not specifically mentioned will be conceivable, so the ordering and grouping given in this application are not exhaustive. In addition, it should be recognized that these stages can be implemented in hardware, firmware, software, or any combination thereof.

[0087] In some embodiments, in order to predict samples in the current block using spatially neighboring reference samples, a horizontal prediction value P is first derived. H , and then derive the refined horizontal prediction value P′ using the weighted sum of the left reference sample (e.g., the reference sample with the same vertical coordinate) and PH H In some embodiments, the refined horizontal prediction value P′ H For current sample prediction (eg, corresponding to SMOOTH-H mode). In some embodiments, the vertical prediction value P is derived. V , and then use the top reference sample (e.g., the reference sample with the same horizontal coordinate) to compare with P V The refined horizontal prediction value P′V is derived by taking the weighted sum ofV For current sample prediction (eg, corresponding to SMOOTH-V mode). In some embodiments, the combined (final) prediction value is derived as P′ H and P′ V In some embodiments, the combined prediction value P is used for current sample prediction (eg, corresponding to SMOOTH-B mode).

[0088] In some embodiments, the horizontal prediction value P is derived using a weighted sum of the left reference sample and one or more top (and / or upper right) reference samples. H For example, the weighted sum may be based on bilinear interpolation filtering. In some embodiments, the vertical prediction value P is derived using a weighted sum of the top reference sample and one or more left (and / or bottom left) reference samples. V For example, the weighted sum may be based on bilinear interpolation filtering. In some embodiments, when generating P H Value and / or P V Before generating the value, the left reference sample and the upper reference sample are filtered. H Value and / or P V Before the values ​​are calculated, a smoothing filter (such as a Gaussian filter or a bilateral filter) may be used to filter the left reference samples and the upper reference samples, wherein the coefficients of the smoothing filter are non-negative integers.

[0089] In some embodiments, the refined horizontal prediction value P′ is derived using the following equation: H . P′ H =(w L ·L+(Nw L )·P H +r) / N Equation 4 - Refined Horizontal Predictions Among them, L is the left sample or the lower left sample, w L is derived using the horizontal coordinate value of the current sample, N is a predefined value (e.g., an integer that is a power of 2, such as 2, 4, 8, 16, 32, 64, 128, 256, 512, or 1024), and r is a rounding offset (e.g., equal to 0.5·N). In some embodiments, wL is derived as K>>((x<<1)>>s), where K is a predefined value (e.g., 16, 32, or 64), x is the horizontal coordinate of the current sample to be predicted, s is a scaling factor based on the block size (e.g., s is equal to (log2(W)+log2(H)+2)>>2), and W and H are the width and height of the block, respectively.

[0090] In some embodiments, the refined vertical prediction value P is derived using the following equation 5: V '. P′ V =(w T ·T+(Nw T )·P V +r) / N Equation 5 - Refined vertical prediction value Where T is the top sample or the upper right sample, w T is derived using the vertical coordinate value of the current sample. T It is derived as K>>((y<<1)>>s), where y is the vertical coordinate of the current sample to be predicted.

[0091] In some embodiments, the combined (final) prediction value P is derived as P H ' and P V In some embodiments, P is derived as P′ H and P′ V where the weights depend on the width and height of the block.

[0092] In some embodiments, the new modes discussed above are applied together with the SMOOTH, SMOOTH-V, and SMOOTH-H modes of AV1 (e.g., corresponding to Equations 1 to 3). In some embodiments, the new modes discussed above replace the SMOOTH, SMOOTH-V, and SMOOTH-H modes of AV1. In some embodiments, the new modes discussed above are signaled as additional modes. For example, the mode can be a new mode in the intra-mode list. In another example, the mode can be signaled in the SMOOTH-B mode, the SMOOTH-V mode, and the SMOOTH-H mode. For example, a smoothing mode index can be signaled to indicate whether a new mode or a pre-existing mode is used.

[0093] Note that performing division (and / or right shift) in an intermediate step may reduce precision (e.g., fractional numbers are truncated due to the number of bits used to store the result of the division). H and / or P′ V Before, calculate P H and / or P V Performing a division operation may result in P′ H and / or P′ V is not very accurate. Therefore, move the division operation to the end of the process (for example, to the end of the calculation of P′ H , P′ V , or the last step of P) is advantageous. In addition, calculating intermediate values ​​without performing division can simplify the hardware logic required to implement the calculation.

[0094] In some embodiments, when calculating the predicted samples for the new SMOOTH-B mode (or hybrid mode), the division (and / or right shift) operation is moved to the last derivation step. For example, the SMOOTH-B mode is calculated as shown in equations 6 to 8 below, where L is the left sample or the bottom left sample and T is the top sample or the top right sample. P′ H =(w L ·L+(Nw L )·P H ) Equation 6 - Refined horizontal prediction value without division P′ V =(w T ·T+(Nw T )·P V ) Equation 7 - Refined vertical prediction value without division P=(P′ H +P′ V +r) / (2·N) Equation 8 - Combined Prediction Value

[0095] In some embodiments, as shown in the following equations 9 to 13, when calculating P H and P V The division (and / or right shift) operations in are also moved to the last derivation step of the combined (final) prediction value. For example, during the internal calculation, a common value may be multiplied so that each prediction value has the same divisor (or shift value). P V =(T·(H-1-y)+BL·(y+1))·W Equation 9 - Vertical Prediction P′ V =(w T ·W·H·T+(Nw T )·P V ) Equation 10 - Refined vertical prediction value without division P H =(L·(W-1-x)+TR·(x+1))·H Equation 11 - Horizontal Prediction Value P′ H =(w L ·W·H·L+(Nw L )·P H ) Equation 12 - Refined horizontal prediction value without division P=(P′ H +P′ V+r) / (2·N·W·H) Equation 13 - Combined Prediction Value

[0096] In equations 9 to 13, r is a rounding offset (e.g., equal to N·W·H). In equations 9 to 13, multiplication by W or H may be a left shift of log2(W) or log2(H), and division by (2·N·W·H) may be a right shift of (1+log2(W)+log2(H)+log2(N)). In equations 9 to 13, intermediate values ​​(e.g., P V , P′ V , P H and P′ H ) is performed without using division or right shift operations.

[0097] In some embodiments, as shown in the following equations 14 to 18, when calculating P H , P′ H , P V and / or P′ V When , a first right shift value is performed at the end of the corresponding calculation, and a second right shift operation is performed in the calculation of P (e.g., to compensate for the first right shift), where BL is the lower left reference sample and TR is the upper right reference sample. P V =((T·(H-1-y)+BL·(y+1))·W+r V1 )>>s V1 Equation 14 - Vertical prediction value with shift P′ V =((w T ·W·H·T+(Nw T )·P V )+r V2 )>>s V2 Equation 15 - Refined vertical prediction with shift P H =((L·(W-1-x)+TR·(x+1))·H+r H1 )>>s H1 Equation 16 - Horizontal prediction with shift P′ H =((w L ·W·H·L+(Nw L )·P H )+r H2 )>>s H2 Equation 17 - Refined Horizontal Prediction with Shift Equation 18 - Combined prediction value with shift

[0098] In some embodiments, (for example, if s V1 +s V2 Equal to s H1 +s H2 ) The final prediction P is P H and P V The weighted average of is derived as shown in Equation 19 below: Equation 19 - Combined Prediction Value Where r is the rounding offset, an example value of which is

[0099] (A1) On the one hand, some embodiments include a method of video encoding (e.g., method 600). In some embodiments, the method is performed on a computing system (e.g., server system 112) having a memory and a control circuit. In some embodiments, the method is performed on an encoding module (e.g., codec module 320). In some embodiments, the method is performed on a source encoding component (e.g., source encoder 202), an encoding engine (e.g., encoding engine 212), and / or an entropy encoder (e.g., entropy encoder 214). The method includes: (i) receiving video data including a plurality of blocks, the plurality of blocks including a first block, wherein the first block is to be encoded using a smooth mode; (ii) identifying a reference sample set of the first block; (iii) deriving a first prediction value of the first block, wherein the first prediction value is derived via a linear interpolation function, the linear interpolation function involving a first reference sample in the reference sample set and a height and a width of the first block; and (iv) encoding the first block based on the first prediction value. In some embodiments, the linear interpolation function does not include a division operation or a shift operation. For example, the smooth mode is a SMOOTH-H mode, a SMOOTH-V mode, or a combined (SMOOTH-B) mode.

[0100] (A2) In some embodiments of A1, the method further comprises: (i) deriving a refined first prediction value for the first block; and (ii) encoding the first block based on the refined first prediction value. In some embodiments, the refined first prediction value can be derived without using a division operation or a shift operation.

[0101] (A3) In some embodiments of A2, the method further includes: (i) deriving a second prediction value for the first block, wherein the second prediction value is derived via a second linear interpolation function, the second linear interpolation function involving a second reference sample of a reference sample set and a height and a width of the first block; (ii) deriving a refined second prediction value for the first block; and (iii) deriving a combined prediction value using the refined first prediction value and the refined second prediction value, wherein the first block is encoded using the combined prediction value. In some embodiments, the second linear interpolation function does not include a division operation or a shift operation. In some embodiments, the combined prediction value P is derived using a division operation.

[0102] (A4) In some embodiments of any one of A1-A3, the first prediction value is a horizontal prediction value P H , and the first interpolation function implements Equation 11.

[0103] (A5) In some embodiments of any one of A1-A3, the first prediction value is a vertical prediction value P V , and the first interpolation function implements Equation 9.

[0104] (A6) In some embodiments of any one of A1-A5, deriving the first prediction value of the first block includes: performing a right shift operation.

[0105] (A7) In some embodiments of any one of A1-A6, the method further includes: transmitting the encoded first block via a video code stream.

[0106] (B1) On the other hand, some embodiments include a method for video decoding. In some embodiments, the method is performed on a computing system (e.g., server system 112) having a memory and a control circuit. In some embodiments, the method is performed on a decoding module (e.g., codec module 320). In some embodiments, the method is performed on a parser (e.g., parser 254), a motion prediction component (e.g., motion compensation prediction unit 260) and / or an intra-frame prediction component (e.g., intra-frame picture prediction unit 262). The method includes: (i) receiving video data including a plurality of blocks (e.g., an encoded video sequence) from a video code stream, the plurality of blocks including a first block, wherein the first block is encoded using a smooth mode; (ii) identifying a reference sample set of the first block; (iii) deriving a first prediction value of the first block, wherein the first prediction value is derived via a linear interpolation function, the linear interpolation function involving a first reference sample in the reference sample set and a height and a width of the first block; and (iv) decoding the first block based on the first prediction value.

[0107] (B2) In some embodiments of B1, the linear interpolation function does not include a division operation or a shift operation. For example, all division operations and / or shift operations are performed only in the derivation of the final prediction value.

[0108] (B3) In some embodiments of B1 or B2, the method further includes: (i) deriving a refined first prediction value of the first block; and (ii) decoding the first block based on the refined first prediction value. In some embodiments, the refined first prediction value is derived using a weighted sum of a first reference sample in a reference sample set and the first prediction value. In some embodiments, deriving the refined first prediction value includes: performing a right shift operation. For example, P′ is derived using Equation 15 and Equation 17. V and P′ H , where s V2 and H2 is the right shift value.

[0109] (B4) In some embodiments of B3, the refined first prediction value is derived without using a division operation or a shift operation. For example, the first prediction value is a horizontal prediction value, and the refined first prediction value is derived using Equation 6 or Equation 12.

[0110] (B5) In some embodiments of B3 or B4, the method further includes: (i) deriving a second prediction value for the first block, wherein the second prediction value is derived via a second linear interpolation function, the second linear interpolation function involving a second reference sample in a reference sample set and a height and a width of the first block; (ii) deriving a refined second prediction value of the first block; and (iii) deriving a combined prediction value using the refined first prediction value and the refined second prediction value, wherein the first block is decoded using the combined prediction value. In some embodiments, the refined second prediction value is derived using a weighted sum of a second reference sample in a reference sample set and a second prediction value. In some embodiments, deriving the second prediction value of the first block includes performing a right shift operation. For example, P is derived using Equation 14. V , and use Equation 16 to derive P H , where s V1 and H1 is the right shift value. In some embodiments, the combined prediction value is derived using Equation 18. In some embodiments, s V1 +s V2 Equal to s H1 +s H2 , and through P H and P V The combined prediction P is derived by taking the weighted P average of . For example, P is derived using Equation 19, where r is the rounding offset, whose example value is

[0111] (B6) In some embodiments of B5, the second linear interpolation function does not include a division operation or a shift operation. As an example, the second prediction value is a vertical prediction value, and the refined second prediction value is derived using Equation 7 or Equation 10.

[0112] (B7) In some embodiments of B5 or B6, a division operation is used to derive the combined prediction value P. For example, P is derived using Equation 8 or Equation 13.

[0113] (B8) In some embodiments of any one of B1-B7, the first prediction value is a horizontal prediction value P H , and the first interpolation function implements equation 11: P H =(L·(W-1-x)+TR·(x+1))·H. In this function, L is the left reference sample, W is the width of the first block, TR is the upper right reference sample, H is the height of the first block, and x is the horizontal coordinate of the sample to be predicted.

[0114] (B9) In some embodiments of any one of B1-B7, the first prediction value is a vertical prediction value P V , and the first interpolation function implements equation 9: P V =(T·(H-1-y)+BL·(y+1))·W. In this function, T is the top reference sample, W is the width of the first block, BL is the bottom left reference sample, H is the height of the first block, and y is the vertical coordinate of the sample to be predicted.

[0115] (B10) In some embodiments of any one of B1-B9, deriving the first prediction value of the first block includes: performing a right shift operation. For example, the first prediction value is a horizontal prediction value P H , and the first prediction value is derived using equation 16. As another example, the first prediction value is the vertical prediction value P V , and use equation 14 to derive the first predicted value.

[0116] (C1) In another aspect, some embodiments include a method of video decoding (e.g., method 650). In some embodiments, the method is performed on a computing system (e.g., server system 112) having a memory and a control circuit. In some embodiments, the method is performed on a decoding module (e.g., codec module 320). In some embodiments, the method is performed on a parser (e.g., parser 254), a motion prediction component (e.g., motion compensation prediction unit 260), and / or an intra-frame prediction component (e.g., intra-frame picture prediction unit 262). The method includes: (i) receiving video data including multiple blocks (e.g., an encoded video sequence) from a video code stream, the multiple blocks including a first block, wherein the first block is encoded using one of multiple smoothing modes; (ii) identifying a reference sample set of the first block; (iii) deriving a first prediction value (e.g., a combined prediction value) of the first block, wherein the first prediction value is derived from a second prediction value (e.g., a refined prediction value), wherein the second prediction value is derived without using a division operation or a shift operation, and wherein the last step of deriving the first prediction value includes a division operation or a shift operation; and (iv) decoding the first block using the first prediction value.

[0117] (C2) In some embodiments of C1, the second prediction value is a refined prediction value derived from a third prediction value (e.g., a horizontal prediction value or a vertical prediction value) (e.g., using Equation 10, Equation 12, Equation 15, or Equation 17), and the third prediction value is derived without using a division operation or a shift operation.

[0118] (C3) In some embodiments of C2, the third prediction value is derived via a linear interpolation function involving a first reference sample in the reference sample set and a height and a width of the first block.

[0119] (C4) In some embodiments of C3, the third prediction value is a horizontal prediction value P H , and the linear interpolation function is defined by Equation 11.

[0120] (C5) In some embodiments of C3, the third prediction value is a vertical prediction value P V , and the linear interpolation function is defined by Equation 9.

[0121] (C6) In some embodiments of any one of C1-C5, the method further includes: deriving a fourth prediction value for the first block (e.g., using Equation 10, Equation 12, Equation 15, or Equation 17), wherein the first prediction value is derived from a combination of a second prediction value (e.g., a horizontal prediction value) and a fourth prediction value (e.g., a vertical prediction value) (e.g., using Equation 13, Equation 18, or Equation 19).

[0122] (C7) In some embodiments of C6, the fourth prediction value is derived without using a division operation or a shift operation.

[0123] (C8) In some embodiments of C6 or C7, the fourth prediction value is a refined prediction value derived from the fifth prediction value (eg, the horizontal prediction value or the vertical prediction value).

[0124] (C9) In some embodiments of C8, the fifth prediction value is derived via a second linear interpolation function involving a second reference sample in the reference sample set and a height and a width of the first block.

[0125] (C10) In some embodiments of C9, the second linear interpolation function does not include a division operation or a shift operation.

[0126] (C11) In some embodiments of any one of C1-C10, deriving the first prediction value of the first block includes: performing a right shift operation (e.g., using Equation 18).

[0127] (C12) In some embodiments of any one of items C1-C11, the first prediction value is derived using Equation 13.

[0128] (C13) In some embodiments of any one of C1-C11, the first prediction value is derived using Equation 18 or Equation 19.

[0129] On the other hand, some embodiments include a computing system (e.g., server system 112), the computing system including a control circuit (e.g., control circuit 302) and a memory (e.g., memory 314) coupled to the control circuit. The memory stores one or more instruction sets configured to be executed by the control circuit, the one or more instruction sets including: instructions for executing any method described in this application (e.g., A1-A7, B1-B10, and C1-C13 above).

[0130] On the other hand, some embodiments include non-transitory computer-readable storage media for one or more instruction sets executed by control circuitry of a computing system, the one or more instruction sets including instructions for performing any of the methods described herein (e.g., A1-A7, B1-B10, and C1-C13 described above).

[0131] It should be understood that although the terms "first", "second", etc. may be used in the present application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element.

[0132] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit the claims. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "the" are intended to also include plural forms. It should also be understood that the term "and / or" used in this application refers to any combination and all combinations including one or more related listed items. It should be understood that when used in this specification, the term "comprising" specifies the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements and / or components.

[0133] As used herein, the term "if" may be interpreted to mean "when" or "upon" or "in response to determining" or "upon determining" or "in response to detecting" that the antecedent is true, depending on the context. Likewise, the phrase "if it is determined that [the antecedent is true]" or "if [the antecedent is true]" or "when [the antecedent is true]" may be interpreted to mean "upon determination" or "in response to determining" or "upon determining" or "upon detecting" or "in response to detecting" that the antecedent is true, depending on the context.

[0134] For purposes of explanation, the foregoing description has been described with reference to specific embodiments. However, the foregoing illustrative discussion is not intended to be exhaustive or to limit the claims to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments are chosen and described in order to best explain the principles of operation and practical application, thereby enabling others skilled in the art to implement the present disclosure.

Claims

1. A video decoding method, executed on a computing system having a memory and one or more processors, the method comprising: Receiving video data including a plurality of blocks from a video bitstream, the plurality of blocks including a first block, wherein the first block is encoded using one of a plurality of smoothing modes; identifying a set of reference samples of the first block; deriving a first prediction value for the first block, wherein the first prediction value is derived from a second prediction value, wherein the second prediction value is derived without using a division operation or a shift operation, and wherein a last step of deriving the first prediction value includes a division operation or a shift operation; and The first block is decoded using the first prediction value.

2. The method according to claim 1, wherein: The second prediction value is a refined prediction value derived from a third prediction value, and wherein the third prediction value is derived without using a division operation or a shift operation.

3. The method according to claim 2, wherein: The third prediction value is derived via a linear interpolation function involving a first reference sample in the set of reference samples and a height and a width of the first block.

4. The method according to claim 3, wherein: The third prediction value is the horizontal prediction value P H , and wherein the linear interpolation function is P H =(L·(W-1-x)+TR·(x+1))·H.

5. The method according to claim 3, wherein: The third prediction value is the vertical prediction value P V , and wherein the linear interpolation function is P V =(T·(H-1-y)+BL·(y+1))·W.

6. The method according to claim 1, further comprising: A fourth prediction value for the first block is derived, wherein the first prediction value is derived from a combination of the second prediction value and the fourth prediction value.

7. The method according to claim 6, wherein: The fourth prediction value is derived without using a division operation or a shift operation.

8. The method according to claim 6, wherein: The fourth prediction value is a refined prediction value derived from the fifth prediction value.

9. The method according to claim 8, wherein: The fifth prediction value is derived via a second linear interpolation function involving second reference samples in the set of reference samples and a height and a width of the first block.

10. The method according to claim 9, wherein: The second linear interpolation function does not include a division operation or a shift operation.

11. The method according to claim 1, wherein: Deriving a first prediction value for the first block includes performing a right shift operation.

12. The method according to claim 1, wherein: Use P = (P ' H +P′ V +r) / (2·N·W·H) derives the first predicted value P.

13. The method according to claim 1, wherein: use The first prediction value P is derived.

14. A computing system comprising: Control circuit; Memory; as well as One or more instruction sets, stored in the memory and executed by the control circuit, the one or more instruction sets including instructions for: Receiving video data including a plurality of blocks from a video bitstream, the plurality of blocks including a first block, wherein the first block is encoded using one of a plurality of smoothing modes; identifying a set of reference samples of the first block; deriving a first prediction value for the first block, wherein the first prediction value is derived from a second prediction value, wherein the second prediction value is derived without using a division operation or a shift operation, and wherein a last step of deriving the first prediction value includes a division operation or a shift operation; and The first block is decoded using the first prediction value.

15. The computing system of claim 14, wherein: The second prediction value is a refined prediction value derived from a third prediction value, and wherein the third prediction value is derived without using a division operation or a shift operation.

16. The computing system of claim 14, wherein: The one or more sets of instructions also include instructions for deriving a fourth prediction value for the first block, wherein the first prediction value is derived from a combination of the second prediction value and the fourth prediction value.

17. The computing system of claim 14, wherein: Deriving a first prediction value for the first block includes performing a right shift operation.

18. A non-transitory computer-readable storage medium storing one or more sets of instructions configured to be executed by a computing device having control circuitry and memory, the one or more sets of instructions comprising instructions for: Video data including a plurality of blocks is received from a video code stream, wherein the plurality of blocks include a first block, wherein: The first block is encoded using one of a plurality of smoothness modes; identifying a set of reference samples of the first block; deriving a first prediction value for the first block, wherein the first prediction value is derived from a second prediction value, wherein the second prediction value is derived without using a division operation or a shift operation, and wherein a last step of deriving the first prediction value includes a division operation or a shift operation; and The first block is decoded using the first prediction value.

19. The non-transitory computer-readable storage medium of claim 18, wherein: The second prediction value is a refined prediction value derived from a third prediction value, and wherein the third prediction value is derived without using a division operation or a shift operation.

20. The non-transitory computer-readable storage medium of claim 18, wherein: The one or more sets of instructions also include instructions for deriving a fourth prediction value for the first block, wherein the first prediction value is derived from a combination of the second prediction value and the fourth prediction value.

Images