Switchable interpolation filter

Abstract

A method and apparatus for improving compression efficiency in video compression schemes enable the use of switchable interpolation filters. The interpolation filters are used on multiple segments of a video block to achieve better motion compensation prediction and improved video encoding / decoding efficiency. In one embodiment, an interpolation filter index for triangular segmentation of the block is derived from motion information candidate parameters, such that each unidirectional motion compensation can use a different value of the interpolation filter index. In another embodiment, the interpolation filter index is a function of characteristics of the video segmentation, such as shape or area.

Description

Technical Field

[0001] At least one embodiment in this section relates to a method or apparatus for video encoding or decoding. Background Technology

[0002] To achieve high compression efficiency, image and video codec schemes typically employ prediction (including spatial and / or motion vector prediction) and transform to utilize spatial and temporal redundancy in the video content. Generally, intra-frame or inter-frame prediction is used to leverage intra-frame or inter-frame correlations, followed by transforming, quantizing, and entropy encoding / decoding of the differences between the original and predicted images (usually represented as prediction error or prediction residual). To reconstruct the video, the compressed data is decoded through the inverse process corresponding to entropy encoding / decoding, quantization, transforming, and prediction. Multiple codec tools can be used during the encoding / decoding and decoding processes.

[0003] In traditional video codec standards, when performing inter-frame encoding and decoding on blocks, motion vectors are not directly encoded and decoded; instead, the motion vector difference (MVD) between the motion vector (MV) and the motion vector prediction (MVP) is encoded and decoded. AMVR tools allow for a reduction in the number of bits used to encode and decode the MVD by signaling the precision of the MVD (IMV) (and, in some implementations, the precision of the motion vector (MV)). Typical IMV values ​​are FULL_PEL, HALF_PEL, or QUARTER_PEL. For example, in the Multi-Functional Video Codec standard, the IMV is derived from the syntax element "amvr_precision_idx". Summary of the Invention

[0004] The shortcomings and disadvantages of the prior art can be addressed by the general aspects described herein, which relate to block shape adaptive intra-prediction direction in encoding and decoding.

[0005] According to a first aspect, a method is provided. The method includes the steps of: obtaining a filter index corresponding to an interpolation filter; filtering a prediction of at least one video block using the corresponding filter; and encoding the at least one video block using the filtered prediction.

[0006] According to a second aspect, a method is provided. The method includes the steps of: obtaining a filter index corresponding to an interpolation filter; filtering a prediction of at least one video block using the corresponding filter; and decoding the at least one video block using the filtered prediction.

[0007] According to another aspect, an apparatus is provided. The apparatus includes a processor. The processor can be configured to encode video blocks or decode bitstreams by performing any of the methods described above.

[0008] According to another general aspect of at least one embodiment, an apparatus is provided comprising means according to any one of the decoding embodiments; and at least one of the following: (i) an antenna configured to receive a signal including a video block; (ii) a band limiter configured to limit the received signal to a band including the video block; or (iii) a display configured to display an output representing the video block.

[0009] According to another general aspect of at least one embodiment, a non-transitory computer-readable medium is provided that contains data content generated according to any one of the embodiments or variations of the encoding embodiment or variation.

[0010] According to another general aspect of at least one embodiment, a signal is provided that includes video data generated according to any one of the encoding embodiments or variations.

[0011] According to another general aspect of at least one embodiment, the bitstream is formatted to include data content generated according to any one of the encoding embodiments or variations.

[0012] According to another general aspect of at least one embodiment, a computer program product is provided, the computer program product including instructions that, when executed by a computer, cause the computer to perform any one of the embodiments or variations of the decoding embodiment or variation.

[0013] These and other aspects, features, and advantages of the general aspects will become apparent from the following detailed description of exemplary embodiments read in conjunction with the accompanying drawings. Attached Figure Description

[0014] Figure 1 An example of an interpolation filter proposed in a video compression scheme is shown.

[0015] Figure 2 An example flowchart derived from motion compensation filtering in a video compression scheme is shown.

[0016] Figure 3 An example of inter-frame prediction based on triangle segmentation is shown.

[0017] Figure 4 Examples of off-diagonal triangle partitioning and asymmetric partitioning are shown.

[0018] Figure 5 An example of geometric segmentation is shown.

[0019] Figure 6An example of IF index export and usage is shown.

[0020] Figure 7 Another example of IF index export and usage is shown.

[0021] Figure 8 An embodiment of a co-location storage buffer for motion information is shown.

[0022] Figure 9 Examples of unidirectional predictive motion vector selection for a triangular segmentation pattern (left) and weighting factors for sampling in the diagonal (right) are shown.

[0023] Figure 10 Typical standard, general-purpose video compression schemes in which the described embodiments can be implemented are shown.

[0024] Figure 11 Typical standard, general-purpose video decompression schemes in which the described embodiments can be implemented are shown.

[0025] Figure 12 This paper illustrates a proposed derivation, use, and storage of IF index information for double segmentation and / or double prediction using the general aspects described herein.

[0026] Figure 13 An example of using the general aspects described herein to store IF indexes in a motion information buffer is shown.

[0027] Figure 14 An example of deriving an IF from values ​​stored in the motion information of a reference frame buffer is shown.

[0028] Figure 15 An embodiment of a method for encoding using the general aspects described herein is shown.

[0029] Figure 16 An embodiment of a method for decoding using the general aspects described herein is shown.

[0030] Figure 17 An embodiment of an apparatus for encoding or decoding using the general aspects described herein is shown.

[0031] Figure 18 An embodiment of a system for encoding or decoding using the general aspects described herein is shown. Detailed Implementation

[0032] The following general aspects belong to the field of video compression.

[0033] The principle of switchable interpolation filters (IFs) is to improve motion-compensated prediction by selecting an IF (IF-idx) for each block prediction. IFs can typically differ from those with smoothing characteristics (…). Figure 1 ).

[0034] The IF index can be selected based on the codec unit (CU) and can be derived from the codec "imv" index which indicates the resolution of the codec motion vector difference (MVD): if IMV = HALF_PEL, then IF-idx = 1 is selected, otherwise IF-idx = 0.

[0035] In the case of merging, the IF index is not explicitly encoded or decoded, but is derived from the merge candidate.

[0036] In previous work, the IF-index value could be one of two filters (IF-0 or IF-1), but IF-1 could only be used for HALF_PEL motion vector values. Then, if the IF-index is not equal to zero and the MV horizontal (or vertical) component is not HALF_PEL, IF-0 is used. Figure 2 ).

[0037] In the following text, we will consider the case with N = 2 IF filters (IF = 0 and IF = 1) for simplicity, but the invention can be easily extended to the case of N > 2 (IF = 0, ... IF = (N-1)). In this case, the cases of IF = 0 and IF ≠ 0 corresponding to IF = 0 and IF = 1 will be distinguished below. In addition, the "default filter" will be referred to as filter IF = 0.

[0038] Triangular prediction unit

[0039] In previous work, in merge mode, triangular inter-frame prediction allows for the creation of block predictions consisting of two triangular segments. Figure 3 Each triangle segment in the CU uses its own motion for inter-frame prediction; each segment allows only unprediction (unidirectional), meaning each segment has one motion vector and one reference index. Unprediction motion constraints are applied to ensure that, as with regular biprediction, each CU requires only two motion-compensated predictions. If the triangle segmentation pattern is used for the current CU, a signal indicating the direction of the triangle segmentation (diagonal or anti-diagonal) (split-dir) and two merge indices (cand0 and cand1, one per segment) are further sent.

[0040] Asymmetric, extended triangle or geometric segmentation prediction units

[0041] As a generalization of triangular segmentation, the general aspect also applies to CUs segmented into at least two prediction units (PUs), such as, for example, off-diagonal triangular segmentation (as proposed in previous work and in...). Figure 4 The left side depicts asymmetric partitions (as proposed in previous work or in HEVC) and asymmetric partitions (as presented in previous work or in HEVC). Figure 4-Figure 4 (as depicted on the right) or geometric division (as proposed in previous work and in...) Figure 5 (As depicted in the text).

[0042] Figure 6 The derivation of the IF-index is described in AMVR and merge modes. When the accuracy (IMV) of the encoded / decoded MVD is explicitly encoded / decoded, the IF-index is derived from the IMV value. In the case of dual prediction (interDir=3), the same IF-index value is applied to both motion compensations.

[0043] Figure 7 This describes the derivation of the "IF-index" in merge mode. The list of encoding / decoding candidate parameters is constructed from neighboring reconstructed CUs. Explicit encoding / decoding of the candidate index allows selection of the candidate parameter to be used. Candidate parameters typically include MV, reference, and IF-index. For non-merge mode, where the CU is bipredictive, a single value of the IF-index is used. For certain cases (e.g., triangle, affine, etc.), the IF-index is forced to its default value (IF=0).

[0044] For each CU, the IF-index value is typically stored in a co-located memory buffer at a 4×4 resolution. Figure 8 This can be further used to construct a neighbor candidate list. To avoid memory storage burden, a single IF-index is stored. This storage allows the IF-index of subsequent CUs encoded and decoded in merge mode to be derived during the candidate parameter list construction stage. Then, in other works, in the case of double prediction, the stored value is IF = min(IF0, IF1), where IF0 and IF1 are the values ​​of the IF-index used for reference list-0 and reference list-1.

[0045] Question 1: Triangles only use IF-0

[0046] In the case of triangle encoding / decoding, the unidirectional prediction candidate list has up to 6 unidirectional prediction candidates. This list is directly derived from the merge candidate list constructed according to the extended merge prediction process. Let n denote the index of the unidirectional prediction motion in the triangle unidirectional prediction candidate list. The LX motion vector of the nth extended merge candidate (where X equals the parity of n) is used as the nth unidirectional prediction motion vector of the triangle segmentation pattern. These motion vectors are... Figure 9 The value is marked as "x". In the absence of a corresponding LX motion vector for the nth extended merge candidate, the L(1-X) motion vector of the same candidate is used instead as the unidirectional predicted motion vector for the triangle segmentation pattern.

[0047] Aside from the sampling in the diagonal portion, which consists of two weighted unidirectional predictions (mixed), the sampling of the PU prediction block is primarily composed of unidirectional predictions. Therefore, in the case of the triangular pattern, using a dual prediction rule regarding the IF-index may be counterproductive.

[0048] Question 2 :

[0049] For dual predictions, the IF-index is shared by both unidirectional predictions. However, this can be counterproductive if one prediction is HALF_PEL and the other is not, and the IF equals 1.

[0050] The affected codec modules are Figure 10 The encoder motion compensation (170) and reference frame buffer (motion information) (180) and Figure 11 The corresponding decoder motion compensation (275) and reference frame buffer (motion information) (280).

[0051] Example 1 :

[0052] In the motion compensation process, various interpolation filters can be selected, and CU decoding means that in the case of performing two motion compensations with two different MVs and / or reference frames (e.g., double prediction or triangle or geometry), possibly different IF-index values ​​(IF-0 and IF-1) can be applied.

[0053] Example 2 :

[0054] (Example 1) + In the case of CU decoding mode "triangle", the IF-index value of each triangle segment is derived from each motion information candidate parameter, so that each unidirectional motion compensation can use a different IF-index value. Figure 12 ).

[0055] Example 3 :

[0056] (Example 1 or Example 2) where a single value of IF-index is stored in the motion information buffer and is calculated as min(IF-0, IF-1) Figure 12 ) or max(IF-0,IF-1).

[0057] In the variant, a single value of IF-index is a function of the two segmented shapes.

[0058] a) This function depends on the relative area ratio between the two partitions. For example, if region-0 > region-1, then IF-index = IF-0.

[0059] b) Or the function depends on the positions of these two regions. For example, if region -1 is lower than region -0 (e.g.: Figure 6 If the three lower right segments in the code are used, then IF-index = IF-1. In practice, this function works well for (most) cases where the subsequent CU to be decoded is lower than the current CU. In another example, if region -1 is to the right of region -0 (e.g., ... Figure 6 If the three upper right segments in the code are used, then IF-index = IF-1. In practice, this function works well when (most of the time) the subsequent CU to be decoded is to the right of the current CU.

[0060] Example 4 :

[0061] The transition of the IF-index value into the motion information buffer is similar to the transition of the MV and ref-idx parameters. In cases where the two segmented regions do not completely overlap (e.g., triangular or geometric segmentation), the stored IF-index value corresponds to a unidirectional motion compensation region at the resolution of the motion information buffer (e.g., 8×8).

[0062] In the motion information buffer region, some of the PU samples have been calculated with more than one unidirectional motion compensation, and the IF-index value spanned is equal to min(IF-0,IF-1) or max(IF-0,IF-1).

[0063] In the variant, and similarly as in embodiment 3, the spanned IF-index value is a function of the relative portions of the two segmented shapes. Figure 13 ).

[0064] Example 5 :

[0065] (Example-1 or Example-2) is only applicable when the two segmented regions do not completely overlap (e.g., applicable to triangular or geometric PU segmentation but not including classic biprediction).

[0066] The reason is that if (all) prediction samples are averaged together, the benefits of the switchable interpolation filter are reduced due to the double-segment weighting.

[0067] Example 6 :

[0068] In the variant, the reference frame (e.g., the reference frame at position (current CU+MV)) is used. Figure 14 The value of IF-index is retrieved from the motion information buffer.

[0069] In the variant, the value of IF is retrieved only if the location (current-CU+MV) is within a given restricted area (typically the CTU of the current CU).

[0070] Example 7 :

[0071] In the case of generalized double forecasts (also known as GBI, double forecast weighted average BPWA, or double forecast with CU-level weighted BCW), double forecast weighting can use unequal weights (w0, w1) to average two single forecasts:

[0072] bi-pred(x)=w0.pred0(x)+w1.pred1(x)

[0073] A function that depends on the values ​​of (w0, w1) is proposed to derive the value of the IF-index used for motion compensation and / or for storage in the motion information buffer. For example:

[0074] a) If w0 > w1, then IF = IF0; otherwise, if w1 > w0, then IF = IF1; otherwise, IF = min(IF0, IF1).

[0075] b) Or if W0 = W1, then IF = 0

[0076] Figure 15 An embodiment of method 1500 under the general aspects described herein is illustrated. The method begins at start block 1501 and control advances to block 1510 to obtain a filter index corresponding to an interpolation filter. Control advances from block 1510 to block 1520 to filter the prediction of at least one video block using the corresponding filter. Control advances from block 1520 to block 1530 for encoding the at least one video block using the filtered prediction.

[0077] Figure 16An embodiment of method 1600 under the general aspects described herein is illustrated. The method begins at start block 1601 and control advances to block 1610 to obtain a filter index corresponding to an interpolation filter. Control advances from block 1610 to block 1620 to filter the prediction of at least one video block using the corresponding filter. Control advances from block 1620 to block 1630 to decode the at least one video block using the filtered prediction.

[0078] Figure 17 An embodiment of an apparatus 1700 for compressing, encoding, or decoding video using an interpolation filter is shown. The apparatus includes a processor 1710 and can be interconnected to a memory 1720 via at least one port. Both the processor 1710 and the memory 1720 may also have one or more additional interconnects for external connection.

[0079] The processor 1710 is also configured to insert or receive information in the bitstream and to use interpolation filters for compression, encoding, or decoding.

[0080] This application describes various aspects, including tools, features, embodiments, models, methods, etc. Many of these aspects are specifically described, and at least the various characteristics are shown, generally in a manner that may sound restrictive. However, this is for clarity and does not limit the application or scope of those aspects. In fact, all the different aspects can be combined and interchanged to provide further aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in previous applications.

[0081] The aspects described and anticipated in this application can be implemented in many different forms. Figure 10 , Figure 11 and Figure 18 Some embodiments have been provided, but other embodiments are contemplated, and Figure 10 , Figure 11 and Figure 18 The discussion does not limit the breadth of implementations. At least one of these aspects typically relates to video encoding and decoding, and at least one other aspect typically relates to transmitting a generated or encoded bitstream. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media having a bitstream generated according to any of the methods stored thereon.

[0082] In this application, the terms "reconstruction" and "decoding" are used interchangeably, as are the terms "pixel" and "sample," and the terms "image," "picture," and "frame." Generally, but not necessarily, the term "reconstruction" is used on the encoder side, while "decoding" is used on the decoder side.

[0083] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0084] The various methods and other aspects described in this application can be used to modify, for example... Figure 10 and Figure 11 The illustrated video encoder 100 and decoder 200 include modules (e.g., intra-frame prediction, entropy encoding / decoding, and / or decoding modules (160, 360, 145, 330)). Furthermore, aspects of the invention are not limited to VVC or HEVC and can be applied to, for example, other standards and recommendations (whether pre-existing or future-developed) and any extensions to such standards and recommendations (including VVC and HEVC). Unless otherwise specified or technically excluded, the aspects described in this application may be used individually or in combination.

[0085] Various numerical values ​​are used in this application. Specific values ​​are for illustrative purposes only, and the aspects described are not limited to these specific values.

[0086] Figure 10 Encoder 100 is shown. Variations of this encoder 100 are expected, but for clarity, encoder 100 is described below without describing all expected variations.

[0087] Before encoding, the video sequence may undergo pre-coding processing (101), such as applying color transformations to the input color picture (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input picture components to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). Metadata may be associated with pre-processing and appended to the bitstream.

[0088] In encoder 100, the frame is encoded by encoder elements as described below. The frame to be encoded is segmented (102) and processed in units such as CUs. Each unit is encoded using, for example, an intra-frame mode or an inter-frame mode. When a unit is encoded in intra-frame mode, it performs intra-frame prediction (160). In inter-frame mode, motion estimation (175) and compensation (170) are performed. The encoder determines (105) which of the intra-frame mode or inter-frame mode is used to encode the unit, and indicates the intra / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (110) the prediction block from the original image block.

[0089] The predicted residual is then transformed (125) and quantized (130). The quantized transform coefficients, motion vectors, and other syntax elements are entropy encoded (145) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, i.e., encode and decode the residual directly without applying the transform or quantization process.

[0090] The encoder decodes the coded block to provide a reference for further prediction. The quantized transform coefficients are dequantized (140) and inverse transformed (150) to decode the prediction residual. The decoded prediction residual and the prediction block are combined (155) to reconstruct the image block. A loop filter (165) is applied to the reconstructed image to perform, for example, deblocking / SAO (sample adaptive offset) filtering to reduce coded artifacts. The filtered image is stored in a reference image buffer (180).

[0091] Figure 11 A block diagram of a video decoder 200 is shown. In decoder 200, the bitstream is decoded by decoder elements, as described below. Video decoder 200 generally performs operations similar to... Figure 10 The encoding channel is the inverse of the decoding channel. Encoder 100 typically also performs video decoding as part of the encoding of video data.

[0092] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 100. First, entropy decoding (230) is performed on the bitstream to obtain transform coefficients, motion vectors, and other encoded / decoded information. Frame segmentation information indicates how the frame should be segmented. Therefore, the decoder can segment (235) the frame based on the decoded frame segmentation information. The transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The decoded prediction residual and prediction block are combined (255) to reconstruct the image block. The prediction block (270) can be obtained from intra-frame prediction (260) or motion-compensated prediction (i.e., inter-frame prediction) (275). A loop filter (265) is applied to the reconstructed image. The filtered image is stored in a reference frame buffer (280).

[0093] The decoded image can also undergo post-decoding processing (285), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or performing the inverse of the remapping process performed in the pre-encoding process (101). The post-decoding process can use metadata derived in the pre-encoding process and signaled in the bitstream.

[0094] Figure 18 A block diagram illustrating an example system implementing various aspects and embodiments thereof is shown. System 1000 may be embodied as a device including the various components described below and configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 1000 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one embodiment, the processing elements and encoder / decoder elements of system 1000 are distributed across multiple ICs and / or discrete components. In various embodiments, system 1000 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 1000 is configured to implement one or more aspects described in this document.

[0095] System 1000 includes at least one processor 1010 configured to execute instructions loaded thereon for implementing various aspects, such as those described in this document. Processor 1010 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 1000 includes at least one memory 1020 (e.g., a volatile memory device and / or a non-volatile memory device). System 1000 includes a storage device 1040 that may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 1040 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0096] System 1000 includes an encoder / decoder module 1030 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 1030 may include its own processor and memory. The encoder / decoder module 1030 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both of an encoding module and a decoding module. Furthermore, the encoder / decoder module 1030 may be implemented as a separate element of system 1000, or may be incorporated within processor 1010 as a combination of hardware and software known to those skilled in the art.

[0097] Program code to be loaded onto processor 1010 or encoder / decoder 1030 to execute the various aspects described in this document may be stored in storage device 1040 and subsequently loaded onto memory 1020 for execution by processor 1010. According to various embodiments, one or more of processor 1010, memory 1020, storage device 1040, and encoder / decoder module 1030 may store one or more items from various projects during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded or partially decoded video, bitstreams, matrices, variables, and intermediate or final results of processing from equations, formulas, operations, and operational logic.

[0098] In some embodiments, the memory within the processor 1010 and / or encoder / decoder module 1030 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 1010 or encoder / decoder module 1030) is used for one or more of these functions. The external memory may be memory 1020 and / or storage device 1040, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory, such as RAM, is used as working memory for video encoding and decoding operations, such as for MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Codec, also known as H.265 and MPEG-H Part 2) or VVC (Various Video Codec, a new standard developed by the Joint Video Experts Group JVET).

[0099] Inputs to the components of system 1000 may be provided by various input devices as shown in block 1130. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) a component (COMP) input terminal (or a set of COMP input terminals); (iii) a universal serial bus (USB) input terminal; and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 18 Other examples not shown include composite video.

[0100] In various embodiments, the input device of block 1130 has associated corresponding input processing elements as known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further band-limiting to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section in various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0101] Furthermore, USB and / or HDMI terminals may include corresponding interface processors for connecting system 1000 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 1010. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processor 1010. Demodulated, error-corrected, and demultiplexed streams are provided to various processing elements, including, for example, processor 1010 and encoder / decoder 1030, which operate in conjunction with memory and storage elements to process the data streams as needed for presentation on the output device.

[0102] Various components of the system 1000 can be housed within an integrated housing, in which the various components can be interconnected and transmit data between them using a suitable connection arrangement (e.g., internal buses as known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards).

[0103] System 1000 includes a communication interface 1050 capable of communicating with other devices via a communication channel 1060. The communication interface 1050 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 1060. The communication interface 1050 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 1060 may be implemented, for example, in a wired and / or wireless medium.

[0104] In various embodiments, data is streamed to or otherwise provided to system 1000 using a wireless network such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers) Wi-Fi network. In these embodiments, Wi-Fi signals are received via a communication channel 1060 and a communication interface 1050 suitable for Wi-Fi communication. The communication channel 1060 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, for use by applications that allow streaming and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 1000, delivering data via an HDMI connection of input block 1130. Still other embodiments use an RF connection of input block 1130 to provide streaming data to system 1000. As described above, various embodiments provide data in a non-streaming manner. Furthermore, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0105] System 1000 can provide output signals to various output devices, including a display 1100, a speaker 1110, and other peripheral devices 1120. The display 1100 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display 1100 can be used in a television, tablet computer, laptop computer, cellular phone (mobile phone), or other device. The display 1100 may also be integrated with other components (e.g., as in a smartphone) or standalone (e.g., an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 1120 include one or more of a standalone digital video disc (or digital multifunction disc) (both terms are DVRs), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 1120 that provide functionality based on the output of system 1000. For example, a disc player performs the function of playing the output of system 1000.

[0106] In various embodiments, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols is used to transmit control signals between system 1000 and display 1100, speaker 1110, or other peripheral devices 1120, enabling device-to-device control with or without user intervention. Output devices may be communicatively coupled to system 1000 via dedicated connections through corresponding interfaces 1070, 1080, and 1090. Alternatively, output devices may be connected to system 1000 via communication interface 1050 using communication channel 1060. Display 1100 and speaker 1110 may be integrated into a single unit with other components of system 1000 in electronic devices such as, for example, televisions. In various embodiments, display interface 1070 includes display drivers, such as, for example, timing controller (TCon) chips.

[0107] For example, if the RF portion of input 1130 is part of a separate set-top box, then display 1100 and speaker 1110 may optionally be separate from one or more other components. In various embodiments where display 1100 and speaker 1110 are external components, the output signal may be provided via a dedicated output connection, including, for example, an HDMI port, a USB port, or a COMP output.

[0108] These embodiments may be implemented by computer software, hardware, or a combination of hardware and software, implemented by processor 1010. As a non-limiting example, these embodiments may be implemented by one or more integrated circuits. As a non-limiting example, memory 1020 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. As a non-limiting example, processor 1010 may be of any type suitable for the technical environment and may encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0109] Various implementations involve decoding. As used herein, "decoding" can encompass, for example, all or part of a process performed on a received encoded sequence to produce a final output suitable for display. In various embodiments, such processes include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such processes also include, or alternatively include, processes performed by a decoder of the various implementations described herein.

[0110] As a further example, in one embodiment, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" specifically refers to a subset of operations or broadly refers to a wider decoding process will be clear based on the specific context of the description and is considered well understood by those skilled in the art.

[0111] Various implementations involve encoding. In a manner similar to the discussion above regarding “decoding,” the term “encoding,” as used herein, can encompass all or part of a process performed on an input video sequence to produce an encoded bitstream. In various embodiments, such processes include one or more processes typically performed by an encoder, such as segmentation, differential coding, transform, quantization, and entropy coding. In various embodiments, such processes also include, or alternatively include, processes performed by the encoders of the various implementations described herein.

[0112] As a further example, in one embodiment, “encoding” refers only to entropy encoding; in another embodiment, “encoding” refers only to differential encoding; and in yet another embodiment, “encoding” refers to a combination of differential encoding and entropy encoding. Whether the phrase “encoding process” specifically refers to a subset of operations or broadly refers to a wider encoding process will be clear based on the specific context of the description and is considered well understood by those skilled in the art.

[0113] Note that the grammatical elements used in this article are descriptive terms. Therefore, the use of other grammatical element names is not excluded.

[0114] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0115] Various implementations can refer to parametric models or rate distortion optimization. Specifically, during the encoding process, a balance or trade-off between rate and distortion is typically considered, often taking into account computational complexity constraints. This can be measured by rate distortion optimization (RDO) or by least mean square (LMS), average absolute error (MAE), or other such measures. Rate distortion optimization is typically formulated as minimizing a rate distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate distortion optimization problem. For example, these approaches may be based on extensive testing of all encoding options, including all considered modes or encoding / decoding parameter values, and a complete evaluation of the encoding / decoding costs and associated distortion of the reconstructed signal after encoding / decoding and decoding. Faster methods can also be used to save encoding complexity, particularly by calculating approximate distortion based on prediction or prediction of the residual signal rather than the reconstructed predicted residual signal. A hybrid of these two approaches can also be used, such as by using approximate distortion for only some of the possible encoding options and full distortion for others. Other methods evaluate only a subset of the possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete assessment of both encoding / decoding costs and associated distortion.

[0116] The implementations and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the features discussed may also be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. Methods may be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0117] References to "an embodiment" or "an embodiment" or "an implementation" or "an implementation," and other variations thereof, mean that a particular feature, structure, characteristic, etc., described in connection with the embodiment is included in at least one embodiment. Therefore, the phrases "in an embodiment" or "in an embodiment" or "in an implementation" or "in an implementation," and any other variations appearing throughout this application, do not necessarily refer to the same embodiment.

[0118] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, one or more of the following: estimation information, calculation information, prediction information, or information retrieved from memory.

[0119] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0120] Additionally, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Furthermore, "receiving" generally refers to one or more of the following during operations such as, for example, storing information, processing information, sending information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0121] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or selecting both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as possible listed.

[0122] Furthermore, as used herein, the term "signaling" refers to (among other things) instructing the corresponding decoder to do something. For example, in some embodiments, the encoder signals a specific one of a plurality of transforms, encoding / decoding modes, or flags. Thus, in one embodiment, the same transform, parameter, or mode is used on both the encoder and decoder sides. Therefore, for example, the encoder may send (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter and others, signaling may be used without sending (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, information is signaled to the corresponding decoder using one or more syntax elements, flags, etc. Although the verb form of the term "signal" has been used above, the term "signal" may also be used as a noun herein.

[0123] It will be apparent to those skilled in the art that implementations can generate various signals formatted to carry information that can be stored or transmitted, for example. The information may include, for example, instructions for performing a method or data generated by one of the implementations. For example, the signal may be formatted to carry a bit stream of the embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0124] We have described several embodiments across various claim classes and types. Features of these embodiments may be provided individually or in any combination. Furthermore, embodiments may include one or more of the following features, devices, or aspects, individually or in any combination, across various claim classes and types:

[0125] • A process or apparatus for encoding or decoding video data using a low-frequency non-separated transform with multiple transform options.

[0126] • A process or apparatus for encoding or decoding video data using low-matrix-based intra-frame prediction with multiple transform options.

[0127] • A process or apparatus for encoding or decoding video data using a low-frequency non-separated transform with implicit multiple transform options.

[0128] • A process or apparatus for encoding or decoding video data using low-matrix-based intra-frame prediction with implicit multiple transform selections.

[0129] • Includes a bitstream or signal of one or more syntax elements or their variants from the described syntax elements.

[0130] • Includes bitstreams or signals that generate grammatical information according to any of the embodiments described.

[0131] • Create and / or send and / or receive and / or decode according to any of the embodiments described.

[0132] • A method, process, apparatus, medium for storing instructions, medium for storing data, or signal according to any of the embodiments described.

[0133] • Insert syntax elements into the signaling that enable the decoder to determine the encoding / decoding mode in a manner corresponding to that used by the encoder.

[0134] • Create and / or send and / or receive and / or decode bit streams or signals including one or more of the syntax elements or variations thereof.

[0135] • A television, set-top box, cellular phone, tablet computer or other electronic device that performs the transformation method according to any of the embodiments described.

[0136] • A television, set-top box, cellular phone, tablet computer or other electronic device that performs the transformation method according to any of the embodiments described to determine and display the resulting image (e.g., using a monitor, screen or other type of display).

[0137] A television, set-top box, cellular phone, tablet computer or other electronic device that selects, band-limits or tunes (e.g., uses a tuner) a channel to receive signals including encoded images and perform transformation methods according to any of the embodiments described.

[0138] A television set, set-top box, cellular phone, tablet computer or other electronic device that receives signals including encoded images over the air (e.g., using an antenna) and performs a transformation method.

Claims

1. A video coding method using a switchable interpolation filter, the method comprising: The prediction of at least one video block is filtered using an interpolation filter, wherein the at least one video block is divided into at least two parts, each part having a corresponding filter index, wherein the filtering of the prediction for each part is performed using an interpolation filter corresponding to the filter index of the part, and when the encoding mode is a triangle segmentation mode, the value of the filter index for each triangle segmentation is derived from each motion information candidate parameter, and a single value of the filter index is stored in a motion information buffer, the single value being calculated as the minimum or maximum value of the corresponding filter index of the triangle segmentation; The prediction is obtained using two motion compensations with two different motion vectors or references; and, The at least one video block is encoded using filtered predictions.

2. A video encoding apparatus using a switchable interpolation filter, the apparatus comprising: Processor, the processor being configured to execute: The prediction of at least one video block is filtered using an interpolation filter, wherein the at least one video block is divided into at least two parts, each part having a corresponding filter index, wherein the filtering of the prediction for each part is performed using an interpolation filter corresponding to the filter index of the part, and when the encoding mode is a triangle segmentation mode, the value of the filter index for each triangle segmentation is derived from each motion information candidate parameter, and a single value of the filter index is stored in a motion information buffer, the single value being calculated as the minimum or maximum value of the corresponding filter index of the triangle segmentation; The prediction is obtained using two motion compensations with two different motion vectors or references; and, The at least one video block is encoded using filtered predictions.

3. A video decoding method using a switchable interpolation filter, the method comprising: The prediction of at least one video block is filtered using an interpolation filter, wherein the at least one video block is divided into at least two parts, each part having a corresponding filter index, wherein the filtering of the prediction for each part is performed using an interpolation filter corresponding to the filter index of the part, and when the encoding mode is a triangle segmentation mode, the value of the filter index for each triangle segmentation is derived from each motion information candidate parameter, and a single value of the filter index is stored in a motion information buffer, the single value being calculated as the minimum or maximum value of the corresponding filter index of the triangle segmentation; The prediction is obtained using two motion compensations with two different motion vectors or references; and, The at least one video block is decoded using filtered predictions.

4. A video decoding apparatus using a switchable interpolation filter, the apparatus comprising: Processor, the processor being configured to execute: The prediction of at least one video block is filtered using an interpolation filter, wherein the at least one video block is divided into at least two parts, each part having a corresponding filter index, wherein the filtering of the prediction for each part is performed using an interpolation filter corresponding to the filter index of the part, and when the encoding mode is a triangle segmentation mode, the value of the filter index for each triangle segmentation is derived from each motion information candidate parameter, and a single value of the filter index is stored in a motion information buffer, the single value being calculated as the minimum or maximum value of the corresponding filter index of the triangle segmentation; The prediction is obtained using two motion compensations with two different motion vectors or references; and, The at least one video block is decoded using filtered predictions.

5. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein triangular inter-frame prediction is used, and the filter index is obtained from motion candidate parameters, and unidirectional motion compensation can use a separate filter index.

6. The method according to claim 1 or claim 3, or the apparatus according to claim 2 or claim 4, wherein the filter index is a function of the characteristics of two or more segments of the video block.

7. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein the filter index is a function comprising two or more relative segments of the video block.

8. The method according to claim 1 or claim 3, or the apparatus according to claim 2 or claim 4, wherein different filter indices are applied when the double-segmented regions do not overlap.

9. The method according to claim 1 or claim 3, or the apparatus according to claim 2 or claim 4, wherein the filter index is obtained from motion information of the reference frame.

10. The method of claim 1 or claim 3, or the apparatus of claim 2 or claim 4, wherein dual prediction is performed using weighted prediction, and the filter index is a function of the weights used in the weighted prediction.

11. An apparatus, the apparatus comprising: The apparatus according to any one of claims 4 to 10; as well as At least one of the following: (i) an antenna configured to receive a signal including the video block; (ii) a bandwidth limiter configured to limit the received signal to a bandwidth including the video block; and (iii) a display configured to display an output representing a video block.

12. A non-transitory computer-readable medium having a computer program and a bit stream stored thereon, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 10 to generate the bit stream.

13. A method for transmitting a bit stream, comprising performing the method according to any one of claims 1 to 10 to generate the bit stream and to transmit the bit stream.

14. A computer program product comprising instructions that, when executed by a computer, cause the computer to perform the method according to any one of claims 1, 3, and 5 to 10.

Images