More accurate 2-tap interpolation filter for video compression
By using a 2-tap filter and a sub-pixel interpolation technology of a weight-heavy-distributed triangle core in video encoding, the problems of high complexity and low efficiency of motion vector sub-pixel interpolation in the prior art are solved, and more efficient videos are achieved Compression and better responsiveness.
Patent Information
- Application Number
- CN201980079722.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-01-24
- Filing Date
- 2019-11-04
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-11-04
AI Technical Summary
The existing video encoding technology has high complexity and low efficiency in the sub-pixel interpolation process of motion vectors, resulting in poor video compression performance, especially in real-time video applications, which is difficult to balance quality and responsiveness.
Use a 2-tap filter as a sub-pixel interpolation filter, sub-pixel interpolation is performed through a weight-distributed triangle core, and the appropriate interpolation filter is selected to provide approximately longer filter performance during the motion estimation and motion compensation phases. Reduce complexity.
Improves the overall performance of video codecs, reduces bit rate, improves video compression efficiency, and improves responsiveness in real-time applications.
Smart Images

Figure CN113170207B_ABST
Abstract
Description
Background Art
[0001] Digital video can be used, for example, for remote business meetings via video conferencing, high-definition video entertainment, video advertising, or sharing of user-generated videos. Due to the large amount of data involved in video data, both transmission and storage require high-performance compression. Various methods have been proposed to reduce the amount of data in a video stream, including compression and other encoding and decoding techniques. These techniques may involve sub-pixel interpolation for fractional motion. Summary of the invention
[0002] One aspect is a method for encoding a block of a video using inter-frame prediction. The method includes: during motion estimation for the block, selecting a first subpixel interpolation filter for subpixel interpolation of fractional motion; during motion compensation, selecting a second subpixel interpolation filter based on the first subpixel interpolation filter; and encoding the second subpixel interpolation filter in a compressed bitstream. The first subpixel interpolation filter is a 2-tap filter. The first subpixel interpolation filter includes weight tuples. Each weight tuple includes two weights and is used for subpixel interpolation at a subpixel position between a first pixel and a second pixel. For at least one weight tuple in the weight tuples, the two weights are not based on a first distance between the subpixel position and the first pixel and a second distance between the subpixel position and the second pixel.
[0003] On the other hand is a device for encoding a block of a video using inter-frame prediction, the device comprising a processor and a memory. The memory comprises instructions executable by the processor to: during motion estimation for the block, select a first sub-pixel interpolation filter for sub-pixel interpolation of fractional motion; select a second sub-pixel interpolation filter based on a rate-distortion cost; and encode the second sub-pixel interpolation filter in a compressed bitstream. The first sub-pixel interpolation filter is a 2-tap filter. The first sub-pixel interpolation filter comprises a weight tuple. Each weight tuple comprises two weights and is used for sub-pixel interpolation at a sub-pixel position between a first pixel and a second pixel. For at least one weight tuple in the weight tuple, the two weights are not based on a first distance between the sub-pixel position and the first pixel and a second distance between the sub-pixel position and the second pixel.
[0004] On the other hand, a device for decoding a block of a video using inter-frame prediction includes a processor and a memory. The memory includes instructions that can be executed by the processor to: decode a subpixel interpolation filter from an encoded bitstream, the subpixel interpolation filter being used for subpixel interpolation of fractional motion; decode motion vectors from the encoded bitstream; and decode the block using the subpixel interpolation filter and the motion vector. The subpixel interpolation filter is a 2-tap filter. The subpixel interpolation filter includes a weight tuple. Each weight tuple includes two weights and is used for subpixel interpolation at a subpixel position between a first pixel and a second pixel. For at least one weight tuple in the weight tuple, the two weights are not based on a first distance between the subpixel position and the first pixel and a second distance between the subpixel position and the second pixel.
[0005] These and other variations are described in more detail below. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] The description herein refers to the drawings, wherein like reference numerals refer to like parts throughout the several views.
[0007] Figure 1 is an illustration of a computing device according to an embodiment of the present disclosure.
[0008] Figure 2 is an illustration of a computing and communication system according to an embodiment of the present disclosure.
[0009] Figure 3 is an illustration of video streams for encoding and decoding according to an embodiment of the present disclosure.
[0010] Figure 4 is a block diagram of an encoder according to an embodiment of the present disclosure.
[0011] Figure 5 is a block diagram of a decoder according to an embodiment of the present disclosure.
[0012] Figure 6 is a block diagram of motion vectors with sub-pixel accuracy according to an embodiment of the present disclosure.
[0013] Figure 7 is a diagram of a bilateral filter according to an embodiment of the present disclosure.
[0014] Figure 8 A 2-tap filter similar to a high-order kernel is illustrated according to an embodiment of the present disclosure.
[0015] Fig. 9 is a flow chart of an example of a process for encoding a block of a video using inter-frame prediction according to an embodiment of the present disclosure.
[0016] Fig.10 is a flow chart of an example of a process for decoding a block of a video using inter-frame prediction according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0017] The video compression scheme may include dividing each image or frame into smaller parts, such as blocks, and generating an output bitstream using techniques for limiting the information included in each block in the output. The encoded bitstream may be decoded to recreate the source image based on the limited information. In some embodiments, the information included in each block in the output may be limited by reducing spatial redundancy, reducing temporal redundancy, or a combination thereof. For example, temporal redundancy or spatial redundancy may be reduced by predicting a frame based on information available to both the encoder and the decoder, and including information representing the difference or residual between the predicted frame and the original frame.
[0018] Coding using temporal similarities (e.g., redundancy) may be referred to as inter-frame prediction. Inter-frame prediction (e.g., motion compensated prediction) attempts to predict the pixel values of a block using one or more blocks from a temporally adjacent frame (i.e., a reference frame) or multiple frames, possibly displaced. That is, motion compensated prediction (MCP) may be used to exploit temporal redundancy between video frames. Temporally adjacent frames are frames that appear earlier or later in time in the video stream than the frame of the block being encoded.
[0019] The accuracy of MCP determines the compression ratio to a large extent. During the encoding process, and as further described below, the video frame can be divided into blocks of various sizes. For each block, motion estimation is performed. For the current block being encoded (i.e., the current block), motion estimation performs a motion search to find the motion vector of the block. The motion vector indicates the position of the prediction block in the reference frame (i.e., the temporally adjacent frame). The prediction block is used to predict the block. Generally, the motion vector can be a fraction. That is, the motion vector can refer to (e.g., point to) a sub-pixel position in the reference frame. That is, the motion vector can be given in sub-pixel units. In other words, the motion vector can be expressed with sub-pixel resolution. Different coding standards support different motion vector resolutions (i.e., sub-pixel units, precision). For example, in the HEVC standard, the brightness motion vector can be given in quarter brightness sample units. For example, in the VP9 standard, the maximum motion vector precision supported is 1 / 16 pixel.
[0020] The sub-pixel values given by the motion vectors can be calculated by interpolation using a filter such as a finite impulse response (FIR) filter. Different interpolation filters can have different numbers of taps, which indicate how many neighboring pixels are used to interpolate the sub-pixel value. A weight (i.e., coefficient) can be associated with each tap. In most video codecs, the number of taps can range from 2 to 8. For example, given an 8-tap filter with eight weights W = (w0, w1, ..., w7) and a set of eight (integer or sub-integer) pixel values A = (A0, A1, ..., A7), the interpolated pixel value I can be calculated as the weighted sum
[0021] As mentioned, MCP may involve the use of sub-pixel interpolation filters that generate filtered sub-pixel values at defined locations between full pixels (also known as integer pixels) along rows, columns, or both. The interpolation filter may be one of several interpolation filters available for use in MCP. Each of the interpolation filters may have a different frequency distribution diagram (e.g., frequency response).
[0022] A sub-pixel interpolation filter may be selected from a set of predefined or available sub-pixel interpolation filter types. For example, in the VP9 standard, the predefined sub-pixel interpolation filter types include smooth interpolation filter (EIGHTTAP_SMOOTH), sharp interpolation filter (EIGHTTAP_SHARP), normal interpolation filter (EIGHTTAP), bilinear interpolation filter (BILINEAR). AV1 includes similarly named interpolation filters. Figure 7 Bilinear filters are further described. Other codecs may have fewer, more, or other interpolation filters. A description of the semantics (eg, frequency response, tap weights, etc.) of the interpolation filter types is not necessary to understand or use the present disclosure.
[0023] The normal interpolation filter (EIGHTTAP) may be an 8-tap Lagrangian interpolation filter. The sharp interpolation filter (EIGHTTAP_SHARP) may be a DCT-based interpolation filter with a sharper response and is used primarily around sharper edges. The smooth interpolation filter (EIGHTTAP_SMOOTH) may be an 8-tap filter that may be designed using a windowed Fourier series method with a Hamming window. As will be appreciated by those skilled in the art, a filter such as Commercial off-the-shelf tools for designing filters (e.g., interpolation filters) are available from The Math Inc. developed a numerical computing environment and proprietary programming language.
[0024] A longer interpolation filter (e.g., an 8-tap filter) can provide higher prediction accuracy than a shorter filter (e.g., a 2-tap filter). For this reason, using a longer interpolation filter results in higher compression. However, while a longer filter (e.g., an 8-tap filter) provides higher prediction accuracy, a shorter filter (e.g., a 2-tap filter) has lower complexity. The complexity of the coding algorithm can be measured in terms of memory bandwidth and the number of operations required by the algorithm. In terms of memory bandwidth, using a longer tap filter (e.g., an 8-tap filter) increases the amount of data to be retrieved from the memory compared to a shorter filter (e.g., a 2-tap filter). Similarly, a longer tap length filter (e.g., an 8-tap filter) increases the number of arithmetic operations required to obtain the interpolated value. For example, an 8-tap filter performs a total of 15 multiplication and addition operations (MACs); that is, 8 multiplications and 7 additions. On the other hand, a 2-tap filter performs only 3 MACs; that is, 2 multiplications and 1 addition.
[0025] Interpolation operations using interpolation filters can be performed during the motion estimation stage in the encoding process and during the motion compensation stage in both the encoding process and the decoding process. The design of the interpolation filter may affect the coding performance and the encoding speed and decoding speed. The design of the interpolation filter may include the design (e.g., selection) of weights (e.g., coefficients) for the interpolation operation.
[0026] The interpolation filter to be used by the decoder to generate the prediction block can be signaled in the header of the frame containing the block to be predicted in the encoded bitstream. To this end, the same interpolation filter can be used to generate sub-pixel prediction blocks for all blocks of the frame. The interpolation filter can also be signaled at the coding unit level. To this end, the same interpolation filter can be used for each block of the coding unit (e.g., each prediction block), thereby generating sub-pixel prediction blocks for the blocks of the coding unit. The coding unit can be a 64x64, smaller or larger pixel block of a video frame, and can be further divided into smaller blocks. The encoder can generate a prediction block based on each of the available interpolation filters.
[0027] The encoder may select (e.g., signal to the decoder in the encoded bitstream) an interpolation filter that results in, for example, an optimal rate-distortion cost. The rate-distortion cost refers to a ratio that balances the amount of distortion (i.e., loss of video quality) with the rate (i.e., number of bits) required for encoding.
[0028] Alternatively, the encoder (during the motion estimation process) can use a low-complexity filter (such as a 2-tap filter) to identify the best sub-pixel resolution (i.e., the sub-pixel resolution corresponding to the best prediction), and then encode a longer interpolation filter in the bitstream for interpolation at the identified sub-pixel resolution during motion compensation. For example, motion estimation can be performed (e.g., implemented, etc.) using an interpolation filter selected by the encoder when finding a motion vector. In an example, an interpolation filter can be a 2-tap filter, such as the bilateral filter FILTER 1 mentioned below. Subsequently, motion compensation can be performed using the found motion vector. The motion compensation stage may include an interpolation filter search. That is, different candidate (e.g., available) interpolation filters can be tried, and an interpolation filter that results in the best rate-distortion cost can be selected. The selected interpolation filter can be transmitted in the bitstream so that the encoder and decoder can use the same interpolation filter. As further described below, and in order to achieve better coding performance, the taps of the selected interpolation filter are generally equal to or greater than the taps of the interpolation filter used during motion estimation. To this end, the selected filter may be one of a 2-tap filter, an EIGHTTAP, an EIGHTTAP_SHARP, or an EIGHTTAP_SMOOTH filter, which are further described below.
[0029] Additionally, some real-time video applications (such as video conferencing, screen sharing, etc.) may sacrifice quality in order to improve the responsiveness of the application. For example, instead of using a longer interpolation filter, a 2-tap interpolation filter may be used in both motion estimation and motion compensation. An example of a 2-tap filter is a bilateral filter, which uses a 2-tap filter as described above. Figure 7 The triangular kernel described.
[0030] The coding performance using a bilateral filter may not be as optimal as the coding performance that can be achieved using a filter with a higher number of taps. For this reason, it is desirable to use a 2-tap filter that can provide coding performance that is similar to the coding performance of a longer filter and better than the coding performance of a bilateral filter.
[0031] Using interpolation filters that have low complexity (eg, interpolation filters with a smaller number of taps) but provide high prediction accuracy that approximates the prediction accuracy of longer interpolation filters may improve the overall performance of a video codec.
[0032] According to embodiments of the present disclosure, weighted redistributed triangular kernels (eg, filters) may be used to obtain coding performance that approximates the performance of longer interpolation filters. Figure 7The triangular filter further described is a 2-tap convolution filter that calculates a weighted average of two neighboring pixels at a sampling point (i.e., an interpolated pixel value) according to the distance from the sampling point to each of the two neighboring pixels. That is, the weight is inversely proportional to the distance.
[0033] The weight redistributed triangle filter is such that for at least some of the sample points (i.e., sample positions, sub-pixel positions, etc.), the weights are not proportional to (e.g., inversely proportional to) the distance. Using the 2-tap interpolation filter described herein better preserves edges in video frames and / or provides more accurate predictions for at least some video blocks. For a collection of videos with edges, using the filter described herein (i.e., the weight redistributed triangle kernel) may result in a bitrate reduction of 0.5% to 0.8%.
[0034] Figure 1 1 is a diagram of a computing device 100 according to an embodiment of the present disclosure. The computing device 100 may include a communication interface 110, a communication unit 120, a user interface (UI) 130, a processor 140, a memory 150, instructions 160, a power supply 170, or any combination thereof. As used herein, the term "computing device" (e.g., apparatus) includes any unit or combination of units capable of performing any method, process, or any one or more parts thereof disclosed herein.
[0035] The computing device 100 may be a fixed computing device, such as a personal computer (PC), a server, a workstation, a minicomputer, or a mainframe computer; or may be a mobile computing device, such as a mobile phone, a personal digital assistant (PDA), a notebook computer, or a tablet PC. Although shown as a single unit, any one or more elements of the computing device 100 may be integrated into any number of separate physical units. For example, the UI 130 and the processor 140 may be integrated into a first physical unit, while the memory 150 may be integrated into a second physical unit.
[0036] The communication interface 110 may be a wireless antenna as shown, a wired communication port such as an Ethernet port, an infrared port, a serial port, or any other wired or wireless means capable of interfacing with a wired or wireless electronic communication medium 180 .
[0037] The communication unit 120 may be configured to transmit or receive signals via a wired or wireless medium 180. For example, as shown, the communication unit 120 is operatively connected to an antenna configured to communicate via wireless signals. Figure 1Although not explicitly shown in FIG. 1 , the communication unit 120 may be configured to transmit, receive, or both via any wired or wireless communication medium, such as radio frequency (RF), ultraviolet (UV), visible light, optical fiber, metal wire, or a combination thereof. Figure 1 While a single communication unit 120 and a single communication interface 110 are shown, any number of communication units and any number of communication interfaces may be used.
[0038] UI 130 can include any unit that can be engaged with the user, such as virtual or physical keyboard, touch pad, display, touch display, loud speaker, microphone, video camera, sensor or any combination thereof. UI 130 can be operatively coupled with processor or with any other element (such as, power supply 170) of computing device 100 as shown in the figure. Although shown as a single unit, UI 130 can include one or more physical units. For example, UI 130 can include an audio interface for performing audio communication with the user, and a touch display for performing visual and touch-based communication with the user. Although shown as a separate unit, communication interface 110, communication unit 120 and UI 130 or their parts can be configured as a combined unit. For example, communication interface 110, communication unit 120 and UI 130 can be implemented as a communication port that can be engaged with an external touch screen device.
[0039] The processor 140 may include any device or system capable of manipulating or processing signals or other existing or later developed information, including an optical processor, a quantum processor, a molecular processor, or a combination thereof. For example, the processor 140 may include a special purpose processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array, a programmable logic controller, microcode, firmware, any type of integrated circuit (IC), a state machine, or any combination thereof. As used herein, the term "processor" includes a single processor or a plurality of processors. The processor may be operatively coupled to the communication interface 110, the communication unit 120, the UI 130, the memory 150, the instruction 160, the power supply 170, or any combination thereof.
[0040] The memory 150 may include any non-transitory computer-usable or computer-readable medium, such as any tangible device that may, for example, contain, store, communicate, or transport instructions 160 for use by or in conjunction with the processor 140, or any information associated therewith. The non-transitory computer-usable or computer-readable medium may be, for example, a solid-state drive, a memory card, a removable medium, a read-only memory (ROM), a random access memory (RAM), any type of disk (including a hard disk, a floppy disk, an optical disk, a magnetic card, or an optical card), an application specific integrated circuit (ASIC), or any type of non-transitory medium suitable for storing electronic information, or any combination thereof. The memory 150 may be connected to, for example, the processor 140, for example, via, for example, a memory bus (not explicitly shown).
[0041] Instructions 160 may include instructions for performing any method, process, or any one or more parts thereof disclosed herein. Instructions 160 may be implemented in hardware, software, or any combination thereof. For example, instructions 160 may be implemented as information stored in memory 150, such as a computer program, which may be implemented by processor 140 to perform any of the corresponding methods, algorithms, aspects, or combinations thereof as described herein. Instructions 160 or a portion thereof may be implemented as a special-purpose processor or circuit, which may include dedicated hardware for implementing any of the methods, algorithms, aspects, or combinations thereof as described herein. Portions of instructions 160 may be distributed between multiple processors on the same machine or different machines, or distributed over a network such as a local area network, a wide area network, the Internet, or a combination thereof.
[0042] The power supply 170 may be any suitable device for powering the computing device 100. For example, the power supply 170 may include a wired power supply; one or more dry cell batteries, such as nickel cadmium (NiCd), nickel zinc (NiZn), nickel metal hydride (NiMH), lithium ion (Li-ion); a solar cell; a fuel cell; or any other device capable of powering the computing device 100. The communication interface 110, the communication unit 120, the UI 130, the processor 140, the instructions 160, the memory 150, or any combination thereof may be operatively coupled to the power supply 170.
[0043] Although shown as separate elements, the communication interface 110, the communication unit 120, the UI 130, the processor 140, the instructions 160, the power supply 170, the memory 150, or any combination thereof may be integrated into one or more electronic units, circuits, or chips.
[0044] Figure 2is an illustration of a computing and communication system 200 according to an embodiment of the present disclosure. The computing and communication system 200 may include one or more computing and communication devices 100A / 100B / 100C, one or more access points 210A / 210B, one or more networks 220, or a combination thereof. For example, the computing and communication system 200 may be a multi-access system that provides communications such as voice, data, video, messaging, broadcast, or a combination thereof to one or more wired or wireless communication devices such as computing and communication devices 100A / 100B / 100C. Although for simplicity, the computing and communication system 200 may include one or more computing and communication devices 100A / 100B / 100C, one or more access points 210A / 210B, one or more networks 220, or a combination thereof. Figure 2 Three computing and communication devices 100A / 100B / 100C, two access points 210A / 210B, and one network 220 are shown, but any number of computing and communication devices, access points, and networks may be used.
[0045] The computing and communication devices 100A / 100B / 100C may be, for example, computing devices such as Figure 1 . For example, as shown, computing and communication devices 100A / 100B may be user devices, such as mobile computing devices, notebook computers, thin clients, or smart phones, and computing and communication device 100C may be a server, such as a mainframe or cluster. Although computing and communication devices 100A / 100B are described as user devices and computing and communication device 100C is described as a server, any computing and communication device may perform some or all of the functions of a server, some or all of the functions of a user device, or some or all of the functions of a server and a user device.
[0046] Each computing and communication device 100A / 100B / 100C may be configured to perform wired or wireless communications. For example, the computing and communication device 100A / 100B / 100C may be configured to transmit or receive wired or wireless communication signals, and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a cellular phone, a personal computer, a tablet computer, a server, a consumer electronics product, or any similar device. Although each computing and communication device 100A / 100B / 100C is shown as a single unit, the computing and communication device may include any number of interconnected elements.
[0047] Each access point 210A / 210B may be any type of device configured to communicate with computing and communication devices 100A / 100B / 100C, network 220, or both via a wired or wireless communication link 180A / 180B / 180C. For example, access point 210A / 210B may include a base station, a base transceiver station (BTS), a Node B, an enhanced Node B (eNode-B), a home Node B (HNode-B), a wireless router, a wired router, a hub, a repeater, a switch, or any similar wired or wireless device. Although each access point 210A / 210B is shown as a single unit, an access point may also include any number of interconnected elements.
[0048] The network 220 may be any type of network configured to provide services such as voice, data, applications, voice over Internet protocol (VoIP), or any other communication protocol or combination of communication protocols over a wired or wireless communication link. For example, the network 220 may be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a mobile or cellular telephone network, the Internet, or any other electronic communication means. The network may use communication protocols such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), the Internet Protocol (IP), the Real-time Transport Protocol (RTP), the Hypertext Transport Protocol (HTTP), or a combination thereof.
[0049] The computing and communication devices 100A / 100B / 100C may communicate with each other via the network 220 using one or more wired or wireless communication links, or via a combination of wired and wireless communication links. For example, as shown, the computing and communication devices 100A / 100B may communicate via wireless communication links 180A / 180B, and the computing and communication device 100C may communicate via wired communication links 180C. Any of the computing and communication devices 100A / 100B / 100C may communicate using any one or more wired or wireless communication links. For example, a first computing and communication device 100A may communicate via a first access point 210A using a first type of communication link, a second computing and communication device 100B may communicate via a second access point 210B using a second type of communication link, and a third computing and communication device 100C may communicate via a third access point (not shown) using a third type of communication link. Similarly, access points 210A / 210B may communicate with network 220 via one or more types of wired or wireless communication links 230A / 230B. Figure 2The computing and communication devices 100A / 100B / 100C are shown communicating via a network 220, but the computing and communication devices 100A / 100B / 100C may communicate with each other via any number of communication links, such as direct wired or wireless communication links.
[0050] Other implementations of the computing and communication system 200 are possible. For example, in an implementation, the network 220 may be an ad hoc network, and one or more of the access points 210A / 210B may be omitted. The computing and communication system 200 may include Figure 2 For example, computing and communication system 200 may include more communication devices, networks, and access points.
[0051] Figure 3 3 is an illustration of a video stream 300 for encoding and decoding according to an embodiment of the present disclosure. A video stream 300, such as a video stream captured by a video camera or a video stream generated by a computing device, may include a video sequence 310. The video sequence 310 may include a sequence of contiguous frames 320. Although three contiguous frames 320 are shown, the video sequence 310 may include any number of contiguous frames 320. Each frame 330 from the contiguous frames 320 may represent a single image from the video stream. The frame 330 may include a block 340. Although Figure 3 Not shown in the figure, a block may include pixels. For example, a block may include a 16×16 pixel group, an 8×8 pixel group, an 8×16 pixel group, or any other pixel group. Unless otherwise indicated herein, the term "block" may include a superblock, macroblock, segment, slice, or any other portion of a frame. A frame, block, pixel, or combination thereof may include display information, such as brightness information, chrominance information, or any other information that can be used to store, modify, convey, or display a video stream or a portion thereof.
[0052] Figure 4 400 according to an embodiment of the present disclosure. The encoder 400 may be used in a computer system such as Figure 1 The computing device 100 shown in FIG. Figure 2 The computing and communication devices 100A / 100B / 100C shown in FIG. 1 are implemented in the device as, for example, a computer software program stored in a data storage unit (such as, Figure 1 The computer software program may include a computer program that can be used by, for example, Figure 1 The encoder 400 may be implemented as dedicated hardware included in the computing device 100, for example.
[0053] The encoder 400 may process a video stream 402 (such as Figure 3 400) is encoded to generate an encoded (compressed) bitstream 404. In some embodiments, the encoder 400 may include a forward path for generating a compressed bitstream 404. The forward path may include an intra / inter prediction unit 410, a transform unit 420, a quantization unit 430, an entropy coding unit 440, or any combination thereof. In some embodiments, the encoder 400 may include a reconstruction path (indicated by intermittent connecting lines) to reconstruct frames for encoding other blocks. The reconstruction path may include a dequantization unit 450, an inverse transform unit 460, a reconstruction unit 470, a loop filter unit 480, or any combination thereof. Other structural variations of the encoder 400 may be used to encode the video stream 402.
[0054] In order to encode the video stream 402, each frame in the video stream 402 may be processed in units of blocks. Therefore, a current block may be identified from blocks in the frame and the current block may be encoded.
[0055] At the intra / inter prediction unit 410, the current block may be encoded using intra prediction, possibly within a single frame, or inter prediction, possibly between frames. Intra prediction may include generating a prediction block from samples in the current frame that have been previously encoded and reconstructed. Inter prediction may include generating a prediction block from samples in one or more previously constructed reference frames. Generating a prediction block for the current block in the current frame may include performing motion estimation to generate a motion vector indicating an appropriate reference block in a reference frame.
[0056] The intra / inter prediction unit 410 may subtract the prediction block from the current block (original block) to generate a residual block. The transform unit 420 may perform a block-based transform, which may include transforming the residual block into transform coefficients in, for example, the frequency domain. Examples of block-based transforms include Karhunen-Loève transform (KLT), discrete cosine transform (DCT), and singular value decomposition transform (SVD). In an example, the DCT may include transforming the block into the frequency domain. The DCT may include using transform coefficient values based on spatial frequency, with the lowest frequency (i.e., DC) coefficients at the upper left corner of the matrix and the highest frequency coefficients at the lower right corner of the matrix.
[0057] The quantization unit 430 may convert the transform coefficients into discrete quantum values, which may be referred to as quantized transform coefficients or quantization levels. The quantized transform coefficients may be entropy encoded by an entropy coding unit 440 to produce entropy encoded coefficients. Entropy coding may include using a probability distribution metric. The entropy encoded coefficients and information for decoding the block, which may include the prediction type used, motion vectors, and quantizer values, may be output to a compressed bitstream 404. The compressed bitstream 404 may be formatted using various techniques such as run length encoding (RLE) and zero run coding.
[0058] The reconstruction path can be used to maintain the encoder 400 with Figure 5 4. The reconstruction path may be similar to the decoding process discussed below and may include dequantizing the quantized transform coefficients at the dequantization unit 450 and inverse transforming the dequantized transform coefficients at the inverse transform unit 460 to produce a derivative residual block. The reconstruction unit 470 may add the prediction block generated by the intra / inter prediction unit 410 to the derivative residual block to create a reconstructed block. The loop filtering unit 480 may be applied to the reconstructed block to reduce distortion, such as blocking artifacts.
[0059] Other variations of the encoder 400 may be used to encode the compressed bitstream 404. For example, a non-transform based encoder may directly quantize the residual block without the transform unit 420. In some embodiments, the quantization unit 430 and the dequantization unit 450 may be combined into a single unit.
[0060] Figure 5 is a block diagram of a decoder 500 according to an embodiment of the present disclosure. The decoder 500 may be used in a computer system such as Figure 1 The computing device 100 shown in FIG. Figure 2 The computing and communication devices 100A / 100B / 100C shown in FIG. 1 are implemented in the device as, for example, a computer software program stored in a computer such as Figure 1 The computer software program may include a computer program such as a computer program that can be used to store data in the memory 150 shown in FIG. Figure 1 The decoder 500 may be implemented as specialized hardware included in the computing device 100, for example.
[0061] The decoder 500 may receive a signal such as Figure 44 and 5. The decoder 500 may be used to decode the compressed bitstream 502 of the compressed bitstream 404 shown in FIG. 4, and the compressed bitstream 502 may be decoded to generate an output video stream 504. The decoder 500 may include an entropy decoding unit 510, a dequantization unit 520, an inverse transform unit 530, an intra / inter prediction unit 540, a reconstruction unit 550, a loop filtering unit 560, a deblocking filtering unit 570, or any combination thereof. Other structural variations of the decoder 500 may be used to decode the compressed bitstream 502.
[0062] The entropy decoding unit 510 may decode the data elements within the compressed bitstream 502 using, for example, context adaptive binary arithmetic decoding to produce a set of quantized transform coefficients. The dequantization unit 520 may dequantize the quantized transform coefficients, and the inverse transform unit 530 may inverse transform the dequantized transform coefficients to produce a derivative residual block, which may be compared to the derivative residual block. Figure 4 4. The decoded bitstream 502 may correspond to the derivative residual block generated by the inverse transform unit 460 shown in FIG. 4. Using the header information decoded from the compressed bitstream 502, the intra / inter prediction unit 540 may generate a prediction block corresponding to the prediction block created in the encoder 400. At the reconstruction unit 550, the prediction block may be added to the derivative residual block to create a reconstructed block. The loop filtering unit 560 may be applied to the reconstructed block to reduce block artifacts. The deblocking filtering unit 570 may be applied to the reconstructed block to reduce block distortion, and the result may be output as the output video stream 504.
[0063] Other variations of the decoder 500 may be used to decode the compressed bitstream 502. For example, the decoder 500 may generate the output video stream 504 without the deblocking filter unit 570.
[0064] In some embodiments, reducing temporal redundancy may include using similarities between frames to encode frames using a relatively small amount of data based on one or more reference frames, which may be previously encoded, decoded, and reconstructed frames of a video stream. For example, a block or pixel of a current frame may be similar to a spatially corresponding block or pixel of a reference frame. In some embodiments, a block or pixel of a current frame may be similar to a block or pixel of a reference frame at different portions, and reducing temporal redundancy may include generating motion information indicating a spatial difference or transition between a position of a block or pixel in the current frame and a corresponding position of a block or pixel in a reference frame.
[0065] In some embodiments, reducing temporal redundancy may include identifying a block or pixel in a reference frame or a portion of a reference frame that corresponds to a current block or pixel of a current frame. For example, a reference frame or a portion of a reference frame that may be stored in a memory may be searched for the best block or pixel for encoding the current block or pixel of the current frame. For example, the search may identify a block of a reference frame in which the difference in pixel values between the reference block and the current block is minimized, and may be referred to as a motion search. In some embodiments, the portion of the reference frame searched may be limited. For example, the portion of the reference frame searched, which may be referred to as a search area, may include a limited number of rows of the reference frame. In an example, identifying a reference block may include calculating a cost function, such as a sum of absolute differences (SAD), between pixels of a block in the search area and pixels of the current block. In some embodiments, more than one reference frame may be provided. For example, three reference frames may be selected from eight candidate reference frames.
[0066] In some embodiments, the spatial difference between the position of the reference block in the reference frame and the position of the current block in the current frame can be represented as a motion vector. The difference in pixel values between the reference block and the current block can be referred to as differential data, residual data, or a residual block. In some embodiments, generating a motion vector can be referred to as motion estimation, and the pixels of the current block can be indicated as f based on the position using Cartesian coordinates. x,y Similarly, the pixels of the search area of the reference frame may be indicated as r based on their positions using Cartesian coordinates. x,y A motion vector (MV) of a current block may be determined based on, for example, a SAD between pixels of a current frame and corresponding pixels of a reference frame.
[0067] Although described herein with reference to a matrix or Cartesian representation of a frame for clarity, the frame may be stored, transmitted, processed, or any combination thereof in any data structure such that pixel values may be efficiently represented for a frame or image. For example, the frame may be stored, transmitted, processed, or any combination thereof in a two-dimensional data structure such as a matrix or a one-dimensional data structure such as a vector array.
[0068] In some embodiments, for inter-frame prediction, the encoder 400 may convey information of the encoding of the prediction block at the block endpoint, including but not limited to the prediction mode, prediction reference frame, motion vector (if necessary), and sub-pixel interpolation filter type.
[0069] Figure 6 is a block diagram of a representation of inter-frame prediction using sub-pixel interpolation filters according to an embodiment of the present disclosure. In some embodiments, the Figure 5 The decoder 500 shown in FIG. Figure 4The encoder 400 shown in FIG. 4 or a combination thereof implements inter-frame prediction using sub-pixel interpolation filters.
[0070] In some embodiments, inter-frame prediction may include encoding the current frame 610 or a portion thereof, such as block 612 or block 614, with reference to the reference frame 620. For example, the motion vector 632 / 634 may indicate the spatial position of the reference block 622 / 624 in the reference frame 620 relative to the position of the corresponding block 612 / 614 in the current frame 610, the reference block 622 / 624 may be the portion of the reference frame 620 identified as the most accurate match to the corresponding block 612 / 614 in the current frame 610, and the reference block 622 / 624 may be used to generate a prediction block for encoding the current frame 610.
[0071] In some embodiments, the portion 622 of the reference frame 620 that most accurately matches the current block (e.g., block 614) of the current frame 610 may deviate from a block or pixel boundary, and inter-frame prediction may include the use of a sub-pixel interpolation filter. For example, inter-frame prediction may include the use of a motion vector accuracy of 1 / 2 pel, 1 / 4 pel, or 1 / 8 pel, and a sub-pixel interpolation filter may be used.
[0072] In some embodiments, coding the current block may include: identifying a prediction mode for coding the current block from a plurality of candidate prediction modes. For example, the video coder may evaluate candidate prediction modes that may include intra-prediction modes, inter-prediction modes, or both to identify an optimal prediction mode, which may be, for example, a prediction mode that minimizes an error metric such as a rate-distortion cost for the current block.
[0073] In some embodiments, the inter-frame prediction mode may include a new motion vector mode (NewMV), a zero motion vector mode (ZeroMV), a nearest motion vector mode (NearestMV), and a near motion vector mode (NearMV). The new motion vector mode (NewMV) may indicate that a new motion vector of the current block may be explicitly or explicitly signaled in the encoded video stream. In some embodiments, the new motion vector may be signaled in different ways. For example, the new motion vector may be signaled using the difference between the new motion vector and a reference motion vector, such as a motion vector used to encode a previously coded neighboring block. The zero motion vector mode (ZeroMV) may indicate that a zero motion vector (0,0) that may indicate no motion may be used to predict the current block, and an explicit or explicit motion vector for predicting the current block may be omitted from the encoded video stream. In some embodiments, the nearest motion vector mode (NearestMV) may indicate that a motion vector for encoding a previously encoded neighboring block identified as the nearest motion vector may be used to predict the current block, and an explicit or explicit motion vector for predicting the current block may be omitted from the encoded video stream. In some embodiments, the near motion vector mode (NearMV) may indicate that a motion vector identified as a near motion vector used to encode a previously encoded neighboring block may be used to predict the current block, and an explicit or explicit motion vector used to predict the current block may be omitted from the encoded video stream.
[0074] In some embodiments, the encoder may identify candidate motion vectors for encoding the current block. For example, the candidate motion vectors may include a zero motion vector, a new motion vector, a near motion vector, and a nearest motion vector. In some embodiments, the encoder may evaluate adjacent or proximal, previously encoded blocks to identify the near motion vector and the nearest motion vector. For example, the near motion vector may be identified from a first adjacent or proximal, previously encoded block, and the nearest motion vector may be identified from a second adjacent or proximal, previously encoded block.
[0075] In some embodiments, the decoder may identify candidate motion vectors for decoding the current block. For example, the candidate motion vectors may include a near motion vector and a most recent motion vector. In some embodiments, the decoder may evaluate adjacent or immediately preceding, previously decoded blocks to identify the near motion vector and the most recent motion vector. For example, the near motion vector may be identified from a first adjacent or immediately preceding, previously decoded block, and the most recent motion vector may be identified from a second adjacent or immediately preceding, previously decoded block.
[0076] In some embodiments, coding the current block may include identifying an interpolation filter for predicting the current block. For example, the interpolation filter may be selected from candidate interpolation filters. In some embodiments, the candidate interpolation filter may be a filter with 1 / 16 pel accuracy and may include a bilinear filter, an 8-tap filter (EIGHTTAP), a sharp 8-tap filter (EIGHTTAP_SHARP), a smooth 8-tap filter (EIGHTTAP_SMOOTH), any other filter described herein (e.g., Filter 2 and Filter 3 described below), or a combination thereof.
[0077] In practice, as will be appreciated by those skilled in the art, interpolation filtering may be separated into two 1-dimensional filtering paths. For example, a first path may perform interpolation filtering in the x-direction; and a second path performs interpolation filtering in the y-direction. However, to simplify explanation, interpolation filtering is described herein only with respect to 1-dimensional filtering.
[0078] Figure 7 700 is a diagram of a bilateral filter according to an embodiment of the present disclosure. In an example, an encoder may use a bilateral filter during a motion estimation stage, such as with respect to Figure 4 In the example, and to ensure fast encoding and decoding, Figure 4 The intra / inter prediction unit 410 and the reconstruction unit 470 and Figure 5 The reconstruction unit 550 may exclusively use a 2-tap bilinear filter. The bilateral filter is referred to herein as filter 1.
[0079] Graph 710 illustrates the operation of a bilateral filter (i.e., filter 1). As shown in graph 710, P0 and P1 are pixel values of two consecutive pixels. In an example, pixel values P0 and P1 are in the same row (or equivalently, the same column) of pixels (at integer or sub-pel positions) of a video block. For example, pixel value P0 may be at horizontal position x0 of a pixel row; and pixel value P1 may be at horizontal position x1 of a pixel row. A sub-pixel value P at position x will be calculated (i.e., interpolated) based on pixel values P0 and P1. Sub-pixel value P may be calculated using formula (1).
[0080] P=P0*D(x,x1)+P1*D(x,x0) (1)
[0081] Formula (1) can be described as a weighted sum of two neighboring pixels. The weights (i.e., D(x, x0) and D(x, x1)) are the normalized distances between position x and positions x0 and x1, respectively. For example, if x
[0082] Halfway between x0 and x1, the weights may be D(x, x0) = D(x, x1) = 0.5. For example, if x is 1 / 4 of the distance between x0 and x1, and is closer to x0 than x1, the weights may be D(x, x0) = 0.25 and D(x, x1) = 0.75. That is, the distance (i.e., the weight) indicates the amount (i.e., how much) a pixel will contribute to the interpolated value.
[0083] Equation (1) can be rewritten in the general form of Equation 2, where F0 and F1 are the filter coefficients:
[0084] P=(P0*F0)+(P1*F1) (2)
[0085] As mentioned above, a bilinear interpolation filter can be designed based on the corresponding pixel distance. Assuming the motion vector accuracy is 1 / 16 pixel, the coefficients of the bilateral filter (ie, filter 1) can be as shown in Table I.
[0086] Table I – Filter 1 weights
[0087]
[0088] It should be noted that because it is desirable to perform integer operations during the coding operation, the weights of filter 1 are expressed as integers using 7-bit precision. Other integer precisions may also be used. To illustrate the use of a bilateral filter, assume that two consecutive pixel values are P0=220 and P1=63. The sub-pixel at the 3 / 8 pixel position can be obtained by P(3 / 8 pixel)=(220*80+48*63)>>7, where ">>" is a right shift operation. For this purpose, P(3 / 8)=161. As described with respect to Table I, those skilled in the art can easily determine the weights of a bilateral filter for a sub-pixel precision other than 1 / 16.
[0089] The purpose of interpolation is to estimate unknown sub-pixels from neighboring pixels. Interpolation attempts to reconstruct the original continuous image from the sampled image at integer pixel locations. Reconstruction is a convolution process using a defined kernel. For example, the above bilinear interpolation filter uses a triangular kernel. A triangular filter is shown in triangular kernel 720.
[0090] The triangular kernel 720 acts as a linear interpolation filter by taking a weighted average of two adjacent values based on the distance of the sample to the two adjacent values. The convolution operation can be implemented as shown in the following example. The triangular kernel 722 is centered directly above the full pixel position x0, and the peak of the triangular kernel 722 is the pixel value P0. The triangular kernel 722 shows the contribution of the pixel value P0 to the final interpolated pixel value at the sub-pixel position. Similarly, the triangular kernel 724 is centered directly above the full pixel position x1, and the peak of the triangular kernel 724 is the pixel value P1. The triangular kernel 724 shows the contribution of the pixel value P1 to the final interpolated pixel value at the sub-pixel position. For example, as Figure 7 As shown in , to calculate the 3 / pixel between the sample at position x, i.e., x0 (with pixel value P0) and x1 (with pixel value P1), the contribution of P0 (i.e., P0x) and the contribution of P1 (i.e., P1x) are added together, where P0x=P0*W0 and P1x=P1*W1. The weights to be used are W0=0.25 and W1=0.75 (or equivalently, 32 and 96 using 7-bit integers). For this reason, the convolution operation results in P=0.25P0+0.75P1.
[0091] Drawing 730 depicts the weights of the triangular kernel 720 of the above-mentioned bilateral filter (i.e., filter 1). As shown in drawing 730, the weights of the triangular kernel 720 form an isosceles triangle. To this end, the bilateral filter can be characterized as a 2-tap filter having a set of weight tuples (see Table 1); and each tuple includes two weights and is used to perform sub-pixel interpolation at corresponding sub-pixel positions between a first pixel and a second pixel. For each tuple in the weight tuple, the two weights are based on a first distance between the sub-pixel position and the first pixel and a second distance between the sub-pixel position and the second distance. In other words, the drawing of the weight tuples of the bilateral filter forms a triangle (e.g., an isosceles triangle). The first pixel can be at an integer pixel position or a sub-pixel position. The second pixel can be at an integer pixel position or a sub-pixel position.
[0092] Graph 740 illustrates recovering a continuous portion 742 of an image (i.e., a frame, a block of a frame, etc.) given pixel values at four integer positions x0 to x3. Kernels 744 to 750 illustrate triangular kernels used to derive (i.e., calculate, obtain, etc.) the continuous portion 742 from the pixel values at four integer positions x0 to x3. It should be noted that only one side of kernel 748 and only one side of kernel 750 are shown in graph 740 because other required pixel values (i.e., pixels before x0 and pixels after x3) are not shown in graph 740.
[0093] As already mentioned, the kernel can provide an answer to the following question: How many pixel values at a certain position are used in the interpolated pixel value? For example, and as can be seen in graph 740, at x1, the pixel weight is 1 (indicated by the alignment of the center of kernel 744 with pixel position x1). The weights for x0 and x2 are zero. That is, the pixel value is that positions x0 and x2 do not contribute any of their corresponding values to x1. Similarly, at x2, the pixel weight is 1 (indicated by the alignment of the center of kernel 746 with pixel position x2). For example, for sub-pixel values at positions between x0 and x1, the corresponding values obtained along kernels 744 and 748 can be added to obtain the pixel value on continuous portion 742.
[0094] It is desirable to use a kernel that provides a close approximation of an ideal low-pass filter in the frequency domain. An ideal low-pass filter can be described as a filter that completely eliminates all frequencies above a cutoff frequency while passing those frequencies below the cutoff frequency unchanged. That is, an ideal low-pass filter restores only those frequencies that were in the original image.
[0095] Although the above bilateral filter (i.e., Filter 1) is a low complexity filter, it does not adequately approximate an ideal low-pass filter in the frequency domain. The Fourier transform of the triangular kernel is sinc 2 Function. Sinc 2 The function includes many side lobes and, therefore, introduces high frequencies in the filtered result (ie, the interpolated value).For this reason, a triangular kernel (eg, a bilateral filter) as a 1st order filter does not provide as good results as a higher order kernel.
[0096] Figure 8 A 2-tap filter kernel similar to a high-order kernel according to an embodiment of the present disclosure is illustrated. Filter kernel 810 and filter kernel 820, referred to as filter 2 and filter 3, respectively, are described. Each of filter kernels 810 and 820 defines a 2-tap filter. Therefore, each of filter kernels 810 and 820 can only use two consecutive pixels in the interpolation operation. Similar to a high-order kernel may mean exhibiting a similar response in the frequency domain.
[0097] As further described below, each of the filter kernels 810 and 820 can be characterized as a 2-tap filter having a set of weight tuples (see Tables II and III). Each tuple includes two weights and is used for sub-pixel interpolation at corresponding sub-pixel positions between a first pixel and a second pixel. For at least some of the weight tuples, the two weights are not based on a first distance between the sub-pixel position and the first pixel and / or a second distance between the sub-pixel position and the second distance. In other words, the drawing of the weight tuple of filter 2 (i.e., filter kernel 810) does not form a triangle (e.g., an isosceles triangle, such as described with respect to drawing 730); and the drawing of the weight tuple of filter 3 (i.e., filter kernel 820) does not form a triangle. That is, the sides of each of the filter kernels 810 and 820 include a bend and are not straight lines. The first pixel can be at an integer pixel position or a sub-pixel position. The second pixel can be at an integer pixel position or a sub-pixel position.
[0098] The weights of at least some of the tuples of filter kernels 810 and 820 are not distributed based on the distance between the corresponding sub-pixel position and the pixel position used in the interpolation operation.
[0099] Table II lists the weight tuples for the filter kernel 810 (Filter 2) at a given sub-pixel position. Table III lists the weight tuples for the filter kernel 820 (Filter 3) at a given sub-pixel position. Similar to Table I, the weights for Filter 2 and Filter 3 are represented as integers using 7-bit precision.
[0100] Table II – Filter 2 weights
[0101]
[0102] Table III – Filter 3 weights
[0103]
[0104] Filter 2 and Filter 3 (illustrated by filter kernel 810 and filter kernel 820, respectively) are weight redistributed triangular kernels. Filter 2 and Filter 3 (i.e., weight redistributed triangular kernels) are so called because, while the overall shape may resemble a triangle, the weights are redistributed so that for at least some of the weight tuples, a higher weight is assigned to the pixel than the weight assigned to the closer pixel by Filter 1. It should be understood that with respect to Tables I to III, X0 and X1 refer to the positions of the two pixel values being interpolated. The sub-pixel positions are given with respect to the first pixel position (i.e., X0).
[0105] As an illustrative example of weight redistribution, and using 1 / 4 pixel resolution, given that filter 1 uses the weight tuple (96, 32), filter 2 uses (100, 28), thereby assigning higher weights (i.e., 100 is higher than 96) to closer pixels (i.e., x0); and filter 3 uses (102, 26), thereby assigning higher weights (i.e., 102 is higher than 96) to closer pixels (i.e., x0). As another example of weight redistribution, and using 3 / 4 pixel resolution, given that filter 1 uses the weight tuple (32, 96), filter 2 uses (28, 100), thereby assigning higher weights (i.e., 100 is higher than 96) to closer pixels (i.e., x1); and filter 3 uses (26, 102), thereby assigning higher weights (i.e., 102 is higher than 96) to closer pixels (i.e., x1).
[0106] The frequency of the Fourier transform of filter 2 is not as high as that of the triangular kernel (i.e., filter 1). For this reason, filter 2 can be used to achieve better coding performance than the triangular kernel. In order to design a filter similar to filter 2 (e.g., a weight-redistributed triangular filter), a relatively high cutoff frequency (C) can be selected. The cutoff frequency can be 0.5, 0.75, or other cutoff frequencies. A cutoff frequency C=0.75 was used in the design of filter 2. Filter 2 provides a "sharper" response than the bilateral filter (i.e., filter 1). For this reason, similar to the sharp interpolation filter (EIGHTTAP_SHARP), filter 2 can preserve edges in video frames better than bilateral filters. Filter 3 is a Gaussian-like kernel. Because the Fourier transform of a Gaussian filter is Gaussian, filter 3 can be considered a good low-pass filter. Although weight redistribution techniques are described with respect to filters 2 and 3, other techniques can also be used to obtain a 2-tap filter with weights, and the weights, unlike filter 1, are not based on pixel distances at least for some of the sub-pixel positions.
[0107] Fig. 9 is a flowchart of an example of a process 900 for encoding a block of a video using inter-frame prediction according to an embodiment of the present disclosure. The block is encoded using inter-frame prediction. In some embodiments, the block may be encoded in a manner such as Figure 4 The encoder 400 shown in FIG. 4 is implemented in an encoder to encode blocks of a video using inter-frame prediction.
[0108] In an example, and at a high level, a reference block can be identified in a reference frame based on a motion vector, and an interpolation filter can be used to generate a prediction block from the reference block. In some embodiments, the residual or prediction error between the block and the prediction block can be included in a Figure 4The compressed bitstream 404 may be included in the compressed bitstream of the compressed bitstream 404. In some embodiments, the motion vector and the identifier of the interpolation filter used to code the block may be included in the compressed bitstream, such as in the header of the first block. In some embodiments, the motion vector used to code the block may be associated with the interpolation filter used to code the block.
[0109] In the example, and as is well known, inter-frame prediction in an encoder may require two separate processes: a motion estimation process and a motion compensation process. The motion estimation process may be performed by, for example, Figure 4 The prediction unit of the intra / inter prediction unit 410 is performed. Figure 4 The reconstruction unit 470 of the reconstruction unit is used to perform the motion compensation process.
[0110] During the motion estimation process, a motion vector for the block may be determined. In some embodiments, encoding the block may include identifying a motion vector for encoding the block. In some embodiments, the motion vector may be identified to encode the block, and an interpolation filter may be identified to encode the block. In an example, the interpolation filter may be identified when the motion vector has sub-pixel accuracy. To this end, samples (i.e., pixel values) at integer pixel positions may be used to interpolate samples (i.e., pixel values) at fractional positions.
[0111] Interpolation filters are used during the interpolation process. There may be several filters available. In an example, the interpolation filters available during motion estimation (i.e., candidate interpolation filters) may include the above-mentioned EIGHTTAP_SMOOTH, EIGHTTAP_SHARP, EIGHTTAP, filter 1, filter 2, and filter 3. In another example, there may be more, fewer, or other filters available.
[0112] In some embodiments, an interpolation filter for encoding a block may be identified based on a motion vector identified for encoding the block. In some embodiments, identifying the interpolation filter may include evaluating candidate interpolation filters to identify an optimal interpolation filter. In an example, each of the available candidate interpolation filters may be evaluated to determine an optimal interpolation filter. The optimal interpolation filter may be the filter that results in the minimum residual error between the block and the predicted block.
[0113] In another example, only some of the candidate interpolation filters are evaluated. As described above, EIGHTTAP_SMOOTH, EIGHTTAP_SHARP, and EIGHTTAP are 8-tap filters; whereas filter 1, filter 2, and filter 3 are 2-tap filters. For some applications (e.g., video conferencing, screen sharing, etc.), only 2-tap filters are evaluated. This can increase compression speed, even at the expense of lower quality compression. More generally, shorter filters are evaluated during the motion estimation process. To this end, the identifier of the selected interpolation filter is included in the compressed bitstream.
[0114] In some encoding examples, a fast search motion estimation technique may be used. Such a fast motion estimation technique may include using a shorter filter (e.g., a 2-tap filter, such as filter 1, filter 2, and filter 3) during the motion estimation stage, but using a corresponding longer filter during motion compensation, as further described below.
[0115] Using one of Filter 1, Filter 2, and Filter 3 may include identifying a desired sub-pixel accuracy of the motion vector and using a corresponding weight tuple. For example, if the desired sub-pixel accuracy is 3 / 8 pixel, using Filter 2 as an interpolation filter includes selecting a weight tuple (82, 46) as shown in Table II for the interpolation operation.
[0116] In the example, there may be a corresponding relationship between the shorter filter and the longer filter. That is, the shorter filter may have a frequency response similar to or approximate to the corresponding longer filter. For this reason, when a shorter filter is selected during motion estimation, the corresponding longer filter can be encoded (e.g., included) in the compressed bitstream so that the decoder can use the longer filter for decoding. In another example, the corresponding relationship may not be based on frequency response similarity. Interpolation filter encoding in compressed bitstream can mean that the indication, identifier, etc. of the interpolation filter are encoded in the compressed bitstream. In the example, if filter 2 is selected during the motion estimation stage, the EIGHTTAP_SHARP filter can be encoded in the compressed bitstream. In the example, if filter 3 is selected during the motion estimation stage, and even if the corresponding relationship between filter 3 and EIGHTTAP_SMOOTH may be weaker than the corresponding relationship between filter 2 and the EIGHTTAP_SHARP filter, the EIGHTTAP_SMOOTH filter can be encoded in the compressed bitstream. In an example, if filter 1 is selected during the motion estimation stage, the EIGHTTAP filter may be encoded in the compressed bitstream. Other correspondences between shorter filters and longer filters are also possible.
[0117] During the motion compensation stage, motion vectors, interpolation filters and residuals are decoded from the compressed bitstream and used to reconstruct the block.
[0118] Returning to process 900, at 910, process 900 selects a first subpixel interpolation filter. As described above, the first subpixel interpolation filter may be selected during motion estimation for a block. The first subpixel interpolation filter may be used for subpixel interpolation of fractional motion (ie, at subpixel locations).
[0119] The first sub-pixel interpolation filter may be a 2-tap filter. To this end, the first sub-pixel interpolation filter may be one of the above-mentioned filters 2 or 3, or any other weight redistribution triangular kernel that exhibits the characteristics described herein. As further described above, such as in Table II and Table III, the first sub-pixel interpolation filter includes a weight tuple. Each weight tuple includes two weights and is used for sub-pixel interpolation at a corresponding sub-pixel position between a first pixel and a second pixel. In an example, the first pixel and the second pixel may be pixels at integer pixel positions. In an example, at least one of the first pixel or the second pixel may be at a sub-pixel position. That is, at least one of the first pixel or the second pixel itself may be an interpolated pixel. In an example, interpolation may be divided into multiple stages. To this end, for example, a first interpolation operation interpolates a 1 / 2 pixel value from a full pixel adjacent to a pixel, and a second interpolation operation further interpolates other sub-pixel values using the 1 / 2 pixel value.
[0120] As mentioned, the first sub-pixel interpolation filter may be Filter2. To this end, and using 7-bit precision, and as described in Table II, the weight tuples include tuples (128,0), (122,6), (116,12), (108,20), (100,28), (92,36), (82,46), (74,54), (64,64), (54,74), (46,82), (36,92), (28,100), (20,108), (12,116), and (6,122) for sub-pixel precision for full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel, and 15 / 16 pixel, respectively.
[0121] As mentioned, the first sub-pixel interpolation filter may be Filter3. To this end, and using 7-bit precision, and as described in Table III, the weight tuples include tuples (128,0), (124,4), (118,10), (110,18), (102,26), (94,34), (84,44), (74,54), (64,64), (54,74), (44,84), (34,94), (26,102), (18,110), (10,118), and (4,124) for corresponding sub-pixel precision for full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel, and 15 / 16 pixel.
[0122] As mentioned, and as regards Figure 7 As described, the weight tuples for the bilateral (i.e., filter 1) are such that the weights are inversely proportional to the distance between the interpolation position (i.e., the sub-pixel position) and the position of the pixel used in the interpolation. In contrast, the first sub-pixel interpolation is characterized in that, for at least one of the weight tuples, the two weights are not based on a first distance between the sub-pixel position and the first pixel and a second distance between the sub-pixel position and the second pixel.
[0123] In an example, and as described above, the first sub-pixel interpolation may cause the weights to be redistributed so that, for at least some of the weight tuples, a higher weight is assigned to the pixel than a weight assigned to a closer pixel by filter 1. In other words, where the sub-pixel position is closer to the first pixel than the second pixel, for at least one of the weight tuples, a first weight of the one weight tuple is higher than the bilateral weight based on the second distance, the first weight being used for the first pixel.
[0124] In an example, selecting the first subpixel interpolation filter may include selecting a motion vector having a fractional resolution during motion estimation for the block, and determining interpolated values at the fractional resolution using the first subpixel interpolation filter to generate the predicted block.
[0125] At 920, process 900 selects a second sub-pixel interpolation filter. The second sub-pixel interpolation filter may be based on the first sub-pixel interpolation filter. In an example, the second sub-pixel interpolation filter may be used during a motion compensation process, such as by a reconstruction unit of an encoder. In an example, the second sub-pixel interpolation filter may be the first sub-pixel interpolation filter. In an example, the second sub-pixel interpolation filter may be selected based on a rate-distortion cost. For example, the second sub-pixel interpolation filter may be one of the longer interpolation filters that results in a minimum rate-distortion cost. In an example, the second sub-pixel interpolation filter may be selected based on a frequency response of the first sub-pixel interpolation filter. For example, the second sub-pixel interpolation filter may be selected based on a correspondence between the first interpolation filter and the second interpolation filter.
[0126] At 930, process 900 encodes the second subpixel interpolation filter in the compressed bitstream. For example, process 900 can encode an identifier of the second subpixel interpolation filter.
[0127] Fig.10 is a flowchart of an example of a process 1000 for decoding a block of a video using inter-frame prediction according to an embodiment of the present disclosure. In some embodiments, the Figure 5 Decoding a block of a video using inter-frame prediction is implemented in a decoder of the decoder 500 shown in FIG. 1 . In some embodiments, the decoding of a block of a video using inter-frame prediction can be implemented at least in part in an encoder, such as by Figure 4 The reconstruction unit 470 of the encoder enables decoding of blocks of video using inter-frame prediction.
[0128] Decoding the block may include decoding a motion vector used to decode the block from a compressed bitstream. In some embodiments, decoding the block may include decoding a motion vector and an identifier of an interpolation filter, such as a sub-pixel interpolation filter used to decode the block, from the compressed bitstream. In some embodiments, decoding the block may include decoding a residual or prediction error of the block from the compressed bitstream. In some embodiments, decoding the block may include: identifying a reference block in a reference frame based on the decoded motion vector; generating a predicted block from the reference block using an interpolation filter indicated by a decoded interpolation filter identifier; and generating a reconstructed block by adding the decoded residual to the predicted block. In some embodiments, the motion vector, the identifier of the interpolation filter, or a combination thereof may be decoded from a header of the block in the compressed bitstream. In some embodiments, the reconstructed block may be included in an output, such as an output for presentation. The compressed bitstream may be as described with respect to Figure 5The compressed bit stream 502 of FIG. 502 may be described as being compressed by a reconstruction path (such as Figure 4 The reconstruction path described by the received bit stream is described by the reconstruction path described by the received bit stream.
[0129] At 1010, process 1000 decodes a subpixel interpolation filter from a compressed bitstream. The subpixel interpolation filter can be used for subpixel interpolation of fractional motion. The subpixel interpolation filter can be a 2-tap filter. The subpixel interpolation filter includes weight tuples. Each tuple includes two weights and is used for subpixel interpolation at a subpixel position between a first pixel and a second pixel. For at least one weight tuple in the weight tuple, the two weights are not based on a first distance between the subpixel position and the first pixel and a second distance between the subpixel position and the second pixel. This is in contrast to the weight tuple of the bilateral filter (Filter 1).
[0130] In an example, and as described above, sub-pixel interpolation can cause weights to be redistributed so that, for at least some of the weight tuples, a higher weight is assigned to the pixel than a weight assigned to a closer pixel by filter 1. In other words, where a sub-pixel position is closer to a first pixel than a second pixel, for at least one of the weight tuples, a first weight of the one weight tuple is higher than the bilateral weight based on the second distance, wherein the first weight is for the first pixel.
[0131] The sub-pixel interpolation filter can be filter 2. To this end, and using 7-bit precision, and as described in Table II, the weight tuples include tuples (128,0), (122,6), (116,12), (108,20), (100,28), (92,36), (82,46), (74,54), (64,64), (54,74), (46,82), (36,92), (28,100), (20,108), (12,116), and (6,122) for sub-pixel precision for full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel, and 15 / 16 pixel, respectively.
[0132] The sub-pixel interpolation filter may be filter 3. To this end, and using 7-bit precision, and as described in Table III, the weight tuples include tuples (128,0), (124,4), (118,10), (110,18), (102,26), (94,34), (84,44), (74,54), (64,64), (54,74), (44,84), (34,94), (26,102), (18,110), (10,118), and (4,124) for corresponding sub-pixel precision for full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel, and 15 / 16 pixel.
[0133] At 1020, process 1000 decodes the motion vector from the compressed bitstream. At 1030, process 1000 decodes the block using the sub-pixel interpolation filter and the motion vector. For example, decoding the motion vector may include identifying a reference frame. Using the reference frame, the motion index, and the sub-pixel interpolation filter, a prediction block may be generated. Additionally, a residual may be decoded from the compressed bitstream. The residual may be added to the prediction block to reconstruct the block.
[0134] The words "example" or "exemplary" are used herein to mean serving as an example, instance or illustration. Any aspect or design described herein as "example" or "exemplary" is not necessarily to be interpreted as being more preferred or advantageous than other aspects or designs. Rather, the use of the words "example" or "exemplary" is intended to present concepts in a specific style. As used in this application, the term "or" is intended to represent an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clearly seen from the context, "X includes A or B" is intended to represent any one of the natural inclusive arrangements. That is, if X includes A; X includes B; or X includes both A and B, "X includes A or B" is satisfied under any of the above examples. In addition, the articles "one" and "an" as used in this application and the appended claims should generally be interpreted as meaning "one or more", unless otherwise specified or clearly pointed to the singular form from the context. In addition, unless so described, the use of the terms "embodiment" or "one embodiment" or "implementation" or "one implementation" throughout the text is not intended to represent the same embodiment or implementation. As used herein, the terms "determine" and "identify" or any variation thereof include the use of Figure 1 One or more of the devices shown in the are selected, ascertained, calculated, searched, received, determined, established, obtained, or otherwise identified or determined in any manner.
[0135] In addition, for simplicity of explanation, although the drawings and descriptions herein may include a sequence or series of steps or stages, the elements of the methods or processes disclosed herein may occur in various orders and / or simultaneously. In addition, the elements of the methods disclosed herein may occur with other elements that are not clearly presented and described herein. In addition, all elements of the methods described herein may not be required to implement the methods or processes according to the disclosed subject matter.
[0136] Implementations of a transmission station (such as computing and communication device 100A) and / or a receiving station (such as computing and communication device 100B), and the algorithms, methods, instructions, executable instructions, etc. stored thereon and / or implemented thereby, may be implemented in hardware, software, or any combination thereof. The hardware may include, for example, a computer, an intellectual property (IP) core, an application specific integrated circuit (ASIC), a programmable logic array, an optical processor, a programmable logic controller, microcode, a microcontroller, a server, a microprocessor, a digital signal processor, or any other suitable circuit. In the claims, the term "processor" should be understood to cover any of the aforementioned hardware, either individually or in combination. The terms "signal" and "data" are used interchangeably. In addition, the parts of the transmission station and the receiving station do not necessarily have to be implemented in the same manner.
[0137] Furthermore, in one embodiment, for example, a transmitting station or a receiving station may be implemented using a computer program that, when executed, implements any of the corresponding methods, algorithms, and / or instructions described herein. Additionally or alternatively, for example, a special purpose computer / processor may be utilized that may include specialized hardware for implementing any of the methods, algorithms, or instructions described herein.
[0138] Transmission station and receiving station can be realized on the computer in real-time video system for example.Alternatively, transmission station can be realized on server, and receiving station can be realized on the equipment (such as, handheld communication device) separated from server.In this example, transmission station can use encoder 400 to encode content into the video signal of encoding, and transmit the video signal of encoding to communication device.Then, communication device can use decoder 500 to decode the video signal of encoding then.Alternatively, communication device can decode the content stored locally on communication device, for example, the content not transmitted by transmission station.Other suitable transmission station embodiments and receiving station embodiments are also available.For example, receiving station may be usually fixed personal computer rather than portable communication device, and / or the equipment comprising encoder 400 may also comprise decoder 500.
[0139] In addition, all or part of the embodiments may take the form of a computer program product accessible from, for example, a tangible computer usable or computer readable medium. A computer usable or computer readable medium may be, for example, any device that can tangibly contain, store, communicate, or transport a program for use by or in conjunction with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media may also be used.
[0140] The above embodiments have been described to allow for easy understanding of the present application and are not intended to be limiting. On the contrary, the present application covers various modifications and equivalent arrangements included within the scope of the appended claims, which scope should be given the broadest interpretation to cover all such modifications and equivalent structures as permitted by law.
Claims
1. A method for encoding a block of a video using inter-frame prediction, the method comprising: selecting, during motion estimation for the block, a first subpixel interpolation filter for subpixel interpolation of fractional motion; Wherein, the first sub-pixel interpolation filter is a 2-tap filter, wherein the first sub-pixel interpolation filter comprises weight tuples, each weight tuple consisting of two weights and used for sub-pixel interpolation at a sub-pixel position between a first pixel and a second pixel, the sub-pixel position corresponding to a sub-pixel precision, wherein, for at least one of the weight tuples, the two weights are not based on a first distance between the sub-pixel position and the first pixel and a second distance between the sub-pixel position and the second pixel, wherein the first pixel and the second pixel are located at integer pixel positions, The sub-pixel position is located in the same row or column between the first pixel and the second pixel, and wherein the first sub-pixel interpolation filter is designed as a low-pass filter to pass less high frequencies than a triangular filter; During motion compensation, selecting a second subpixel interpolation filter based on the first subpixel interpolation filter; and The second sub-pixel interpolation filter is encoded in the compressed bitstream.
2. The method according to claim 1, wherein: In a case where the sub-pixel position is closer to the first pixel than to the second pixel, for at least one of the weight tuples, a first weight of the one weight tuple is higher than a bilateral weight based on the second distance, wherein the first weight is used for the first pixel.
3. The method according to claim 1, wherein: Selecting the first sub-pixel interpolation filter for sub-pixel interpolation of the fractional motion during the motion estimation for the block comprises: During said motion estimation for said block, selecting a motion vector having a fractional resolution; and Interpolated values at the fractional resolution are determined using the first sub-pixel interpolation filter to generate a prediction block.
4. The method according to claim 1, wherein: The second sub-pixel interpolation filter is the first sub-pixel interpolation filter.
5. The method according to claim 1, wherein: The second sub-pixel interpolation filter is selected based on a frequency response of the first sub-pixel interpolation filter.
6. The method according to claim 1, wherein: The second sub-pixel interpolation filter is selected based on a correspondence between the first sub-pixel interpolation filter and the second sub-pixel interpolation filter.
7. The method according to claim 1, wherein: In the case of 7-bit precision, the weight tuple includes (128,0), (122,6), (116,12), (108,20), (100,28), (92,36), (82,46), (74,54), (64,64), (54,74), (46,82), (36,92), (28,100), (20,108), (12,116) and (6,122) for corresponding sub-pixel precisions of full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel and 15 / 16 pixel.
8. The method according to claim 1, wherein: In the case of 7-bit precision, the weight tuple includes (128,0), (124,4), (118,10), (110,18), (102,26), (94,34), (84,44), (74,54), (64,64), (54,74), (44,84), (34,94), (26,102), (18,110), (10,118) and (4,124) for corresponding sub-pixel precisions of full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel and 15 / 16 pixel.
9. An apparatus for encoding a block of a video using inter-frame prediction, the apparatus comprising: A processor configured to: During motion estimation for the block, selecting a first sub-pixel interpolation filter for sub-pixel interpolation of fractional motion, Wherein, the first sub-pixel interpolation filter is a 2-tap filter, wherein the first sub-pixel interpolation filter comprises weight tuples, each weight tuple consisting of two weights and used for sub-pixel interpolation at a sub-pixel position between a first pixel and a second pixel, the sub-pixel position corresponding to a sub-pixel precision, wherein, for at least one of the weight tuples, the two weights are not based on a first distance between the sub-pixel position and the first pixel and a second distance between the sub-pixel position and the second pixel, wherein the first pixel and the second pixel are located at integer pixel positions, The sub-pixel position is located in the same row or column between the first pixel and the second pixel, and wherein the first sub-pixel interpolation filter is designed as a low-pass filter to pass less high frequencies than a triangular filter; selecting a second sub-pixel interpolation filter based on a rate-distortion cost; and The second sub-pixel interpolation filter is encoded in the compressed bitstream.
10. The device according to claim 9, wherein: The processor is configured to: The second sub-pixel interpolation filter is used during motion compensation.
11. The device according to claim 9, wherein: In the case where the sub-pixel position is closer to the first pixel than the second pixel, for the at least one of the weight tuples, the first weight of the one weight tuple and for the first pixel is higher than the bilateral weight based on the second distance.
12. The device according to claim 9, wherein: Selecting the first sub-pixel interpolation filter for sub-pixel interpolation of the fractional motion during the motion estimation for the block comprises: During said motion estimation for said block, selecting a motion vector having a fractional resolution; and Interpolated values at the fractional resolution are determined using the first sub-pixel interpolation filter.
13. The device according to claim 9, wherein: The second sub-pixel interpolation filter is the first sub-pixel interpolation filter.
14. The device according to claim 9, wherein: The second sub-pixel interpolation filter is selected based on a frequency response of the first sub-pixel interpolation filter.
15. The device according to claim 9, wherein: In the case of 7-bit precision, the weight tuple includes (128,0), (122,6), (116,12), (108,20), (100,28), (92,36), (82,46), (74,54), (64,64), (54,74), (46,82), (36,92), (28,100), (20,108), (12,116) and (6,122) for corresponding sub-pixel precisions of full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel and 15 / 16 pixel.
16. The device according to claim 9, wherein: In the case of 7-bit precision, the weight tuple includes (128,0), (124,4), (118,10), (110,18), (102,26), (94,34), (84,44), (74,54), (64,64), (54,74), (44,84), (34,94), (26,102), (18,110), (10,118) and (4,124) for corresponding sub-pixel precisions of full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel and 15 / 16 pixel.
17. An apparatus for decoding a block of a video using inter-frame prediction, the apparatus comprising: A processor configured to: decoding a sub-pixel interpolation filter from an encoded bitstream, the sub-pixel interpolation filter being used for sub-pixel interpolation of fractional motion, Wherein, the sub-pixel interpolation filter is a 2-tap filter, wherein the sub-pixel interpolation filter comprises weight tuples, each weight tuple consisting of two weights and used for sub-pixel interpolation at a sub-pixel position between a first pixel and a second pixel, wherein, for at least one of the weight tuples, the two weights are not based on a first distance between the sub-pixel position and the first pixel and a second distance between the sub-pixel position and the second pixel, wherein the first pixel and the second pixel are located at integer pixel positions, The sub-pixel position is located in the same row or column between the first pixel and the second pixel, and wherein the sub-pixel interpolation filter is designed as a low-pass filter to pass less high frequencies than a triangular filter; decoding motion vectors from the encoded bitstream; and The block is decoded using the sub-pixel interpolation filter and the motion vector.
18. The device according to claim 17, wherein: In the case where the sub-pixel position is closer to the first pixel than the second pixel, for the at least one of the weight tuples, the first weight of the one weight tuple and for the first pixel is higher than the bilateral weight based on the second distance.
19. The device according to claim 17, wherein: In the case of 7-bit precision, the weight tuple includes (128,0), (122,6), (116,12), (108,20), (100,28), (92,36), (82,46), (74,54), (64,64), (54,74), (46,82), (36,92), (28,100), (20,108), (12,116) and (6,122) for corresponding sub-pixel precisions of full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel and 15 / 16 pixel.
20. The device according to claim 17, wherein: In the case of 7-bit precision, the weight tuple includes (128,0), (124,4), (118,10), (110,18), (102,26), (94,34), (84,44), (74,54), (64,64), (54,74), (44,84), (34,94), (26,102), (18,110), (10,118) and (4,124) for corresponding sub-pixel precisions of full pixel, 1 / 16 pixel, 1 / 8 pixel, 3 / 16 pixel, 1 / 4 pixel, 5 / 16 pixel, 3 / 8 pixel, 7 / 16 pixel, 1 / 2 pixel, 9 / 16 pixel, 5 / 8 pixel, 11 / 16 pixel, 3 / 4 pixel, 13 / 16 pixel, 7 / 8 pixel and 15 / 16 pixel.
