Combined Motion Compensation Prediction

Abstract

A method and apparatus for combined motion compensated prediction is provided. In one embodiment, a prediction scheme is selected for encoding or decoding a video block. A first composite motion block may be determined by weighting the distance of the video frame from a first reference frame and the distance of the video frame from a second reference frame using one or more quantization weighting factors. A second composite motion block may be determined based on an average of pixel values ​​from a video block of the first reference frame and a video block of the second reference frame. One of the first composite motion block or the second composite motion block is selected and used to generate a predictive block. Alternatively, data encoded in a bitstream containing the video frame may be used to determine the composite motion block used to generate the predictive block. A current block of the video frame may be encoded or decoded using the predictive block.

Description

[Technical Field]

[0001] The present invention relates to hybrid motion compensated prediction. [Background technology]

[0002] A digital video stream represents a video using a series of frames or still images. Digital video is used for a variety of applications, including, for example, video conferencing, high-definition video entertainment, video advertising, or sharing of user-generated video. Digital video streams contain large amounts of data and consume significant computing or communication resources of computing devices for processing, transmitting, or storing the video data. Various approaches have been proposed to reduce the amount of data in video streams, including encoding or decoding techniques. Summary of the Invention

[0003] A method for encoding a current block of a video frame according to an embodiment of the present disclosure comprises determining a first composite motion block by weighting a distance from a first reference frame to the video frame and a distance from a second reference frame to the video frame using one or more quantization weighting factors. The method further comprises determining a second composite motion block based on an average of a plurality of pixel values ​​from the video block of the first reference frame and a plurality of pixel values ​​from the video block of the second reference frame. The method further comprises selecting one of the first composite motion block and the second composite motion block. The method further comprises generating a predictive block using the selected one of the first composite motion block and the second composite motion block. The method further comprises encoding the current block using the predictive block.

[0004] A method for decoding a coded block of a coded video frame according to an embodiment of the present disclosure comprises determining, based on one or more syntax elements coded in a bitstream including the coded video frame, whether the coded block was coded by weighting a plurality of distances from each of a plurality of reference frames to the coded video frame. In response to determining that the coded block was coded by weighting the distances from each of the plurality of reference frames to the coded video frame, the method further comprises determining a composite motion block by weighting a first distance from a first reference frame to the coded video frame and a second distance from a second reference frame to the coded video frame with one or more quantization weighting factors. The method further comprises generating a predictive block using the composite motion block. The method further comprises decoding the coded block using the predictive block.

[0005] An apparatus for decoding an encoded block of an encoded video frame according to an embodiment of the present disclosure comprises a processor configured to execute instructions stored in a non-transitory memory. The instructions include instructions for determining whether the encoded block was encoded by weighting a distance from each of a plurality of reference frames to the encoded video frame based on one or more syntax elements encoded in a bitstream including the encoded video frame. The instructions include instructions for determining, in response to determining that the encoded block was encoded by weighting a distance from each of a plurality of reference frames to the encoded video frame, determining a composite motion block by weighting a first distance from a first reference frame to the encoded video frame and a second distance from a second reference frame to the encoded video frame with one or more quantization weighting factors. The instructions further include instructions for generating a predictive block using the composite motion block. The instructions further include instructions for decoding the encoded block using the predictive block.

[0006] An apparatus for encoding a current block of a video frame according to an embodiment of the present disclosure comprises a processor configured to execute instructions stored in a non-transitory memory, the instructions including instructions for determining a first composite motion block by weighting a distance from a first reference frame to the video frame and a distance from a second reference frame to the video frame using one or more quantization weighting factors, the instructions further including instructions for determining a second composite motion block based on an average of a plurality of pixel values ​​from the video block of the first reference frame and a plurality of pixel values ​​from the video block of the second reference frame, selecting one of the first composite motion block and the second composite motion block, generating a predictive block using the selected one of the first composite motion block and the second composite motion block, and encoding the current block using the predictive block.

[0007] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims and the accompanying drawings.

[0008] This description makes reference to the accompanying drawings, which are described below, in which like reference numerals refer to like features throughout the several views. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a schematic diagram of a video encoding and decoding system; [Figure 2] FIG. 2 is a block diagram of an example of a computing device that may embody a transmitting station or a receiving station. [Figure 3] 1 is a diagram of a typical video stream that is encoded and subsequently decoded. [Figure 4] FIG. 2 is a block diagram of an encoder according to an embodiment of the present disclosure. [Figure 5] FIG. 2 is a block diagram of a decoder according to an embodiment of the present disclosure. [Figure 6] FIG. 10 is a flowchart of an example technique for encoding a video block using a selected prediction scheme. [Figure 7] FIG. 1 is a flowchart of an example technique for decoding a block that was coded using a selected prediction scheme. [Figure 8] 1 illustrates an example of distances between frames of a video sequence. [Figure 9] FIG. 10 is a diagram of an example of a tree for determining quantization weighting factors. DETAILED DESCRIPTION OF THE INVENTION

[0010] Video compression schemes may involve dividing each image or video frame into smaller portions, such as video blocks, and generating an encoded bitstream using techniques that limit the information contained in each of these video blocks. The encoded bitstream can be decoded to recreate the source image from the limited information. In some cases, hybrid motion prediction may be used to predict motion within a current video block by combining motion information from two or more reference frames.

[0011] For example, if two reference frames are used, pixel values ​​from each block of the reference frames may be identified and averaged to determine a compound motion block used to predict the motion of a current video block. However, this averaging-based prediction scheme may not always provide the best performance in encoding or decoding a video sequence. For example, the multiple reference frames may not be equidistant from the video frame that contains the current video block. As a result, the averaging-based prediction scheme may not accurately reflect changes in motion in a video sequence.

[0012] An embodiment of the present disclosure includes encoding or decoding video blocks of a video frame by selecting an optimal composite motion prediction scheme. A first composite motion block may be determined by weighting the distance of the video frame from a first reference frame and the distance of the video frame from a second reference frame using one or more quantized weighting coefficients. A second composite motion block may be determined based on an average of pixel values ​​from a video block of the first reference frame and a video block of the second reference frame. One of the first composite motion block or the second composite motion block is selected and used to generate a predictive block. Alternatively, data encoded in a bitstream including the video frame may be used to determine the composite motion block used to generate the predictive block. Video blocks of the video frame may be encoded or decoded using the predictive blocks.

[0013] For example, during an encoding operation, the encoder may select one of a distance-based prediction scheme (e.g., a first composite motion block) or an averaging-based prediction scheme (e.g., a second composite motion block) based on a rate-distortion value, etc., and encode data indicating the selection into the bitstream from which the video block is encoded. During a decoding operation, the decoder may decode the encoded data from the bitstream to determine which of the distance-based prediction scheme or the averaging-based prediction scheme to select for decoding the encoded block. As used herein, a composite motion block may refer to a group of pixel values ​​determined or otherwise calculated based on a combination of two or more other groups of pixel values.

[0014] Further details of video encoding techniques using frame rotation are described herein, first with reference to systems in which they may be implemented. Figure 1 is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 may be, for example, a computer having a hardware configuration as described in Figure 2. However, other embodiments of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 may be distributed across multiple devices.

[0015] A network 104 may connect the transmitting station 102 and the receiving station 106 for encoding and decoding of video streams. Specifically, the video streams may be encoded at the transmitting station 102, and the encoded video streams may be decoded at the receiving station 106. The network 104 may be, for example, the Internet. The network 104 may be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular network, or any other means of transferring video streams from the transmitting station 102, in this example, to the receiving station 106.

[0016] The receiving station 106 may, in one example, be a computer having the internal hardware configuration as depicted in Figure 2. However, other suitable embodiments of the receiving station 106 are possible. For example, the processing of the receiving station 106 may be distributed across multiple devices.

[0017] Other embodiments of the video encoding and decoding system 100 are possible. For example, an embodiment could omit the network 104. In another embodiment, the video stream can be encoded and stored for later transmission to the receiving station 106 or any other device having memory. In one embodiment, the receiving station 106 receives the encoded video stream (e.g., via the network 104, a computer bus, and / or some other communication path) and stores the video stream for later decoding. In one embodiment, the real-time transport protocol (RTP) is used to transmit the encoded video over the network 104. In another embodiment, a transport protocol other than RTP, such as the Hypertext Transfer Protocol (HTTP), which is based on a video streaming protocol, may be used.

[0018] When used in a videoconferencing system, for example, transmitting station 102 and / or receiving station 106 may include the capability to encode and decode video streams, as described below. For example, receiving station 106 may be a videoconference participant that receives an encoded video bitstream from a videoconference server (e.g., transmitting station 102), decodes and views it, and then encodes and transmits the video bitstream to the videoconference server for decoding and viewing by other participants.

[0019] Figure 2 is a block diagram of an example of a computing device 200 capable of implementing a transmitting station or a receiving station. For example, the computing device 200 may implement one or both of the transmitting station 102 and the receiving station 106 of Figure 1. The computing device 200 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device such as, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, or the like.

[0020] Processor 202 of computing device 200 may be a conventional central processing unit. Alternatively, processor 202 may be any other type of device or devices now existing or later developed that are capable of manipulating or processing information. For example, while the disclosed embodiments may be implemented with a single processor, e.g., CPU 202, as shown, advantages of speed and efficiency may be achieved by using multiple processors.

[0021] The memory 204 in the computing device 200 may, in embodiments, be a read-only memory (ROM) device or a random-access memory (RAM) device. However, any other suitable type of storage device may be used as the memory 204. The memory 204 may include code and data 206 that the processor 202 accesses using the bus 212. The memory 204 may further include an operating system 208 and application programs 210, which include at least one program that enables the processor 202 to perform the techniques described herein. For example, the application programs 210 may include applications 1-N, which further include a video encoding application that executes the techniques described herein. The computing device 200 may also include secondary storage 214, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may include a significant amount of information, they may be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0022] Computing device 200 may also include one or more output devices, such as a display 218. Display 218, in one example, may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. Display 218 may be connected to processor 202 via bus 212. Other output devices that enable a user to program or otherwise use computing device 200 may be provided in addition to or instead of display 218. When the output device is or includes a display, the display may be implemented in a variety of ways, including a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0023] Computing device 200 may include or communicate with imaging device 220, such as a camera, or any other imaging device 220 developed now or in the future, that can detect images, such as images of a user operating computing device 200. Imaging device 220 may be positioned to point toward a user operating computing device 200. In one example, the position and optical axis of imaging device 220 may be configured such that its field of view includes the area immediately adjacent to display 218 from which display 218 is viewable.

[0024] Computing device 200 may include or communicate with a voice sensing device 222, such as, for example, a microphone, or any other voice sensing device now or later developed, that can sense sounds near computing device 200. Voice sensing device 222 may be pointed toward a user operating computing device 200 and configured to receive, for example, voice or other utterances made by the user while the user is operating computing device 200.

[0025] While FIG. 2 depicts the processor 202 and memory 204 of computing device 200 as integrated into a particular unit, other configurations may be utilized. The operations of processor 202 may be distributed across multiple machines (individual machines may have one or more processors), which may be connected directly or via a local area network or other network. Memory 204 may be distributed across multiple machines, such as network-based memory, or may be memory in multiple machines that perform the operations of computing device 200. While depicted herein as a particular bus, bus 212 of computing device 200 may be comprised of multiple buses. Furthermore, secondary storage 214 may be directly connected to other components of computing device 200 or accessed over a network, and may include an integrated unit such as a memory card or multiple units, such as multiple memory cards. Accordingly, computing device 200 may be implemented in a wide variety of configurations.

[0026] FIG. 3 is a diagram of an example of a video stream 300 that is encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At a later stage, the video sequence 302 includes several adjacent frames 304. While three frames are shown as adjacent frames 304, the video sequence 302 may include any number of adjacent frames 304. The adjacent frames 304 may be further subdivided into individual frames, such as frame 306. At a later stage, the frame 306 may be divided into a series of planes or segments 308. The segments 308 may be subsets of frames that allow, for example, parallel processing. The segments 308 may be subsets of frames that allow for separation of video data into separate colors. For example, a frame 306 of color video data may include a luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0027] Regardless of whether a frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which may contain data corresponding to, for example, 16x16 pixels within the frame 306. A block 310 is configured to contain data from the pixel data of one or more segments 308. A block 310 may be any other suitable size, such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or more. Unless otherwise specified, the terms block and macroblock are used interchangeably herein.

[0028] Figure 4 is a block diagram of an encoder 400 according to an embodiment of the present disclosure. The encoder 400 may be implemented within the transmitting station 102 as described above, such as by providing a computer software program stored in a memory, such as memory 204. The computer software program may include machine instructions that, when executed by a processor, such as CPU 202, cause the transmitting station 102 to encode video data in the manner described in Figure 4. The encoder 400 may also be implemented as dedicated hardware included in the transmitting station 102, for example. In a particularly desirable embodiment, the encoder 400 is a hardware encoder.

[0029] The encoder 400 has an intra-prediction / inter-prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy coding stage 408 that use the video stream 300 as input and perform various functions in a forward path (shown by solid connecting lines) to produce an encoded or compressed bitstream 420. The encoder 400 also includes a reconstruction path (shown by dotted connecting lines) that reconstructs frames for encoding future blocks. In FIG. 4, the encoder 400 has the following stages that perform various functions in the reconstruction path: an inverse quantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0030] When the video stream 300 is presented for encoding, each adjacent frame 304, such as frame 306, may be processed block-by-block. In the intra-prediction / inter-prediction stage 402, each block may be coded using intra-frame prediction (also referred to as intra-prediction) or inter-frame prediction (also referred to herein as inter-prediction). In either case, a predictive block may be formed. In the case of intra-prediction, the predictive block may be formed from previously coded and reconstructed samples in the current frame. In the case of inter-prediction, the predictive block may be formed from samples in one or more previously constructed reference frames.

[0031] The predicted block is then subtracted from the current block in an intra-prediction / inter-prediction stage 402 to generate a residual block (also called a residual). A transform stage 404 converts the residual into transform coefficients, e.g., in the frequency domain, using a block-based transform. A quantization stage 406 converts the transform coefficients into discrete quantized values, called quantized transform coefficients, using quantization values ​​or levels. For example, the transform coefficients may be divided by the quantization value and truncated.

[0032] The quantized transform coefficients are then entropy coded by an entropy coding stage 408. The entropy coded coefficients, along with other information used to decode the block (which may include syntax elements used to indicate, for example, the type of prediction used, the transform type, motion vectors, and quantization values), are output in a compressed bitstream 420. The compressed bitstream 420 may be formatted using a variety of techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 is also referred to as a coded video stream or coded video bitstream, and these terms are used interchangeably herein.

[0033] To ensure that the encoder 400 and decoder 500 (described below with reference to FIG. 5 ) use the same reference frame to decode the compressed bitstream 420, a reconstruction path (indicated by the dotted connecting line) is used. The reconstruction path performs functions similar to those performed during the decoding process (described below with reference to FIG. 5 ), including inverse quantizing the quantized transform coefficients in an inverse quantization stage 410 and inverse transforming the inverse quantized transform coefficients in an inverse transform stage 412 to generate a differential residual block (also referred to as a differential residual). In a reconstruction stage 414, a predictive block predicted in the intra-prediction / inter-prediction stage 402 is added to the differential residual to create a reconstructed block. To reduce distortions such as blocking artifacts, a loop filtering stage 416 is applied to the reconstructed block.

[0034] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some embodiments, a non-transform-based encoder can directly quantize the residual signal for a block or frame without using the transform stage 404. In some implementations, the encoder can have the quantization stage 406 and the inverse quantization stage 410 combined into a common stage.

[0035] Figure 5 is a block diagram of a decoder 500 according to an embodiment of the present disclosure. The decoder 500 may be implemented in the receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program includes machine instructions that, when executed by a processor, such as processor 202, cause the receiving station 106 to decode video data in the manner described in Figure 5. The decoder 500 may also be implemented in hardware included in the transmitting station 102 or the receiving station 106, for example.

[0036] Similar to the reconstruction path of the encoder 400 described above, the decoder 500 in one example includes the following stages for performing various functions to generate an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, an inverse quantization stage 504, an inverse transform stage 506, an intra-prediction / inter-prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0037] When the compressed bitstream 420 is presented for decoding, data elements in the compressed bitstream 420 are decoded by an entropy decoding stage 502 to generate a set of quantized transform coefficients. An inverse quantization stage 504 inverse quantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantization value), and an inverse transform stage 506 inverse transforms the inverse quantized transform coefficients to generate differential residuals that are identical to those generated by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 uses an intra-prediction / inter-prediction stage 508 to create prediction blocks that are identical to those generated in the encoder 400 (e.g., in the intra-prediction / inter-prediction stage 402).

[0038] In the reconstruction stage 510, the predicted block is added to the differential residual to create a reconstructed block. A loop filtering stage 512 is applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, a deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking artifacts, and the result is output as an output video stream 516. The output video stream 516 is also referred to as a decoded video stream, and the terms are used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some embodiments, the decoder 500 can generate the output video stream 516 without using the deblocking filtering stage 514.

[0039] Techniques for encoding or decoding multiple video blocks will now be described with reference to Figures 6 and 7. Figure 6 is a flowchart diagram of an example method or technique 600 for encoding a video block using a selected prediction scheme. Figure 7 is a flowchart diagram of an example method or technique 700 for decoding a video block using a selected prediction scheme. One or both of techniques 600 and 700 can be implemented as software programs executed by a computing device, such as transmitting station 102 or receiving station 106. For example, the software program may be stored in a memory, such as memory 204 or secondary storage 214, and may include machine-readable instructions that, when executed by a processor, such as processor 202, cause the computing device to perform technique 600 and / or technique 700. One or both of techniques 600 and 700 may be embodied using specialized hardware or firmware. As mentioned above, some computing devices may have multiple memories or processors, and the operations described in one or both of techniques 600 and 700 may be distributed using multiple processors, memories, or both.

[0040] For ease of explanation, techniques 600 and 700 are each shown and described as a series of steps or operations. However, steps or operations according to the present disclosure may proceed in various orders and / or simultaneously. In addition, other steps or operations not shown and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a technique in accordance with the disclosed subject matter.

[0041] Referring to Figure 6, a flowchart diagram of a technique 600 for encoding a video block using a selected prediction scheme is shown. At 602, a first composite motion block is determined by weighting the distance from a first reference frame to the video frame and the distance from a second reference frame to the video frame using one or more quantization weighting factors. The first reference frame and the second reference frame may be past frames (e.g., frames that appear before the current video frame in display order) or future frames (e.g., frames that appear after the current video frame in display order). For example, the first reference frame may be a past frame and the second reference frame may be a future frame. In another example, the first reference frame and the second reference frame may both be past frames. In yet another example, the first reference frame and the second reference frame may both be future frames.

[0042] Determining the first composite motion block may include determining a first quantization weighting factor and a second quantization weighting factor by comparing a first distance from a first reference frame to a video frame with a second distance from a second reference frame to a video frame. The compared values ​​of the first distance and the second distance may be, for example, absolute values ​​of the first distance and the second distance. The first quantization weighting factor and the second quantization weighting factor are quantized based on the comparison, and the first composite motion block is determined using the first quantization weighting factor and the second quantization weighting factor. For example, as described below with respect to FIG. 9, each of the first quantization weighting factor and the second quantization weighting factor may be quantized to a value of 1, 2, 3, or 4. In another example, the first quantization weighting factor and the second quantization weighting factor may be quantized to other values ​​indicating a relationship between the first distance and the second distance.

[0043] Determining the first quantization weighting factor and the second quantization weighting factor by comparing the first distance with the second distance may include determining whether the first distance is greater than or less than the second distance. In response to determining that the first distance is greater than the second distance, the first quantization weighting factor is determined to have a value of "1" and the second quantization weighting factor has a value of at least "1". In response to determining that the first distance is less than the second distance, the first quantization weighting factor is determined to have a value of at least "1" and the second quantization weighting factor has a value of "1".

[0044] The first composite motion block may be determined, for example, as CMB = ((Ref1_MB*a)) + (Ref2_MB*b)) / (a+b), where Ref1_MB is a block of pixel values ​​of a first reference frame that can be used to predict motion within the current video block, Ref2_MB is a block of pixel values ​​of a second reference frame that can be used to predict motion within the current video block, a is a value determined by applying a first quantization weighting factor to the distance between the first reference frame and the current video frame, and b is a value determined by applying a second quantization weighting factor to the distance between the second reference frame and the current video frame.

[0045] That is, the first value may be determined by applying a first quantization weighting factor to a plurality of pixel values ​​from a video block of a first reference frame, the second value may be determined by applying a second quantization weighting factor to a plurality of pixel values ​​from a video block of a second reference frame, and the first composite motion block may then be determined by dividing the sum of the first value and the second value by the sum of the first quantization weighting factor and the second quantization weighting factor.

[0046] At 604, the second composite motion block is determined based on an average of pixel values ​​from the video block of the first reference frame and pixel values ​​from the video block of the second reference frame. For example, determining the average of the pixel values ​​from the video block of the first reference frame and pixel values ​​from the video block of the second reference frame may include summing pixel values ​​of corresponding positions of the two video blocks and dividing the summed pixel value by 2. In another example, determining the average of the pixel values ​​from the video block of the first reference frame and pixel values ​​from the video block of the second reference frame may include weighting all or a portion of the pixel values ​​from the video block of the first reference frame or the second reference frame before determining the average.

[0047] At 606, one of the first composite motion block and the second composite motion block is selected. Selecting one of the first composite motion block and the second composite motion block includes determining a plurality of rate-distortion values ​​obtained from predicting the motion of the current block using each of the first composite motion block and the second composite motion block. These rate-distortion values ​​may be determined by performing a rate-distortion analysis based on the first composite motion block and the second composite motion block. For example, a first rate-distortion value may be determined for the first composite motion block, and a second rate-distortion value may be determined for the second composite motion block. Then, one of the first composite motion block or the second composite motion block that results in the lower of the first and second rate-distortion values ​​is selected. For example, performing the rate-distortion analysis may include comparing the first rate-distortion value and the second rate-distortion value.

[0048] A rate-distortion value is a ratio that balances the amount of distortion (e.g., video quality degradation) and the rate (e.g., number of bits) for coding a block or other video component. Thus, a prediction scheme that minimizes the rate-distortion value to encode the video block is selected for encoding the current block. For example, if the rate-distortion value of a first composite motion block (e.g., a distance-based prediction scheme) is higher than the rate-distortion value of a second composite motion block (e.g., an averaging-based prediction scheme), this may reflect relatively low motion between the reference frame and the current video frame. However, if the rate-distortion value of the first composite motion block is lower than the rate-distortion value of the second composite motion block, this may reflect relatively high motion between the reference frame and the current video frame.

[0049] At 608, a predictive block is generated using the selected one of the first composite motion block or the second composite motion block. The predictive block may include a plurality of pixel values ​​indicating a prediction of the motion of the current block by the selected one of the first composite motion block or the second composite motion block. Generating the predictive block may include generating a prediction residual based on a difference between the current block and the predictive block. For example, the prediction residual may be generated using a plurality of operations performed by the encoder 400 shown in FIG. 4 (e.g., in the intra / inter prediction stage 402). At 610, the current block is coded using the predictive block. For example, coding the current block using the predictive block may include converting, quantizing, and entropy coding the prediction residual into a coded bitstream (e.g., the compressed bitstream 420 shown in FIG. 4).

[0050] In some implementations, the technique 600 includes encoding one or more syntax elements indicating a selection of the first or second composite motion block into a bitstream in which the current block is encoded. For example, the one or more syntax elements may include a bit. The value of the bit may indicate a prediction scheme associated with a motion vector used to encode the current block. For example, if the first composite motion block, i.e., a distance-based prediction scheme, is used, the value of the bit may be "0." If the second composite motion block, i.e., an averaging-based prediction scheme, is used, the value of the bit may be "1." The one or more syntax elements may be encoded in a frame header of a video frame including the encoded current block.

[0051] Technique 600 may include using more than two reference frames. For example, if three reference frames are used, a first composite motion block may be determined by weighting the distance between the first reference frame and the current video frame with a first quantization weighting factor, weighting the distance between the second reference frame and the current video frame with a second quantization weighting factor, and weighting the distance between the third reference frame and the current video frame with a third quantization weighting factor.

[0052] The composite motion block used to predict motion within the current video block using these three reference frames may be determined as, for example, CMB=((Ref1_MB*(1 / a))+(Ref2_MB*(1 / b))+(Ref3_MB*(1 / c))) / ((1 / a)+(1 / b)+(1 / c)), where Ref1_MB is a block of pixel values ​​from a first reference frame that can be used to predict motion within the current video block, and Ref2_MB is a block of pixel values ​​from a second reference frame that can be used to predict motion within the current video block. , Ref3_MB is a block of pixel values ​​of a third reference frame that can be used to predict motion within the current video block, a is a value determined by applying a first quantization weighting factor to the distance between the first reference frame and the current video frame, b is a value determined by applying a second quantization weighting factor to the distance between the second reference frame and the current video frame, and c is a value determined by applying a third quantization weighting factor to the distance between the third reference frame and the current video frame.

[0053] The technique 600 may include updating a probability model associated with a video frame to indicate whether a distance-based prediction scheme (e.g., a first composite motion block) or an averaging-based prediction scheme (e.g., a second composite motion block) is selected to predict the motion of a current block. For example, a context for the prediction scheme of the current block is determined based on multiple contexts of one or more blocks, among neighboring blocks above or to the left of the current block, and these contexts indicate the prediction scheme selected to encode these neighboring blocks. The context for any block of the video frame may have a first value indicating that a distance-based prediction scheme was used or a second value indicating that an averaging-based prediction scheme was used. The probability model may be updated to reflect the probability of each of these prediction schemes being used.

[0054] Referring to Figure 7, a technique 700 for decoding a block coded using a selected prediction scheme is shown. At 702, one or more syntax elements are decoded from a bitstream including a coded video frame, the coded video frame including the coded block. The one or more syntax elements may be decoded, for example, from a frame header of the coded video frame. The one or more syntax elements may be encoded into the coded bitstream to indicate whether the encoder used to code the coded block selected a distance-based prediction scheme or an averaging-based prediction scheme to predict the motion of the coded block prior to coding.

[0055] At 704, a determination is made as to whether the coded block was coded by weighting a plurality of distances from each of a plurality of reference frames to the coded video frame. The determination may be made based on values ​​of one or more syntax elements decoded from a bitstream including the coded frame. For example, the decoded syntax element may include a particular bit. If the bit has a first value (e.g., 0), the coded block may be determined to have been coded by weighting a plurality of distances from each of a plurality of reference frames to the coded video frame. However, if the bit has a second value (e.g., 1), the coded block may be determined to have not been coded by weighting a plurality of distances from each of a plurality of reference frames to the coded video frame.

[0056] At 706, in response to determining that the coded block was coded by weighting a plurality of distances from each of a plurality of reference frames to the coded video frame, the composite motion block is determined by weighting a first distance from a first reference frame to the coded video frame and a second distance from a second reference frame to the coded video frame with one or more quantization weighting factors. As mentioned above, the first reference frame and the second reference frame may be past frames (e.g., frames that appear before the current video frame in display order) or future frames (e.g., frames that appear after the current video frame in display order).

[0057] The composite motion block may be determined at 706 in the same or similar manner as the embodiment for determining the first composite motion block described with respect to technique 600. However, while an encoder performing technique 600 receives information from the input video stream indicating the display order of a video sequence including a video frame, a first reference frame, and a second reference frame, a decoder performing technique 700 does not receive that information from the input video stream.

[0058] Alternatively, the order of the coded video frames of a video sequence including the coded video frame, the first reference frame, and the second reference frame may be indicated within a bitstream including the coded video frames (and, e.g., one or more previously decoded syntax elements). For example, the coded bitstream may include data indicating a frame index for each of the coded video frames encoded in the coded bitstream. These frame indexes may be used, alone or in conjunction with other data (e.g., packet stamp data, other time offset data, etc.), by a decoder performing technique 700 to determine the display order of the video sequence.

[0059] Alternatively, in response to determining that the coded block was not coded by weighting the distances using one or more quantization weighting factors at 708, a composite motion block may be determined based on an average of a plurality of pixel values ​​of the coded video block of the first reference frame and a plurality of pixel values ​​of the coded video block of the second reference frame. The composite motion block may be determined at 708 in the same or similar manner as the embodiment for determining the second composite motion block described with respect to technique 600.

[0060] At 710, a predictive block is generated using the composite motion block determined at 706 or 708. As described above with respect to technique 600, the predictive block may include a plurality of pixel values ​​indicating a prediction of the motion of the coded block according to the determined composite motion block. Generating the predictive block may include generating a prediction residual based on a difference between the coded block and the predictive block. For example, the prediction residual may be generated using a plurality of operations performed by the decoder 500 shown in FIG. 5 (e.g., in intra / inter prediction stage 508). At 712, the coded block is decoded using the predictive block. For example, decoding the coded block using the predictive block may include reconstructing a video block based on the prediction residual, filtering the reconstructed video block, and outputting the filtered video block to a video stream (e.g., output video stream 516 shown in FIG. 5).

[0061] In some implementations, the technique 700 includes using more than two reference frames. For example, the technique 700 can decode an encoded block using the same number of reference frames as the encoder used to encode the encoded block. An embodiment using more than two reference frames is described above with respect to the technique 600.

[0062] Technique 700 may include updating a probability model associated with the encoded video frame to indicate whether a composite motion block corresponding to a distance-based prediction scheme (e.g., a first composite motion block) or a composite motion block corresponding to an averaging-based prediction scheme (e.g., a second composite motion block) is selected to predict the motion of the encoded block. Embodiments of updating the probability model are described above with respect to technique 600.

[0063] 8 illustrates an example of distances between frames of a video sequence. Video frame 800 includes an encoded or decoded video block 802. Video block 802 may be encoded or decoded using a first reference frame 804 and a second reference frame 806. For example, all or a portion of pixel values ​​of video block 808 of the first reference frame may be combined with all or a portion of pixel values ​​of video block 810 of the second reference frame. The combination of these pixel values ​​may be based on a first distance 812 indicating the display order distance between first reference frame 804 and video frame 800, and on a second distance 814 indicating the display order distance between second reference frame 806 and video frame 800.

[0064] If the first distance 812 and the second distance 814 are equal, pixel values ​​of the video block 808 and pixel values ​​of the video block 810 may be combined equally to predict the motion of the video block 802. However, one of the first distance 812 and the second distance 814 may be greater than the other. In such a case, the farther the two reference frames 804, 806 are from the video frame 800, the less influence they are likely to have on the overall prediction of the motion of the video block 802. In the example shown, the second distance 814 is greater than the first distance 812. Therefore, corresponding pixel values ​​of the video block 808 of the first reference frame should be given a greater weight than corresponding pixel values ​​of the video block 810 of the second reference frame when these pixel values ​​are combined to predict the motion of the video block 802.

[0065] 9 is a diagram of an example tree 900 for determining quantization weighting factors. When using a distance-based prediction scheme for compound motion prediction as described in embodiments of the present disclosure, determining how to weight pixel values ​​from each reference frame helps hide the quantization noise contributed by each of these reference frames. To further reduce the quantization noise, the weights applied to the pixel values ​​of each reference frame are quantization factors. The quantization weighting factor applied to each of multiple groups of pixel values ​​of multiple reference frames may be determined based on a comparison of the distance between each of these reference frames and a video frame including a video block to be encoded or decoded.

[0066] In the shown example, each leaf of the tree 900 includes a pair of quantization weighting factors for predicting motion of a video block using a comparison of a distance D1 (e.g., the first distance 812 shown in FIG. 8) and a distance D2 (e.g., the second distance 814 shown in FIG. 8) or multiple pixel values ​​from a first reference frame and a second reference frame. As mentioned above, the compared values ​​of D1 and D2 may be, for example, the absolute values ​​of D1 and D2.

[0067] At the first leaf, D1 and D2 are compared to determine which is greater. Based on that comparison, tree 900 further compares the ratio of D1 and D2 for different values ​​to determine one of eight different possible pairs of quantization weighting factors, each pair including a first quantization weighting factor applied to multiple pixel values ​​of a video block of a first reference frame and a second quantization weighting factor applied to multiple pixel values ​​of a video block of a second reference frame.

[0068] For example, if D1 is greater than D2 and the ratio of D1 to D2 is less than 1.5, the first quantization weighting factor is "1" and the second quantization weighting factor is "1." If D1 is greater than D2 and the ratio of D1 to D2 is 1.5 or greater but less than 2.5, the first quantization weighting factor is "1" and the second quantization weighting factor is "2." If D1 is greater than D2 and the ratio of D1 to D2 is 2.5 or greater but less than 3.5, the first quantization weighting factor is "1" and the second quantization weighting factor is "3." However, if D1 is greater than D2 and the ratio of D1 to D2 is 3.5 or greater, the first quantization weighting factor is "1" and the second quantization weighting factor is "4."

[0069] In another example, when D2 is greater than D1 and the ratio of D2 to D1 is less than 1.5, the first quantization weighting factor is "1" and the second quantization weighting factor is "1." When D2 is greater than D1 and the ratio of D2 to D1 is 1.5 or greater and less than 2.5, the first quantization weighting factor is "2" and the second quantization weighting factor is "1." When D2 is greater than D1 and the ratio of D2 to D1 is 2.5 or greater and less than 3.5, the first quantization weighting factor is "3" and the second quantization weighting factor is "1." However, when D2 is greater than D1 and the ratio of D2 to D1 is 3.5 or greater, the first quantization weighting factor is "4" and the second quantization weighting factor is "1."

[0070] Therefore, the values ​​of the first quantization weighting factor and the second quantization weighting factor indicate a relationship between a first distance between the first reference frame and the current video frame and a second distance between the second reference frame and the current video frame. For example, if the ratio of the first distance to the second distance indicates that the first distance is twice the second distance, the first quantization weighting factor has a value of "2" and the second quantization weighting factor has a value of "1". In another example, if the ratio of the first distance to the second distance indicates that the second distance is four times or more the second distance, the first quantization weighting factor has a value of "1" and the second quantization weighting factor has a value of "4".

[0071] The encoding and decoding aspects described above illustrate some examples of encoding and decoding techniques. However, it should be understood that encoding and decoding, as those terms are used in the claims, may mean compressing, decompressing, converting, or any other processing or modification of data.

[0072] The word "exemplary" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other aspects or designs. Rather, use of the word "exemplary" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless expressly stated otherwise or clearly indicated otherwise by context, the phrase "X includes A or B" is intended to mean any natural inclusive permutation. That is, "X includes A or B" is satisfied in any of the following cases: X includes A, X includes B, or X includes both A and B. Furthermore, the articles "a" and "an," as used in this application and the appended claims, should generally be construed to mean "one or more," unless expressly stated otherwise or clearly indicated by context directed to the singular form. Furthermore, use of the terms "embodiment" or "one embodiment" in this disclosure is not intended to refer to the same embodiment or implementation unless so stated.

[0073] Embodiments of the transmitting station 102 and / or receiving station 106 (as well as the algorithms, methods, instructions, etc. contained in, stored on, and / or executed by the encoder 400 and decoder 500) may be realized in hardware, software, or any combination thereof. Hardware may include, for example, computers, intellectual property (IP) cores, application-specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or other suitable circuitry. In the claims, the term "processor" should be understood to include any of the foregoing hardware, alone or in combination. The terms "signal" and "data" are used interchangeably. Furthermore, portions of the transmitting station 102 and receiving station 106 need not necessarily be implemented in the same way.

[0074] Furthermore, in one aspect, for example, the transmitting station 102 or the receiving station 106 may be implemented using a general-purpose computer or processor with a computer program that, when executed, executes any of the individual methods, algorithms, and / or instructions described herein. Additionally or alternatively, a special-purpose computer / processor may be utilized that may include, for example, other hardware for executing any of the methods, algorithms, or instructions described herein.

[0075] The transmitting station 102 and the receiving station 106 may be implemented on computers, for example, within a videoconferencing system. Alternatively, the transmitting station 102 may be implemented on a server, and the receiving station 106 may be implemented on a device separate from the server, such as a handheld communication device. In this case, the transmitting station 102 may encode content into an encoded video signal using the encoder 400 and transmit the encoded video signal to the communication device. The communication device may decode the encoded video signal using the decoder 500. Alternatively, the communication device may decode content stored locally on the communication device, e.g., content not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementations are available. For example, the receiving station 106 may be a generally fixed personal computer rather than a portable communication device, and / or the device including the encoder 400 may include the decoder 500.

[0076] Furthermore, all or part of the embodiments of the present disclosure may take the form of a computer program product, for example, accessible from a computer-usable or computer-readable medium. The computer-usable or computer-readable medium may be, for example, any device that can tangibly contain, store, communicate, or transport a program for use by any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media may also be utilized.

[0077] The above-described embodiments, examples, and aspects are provided to facilitate understanding of the present disclosure, and are not intended to limit the present disclosure. The present disclosure is intended to encompass various modifications and equivalent structures that fall within the scope of the appended claims, which scope is to be accorded the broadest interpretation permitted by law so as to encompass all modifications and equivalent structures.

Claims

1. 1. An apparatus for decoding an encoded block of an encoded video frame, comprising: a decoder, the decoder comprising: decoding, from the encoded bitstream comprising the encoded video frame, one or more syntax elements that indicate how to perform prediction for the encoded block; determining whether the one or more syntax elements indicate performing prediction of the coded block using a weighted distance approach; In response to determining that the one or more syntax elements indicate performing prediction of the coded block using the weighted distance approach, selecting a set of quantization weighting factors consisting of a first quantization weighting factor a, a second quantization weighting factor b, and a third quantization weighting factor c from a plurality of sets of predetermined quantization weighting factors based on a first ratio and a second ratio, wherein the first ratio is a ratio between an absolute value of a first distance from a first reference frame to the coded video frame and an absolute value of a second distance from a second reference frame to the coded video frame, and the second ratio is a ratio between an absolute value of the second distance and an absolute value of a third distance from a third reference frame to the coded video frame; forming a first composite motion block CMB determined using the formula CMB=((Ref1_MB*(1 / a))+(Ref2_MB*(1 / b))+(Ref3_MB*(1 / c))) / ((1 / a)+(1 / b)+(1 / c)), where Ref1_MB is a block of pixel values ​​of the first reference frame, Ref2_MB is a block of pixel values ​​of the second reference frame, and Ref3_MB is a block of pixel values ​​of the third reference frame; decoding the coded block using the first composite motion block CMB; In response to determining that the one or more syntax elements do not indicate performing prediction of the coded block using the weighted distance approach, forming a second composite motion block based on an average of the video data of the first reference frame and the video data of the second reference frame; decoding the coded block using the second compound motion block.

2. 2. The apparatus of claim 1, wherein an order of multiple coded video frames included in the coded bitstream is indicated using multiple packet stamps included in the coded bitstream, the multiple coded video frames including the coded video frame, the first reference frame, the second reference frame, and the third reference frame.

3. The decoder 3. The apparatus of claim 1, configured to update a probability model to indicate whether the weighted distance approach was used to perform the prediction of the coded block.

4. The decoder a processor; A memory configured to execute a plurality of instructions stored in the memory.

5. 1. A method of decoding an encoded block of an encoded video frame, comprising: selecting a set of quantization weighting factors consisting of a first quantization weighting factor a, a second quantization weighting factor b, and a third quantization weighting factor c from a plurality of sets of predetermined quantization weighting factors based on a first ratio and a second ratio, wherein the first ratio is a ratio between an absolute value of a first distance from a first reference frame to the coded video frame and an absolute value of a second distance from a second reference frame to the coded video frame, and the second ratio is a ratio between an absolute value of the second distance and an absolute value of a third distance from a third reference frame to the coded video frame; forming a composite motion block CMB determined using the formula CMB=((Ref1_MB*(1 / a))+(Ref2_MB*(1 / b))+(Ref3_MB*(1 / c))) / ((1 / a)+(1 / b)+(1 / c)), where Ref1_MB is a block of pixel values ​​of the first reference frame, Ref2_MB is a block of pixel values ​​of the second reference frame, and Ref3_MB is a block of pixel values ​​of the third reference frame; decoding the coded block using the composite motion block.

6. decoding, from the coded bitstream comprising the coded video frame, one or more syntax elements that indicate how to perform prediction for the coded block; determining whether the one or more syntax elements indicate performing prediction of the coded block using a weighted distance approach; 6. The method of claim 5, wherein the composite motion block is formed in response to determining that the one or more syntax elements indicate performing prediction of the coded block using the weighted distance approach.

7. the composite motion block is a first composite motion block; forming a second composite motion block based on an average of video data of the first reference frame and video data of the second reference frame in response to determining that the one or more syntax elements do not indicate performing prediction of the coded block using the weighted distance approach; The method of claim 6 , further comprising: decoding the coded block using the second composite motion block.

8. 8. The method of claim 5, wherein the first distance, the second distance, and the third distance are determined based on an order of a plurality of coded video frames of a coded bitstream, the plurality of coded video frames including the coded video frame, the first reference frame, the second reference frame, and the third reference frame.

Images