Projected motion field hole filling for motion vector referencing

Abstract

Projected motion field hole filling includes determining a motion field for the current frame using a proper subset of blocks of the current frame, the motion field comprising a respective motion vector projecting onto a reference frame for each block of the proper subset of blocks. For a block of the current frame without a motion vector in the motion field, the respective motion vector for a spatially neighboring block of the proper subset of blocks is reused as a projected motion vector for the block within the motion field. A motion vector candidate list may be determined using the motion field. A reference motion vector for the current block may be selected from the motion vector candidate list. The reference motion vector may be used to encode the current block into an encoded bitstream or decode the current block from an encoded bitstream.

Description

PROJECTED MOTION FIELD HOLE FILLING FOR MOTION VECTORREFERENCINGCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 535,276, filed August 29, 2024, which is incorporated herein in its entirety by reference.BACKGROUND

[0002] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high-definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including compression techniques in encoding and decoding.SUMMARY

[0003] An aspect of the teachings herein is a method for coding a current block of a current frame. The method includes determining a motion field for the current frame using a proper subset of blocks of the current frame, the motion field comprising a respective motion vector projecting onto a reference frame for each block of the proper subset of blocks, for a block of the current frame without a motion vector in the motion field, reusing the respective motion vector for a spatially neighboring block of the proper subset of blocks as a projected motion vector for the block within the motion field, determining a motion vector candidate list using the motion field, selecting a reference motion vector for the current block from the motion vector candidate list, and coding the current block using the reference motion vector, such as encoding the current block into an encoded bitstream using the reference motion vector.

[0004] Another aspect of the teachings herein is a method for coding a current block of a current frame. The method includes determining a motion field for the current frame using aproper subset of blocks of the current frame, the motion field comprising a respective motion vector projecting onto a reference frame for each block of the proper subset of blocks, for a block of the current frame without a motion vector in the motion field, reusing the respective motion vector for a spatially neighboring block of the proper subset of blocks as a projected motion vector for the block within the motion field, determining a motion vector candidate list using the motion field, selecting a reference motion vector for the current block, and coding the current block using the reference motion vector, such as decoding the current block from an encoded bitstream using the reference motion vector.

[0005] Where coding the current block is encoding the current block, a syntax element may be encoded into the encoded bitstream identifying the reference motion vector. A syntax element may be encoded into the encoded bitstream identifying whether to filter the motion field before determining the motion vector candidate list.

[0006] Where coding the current block is decoding, a syntax element may be decoded from the encoded bitstream identifying the reference motion vector. A syntax element may be decoded from the encoded bitstream identifying whether to filter the motion field before determining the motion vector candidate list.

[0007] Reusing the respective motion vector for the spatially neighboring block may include reusing the respective motion vector for a spatially nearest available neighboring block determined using Manhattan distance.

[0008] The motion field may be filtered before determining the motion vector candidate list. Filtering the motion field may include replacing the projected motion vector with an average motion vector generated from multiple motion vectors and leaving the respective motion vector for each block of the proper subset of blocks unchanged. In these implementations, the multiple motion vectors can include the projected motion vector and the respective motion vector for at least one spatially neighboring block.

[0009] Another aspect of the teachings herein is an apparatus for encoding or decoding a current block of a current frame including a processor. The processor is configured to perform any of the methods described herein. For example, a memory may store instructions that, when executed, cause the processor to perform any of the methods described herein.

[0010] Another aspect of the teachings herein is a non-transitory, computer-readable storage medium storing instructions for performing any of the methods described herein.

[0011] Another aspect of the teachings herein is non-transitory, computer-readable storage medium storing an encoded bitstream. The encoded bitstream includes an encoded residual for a current block, the encoded residual encoded by determining a motion field formultiple blocks of the current frame using a proper subset of blocks of the current frame, the motion field comprising a respective motion vector projecting onto a reference frame for each block of the multiple blocks projecting onto a reference frame proper subset of blocks, for a block of the current frame without a motion vector in the motion field, reusing the respective motion vector for a spatially neighboring block of the multiple proper subset of blocks as a projected motion vector for the block within the motion field, determining a motion vector candidate list using the motion field, selecting a reference motion vector for the current block from the motion vector candidate list, and encoding the current block into the encoded bitstream using the reference motion vector.

[0012] The encoded bitstream may include a first syntax element identifying the reference motion vector and / or a second syntax element identifying whether to filter the motion field before determining the motion vector candidate list.

[0013] These and other aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims, and the accompanying figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The description herein refers to the accompanying drawings described below, wherein like reference numerals refer to like parts throughout the several views.

[0015] FIG. 1 is a schematic of a video encoding and decoding system.

[0016] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station.

[0017] FIG. 3 is a diagram of a typical video stream to be encoded and subsequently decoded.

[0018] FIG. 4 is a block diagram of an encoder according to implementations of this disclosure.

[0019] FIG. 5 is a block diagram of a decoder according to implementations of this disclosure.

[0020] FIG. 6 is a flowchart diagram of an example of a technique for encoding or decoding using projected motion field hole filling for motion vector referencing.

[0021] FIG. 7 is an illustration of an example of data used to indicate a display order of video frames of a video sequence.

[0022] FIG. 8 is a diagram of an example of using a motion trajectory for a current frame to predict motion of a current block.

[0023] FIG. 9 is a diagram of an example of using multiple motion trajectories to predict motion of a current block.DETAILED DESCRIPTION

[0024] Video compression schemes may include breaking respective images, or video frames, into smaller portions, such as video blocks, and generating an encoded bitstream using techniques to limit the information included for respective video blocks thereof. The encoded bitstream can be decoded to re-create the source images from the limited information. Encoding or decoding a video block can include predicting motion within that video block, such as with respect to one or more other video blocks in the same video frame or in a different video frame.

[0025] Predicting the motion within a video block is complicated by changes in the speed of motion, such as over multiple video frames in a video sequence. For example, an object moving between a series of consecutive frames may move more quickly between video frames of a first half of that series and more slowly between video frames of a second half of that series. Information indicating a display order of the video frames of that series may be useful in determining which of the video frames may be used to predict motion within a given video block; however, that information is not typically made available to a decoder. Furthermore, there may be multiple different directions of motion between those video frames such that it is typically difficult to effectively identify the possible motion vectors to use to predict motion of a video block and select the most optimal of those motion vectors for the prediction.

[0026] Implementations of this disclosure include using a (e.g., linear) projection model to create a motion field estimation that can estimate different temporal motion trajectories, including those with high velocity. The resulting motion vectors are used as motion vector candidates in a dynamic motion vector referencing system to augment any motion vector candidates from spatial neighbors and optionally from a collocated block to the current block being coded (also called the current coding block). However, many coding blocks of the frame will not have a motion vector within the motion field, which reduces the usefulness of the motion vector referencing for the frame as a whole. The techniques herein describe projected motion field hole filling, which can increase the number of blocks of a frame that benefit from motion vector candidates determined using motion field estimation and hence improve motion vector referencing. The motion vector candidates can be used for codingmotion vectors of the current coding block as described in more detail below after initial reference to a system in which the teachings herein can be implemented.

[0027] FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such as that described in FIG. 2. However, other implementations of the transmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0028] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0029] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0030] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, such as a video streaming protocol based on the Hypertext Transfer Protocol (HTTP).

[0031] When used in a video conferencing system, for example, the transmitting station 102 and / or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a video conference server (e.g., the transmitting station 102) to decode and view and further encodes andtransmits his or her own video bitstream to the video conference server for decoding and viewing by other participants.

[0032] FIG. 2 is a block diagram of an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106 of FIG. 1. The computing device 200 can be in the form of a computing system including multiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like.

[0033] A processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. For example, although the disclosed implementations can be practiced with one processor as shown (e.g., the processor 202), advantages in speed and efficiency can be achieved by using more than one processor.

[0034] A memory 204 in computing device 200 can be a read only memory (ROM) device or a random access memory (RAM) device in an implementation. However, other suitable types of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the processor 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the processor 202 to perform the techniques described herein. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described herein. The computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device.Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0035] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, thedisplay can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0036] The computing device 200 can also include or be in communication with an image-sensing device 220, for example, a camera, or any other image-sensing device 220 now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image-sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0037] The computing device 200 can also include or be in communication with a soundsensing device 222, for example, a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound-sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0038] Although FIG. 2 depicts the processor 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the processor 202 can be distributed across multiple machines (wherein individual machines can have one or more processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0039] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individualframes, for example, a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 can be subsets of frames that permit parallel processing, for example. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can include a luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0040] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise noted, the terms block and macroblock are used interchangeably herein.

[0041] FIG. 4 is a block diagram of an encoder 400 according to implementations of this disclosure. The encoder 400 can be implemented, as described above, in the transmitting station 102, such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0042] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0043] When the video stream 300 is presented for encoding, respective adjacent frames 304, such as the frame 306, can be processed in units of blocks. At the intra / inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intraprediction) or inter- frame prediction (also called inter-prediction). In any case, a predictionblock can be formed. In the case of intra-prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter-prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0044] Next, the prediction block can be subtracted from the current block at the intra / inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated.

[0045] The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0046] The reconstruction path (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below with respect to FIG. 5) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are similar to functions that take place during the decoding process (described below with respect to FIG. 5), including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0047] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In someimplementations, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0048] FIG. 5 is a block diagram of a decoder 500 according to implementations of this disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0049] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0050] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter prediction stage 508 to create the same prediction block as was created in the encoder 400 (e.g., at the intra / inter prediction stage 402).

[0051] At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, the deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as the output video stream 516. The output video stream 516 can also be referred to as a decoded video stream, and the terms will be used interchangeably herein. Other variations of the decoder 500 can beused to decode the compressed bitstream 420. In some implementations, the decoder 500 can produce the output video stream 516 without the deblocking filtering stage 514.

[0052] As described above, one technique for compressing the amount of data to be transmitted in a bitstream is to use inter prediction, which creates a respective prediction block for a current coding block using one or more motion vectors. The prediction block(s) can be used to generate a residual for coding in place of the original coding block. The motion vectors (or identifiers of the motion vectors) for inter-predicted blocks must also be transmitted. To reduce the number of bits required to transmit the motion vectors, a motion vector candidate list may be generated on a block basis, and a motion vector candidate may be used to code a motion vector for predicting the block. For example, the motion vector candidate may be used as the motion vector for predicting the block or the motion vector candidate may be used for generating a difference with the motion vector, also referred to as a motion vector residual, and the motion vector residual is coded in place of the original motion vector in a like manner as the residual block.

[0053] Higher quality motion vector candidates (i.e., motion vector candidates that closely match a motion vector of the current coding block) minimize the amount of data that must be transmitted to recreate the motion vector at a decoder. Motion vector candidates may be obtained from motion vectors used for the prediction of spatially neighboring blocks within the frame on the theory that the motion of such blocks will be similar to the current coding block.

[0054] Temporal motion vector correlation may be exploited by using the motion vector(s) of the collocated block in a previously reconstructed frame as the reference motion vector. While useful to track consistent and slow-motion activities, these motion vector(s) generally do not provide further compression gains over the motion vector(s) of the spatially neighboring blocks. In contrast, motion trajectories with high velocities are not easily captured by conventional techniques. Motion field estimation can be used to linearly project the available motion vectors from the reference frame to build a set of motion trajectories that go through the current coding frame. Coding block positions in the coding frame can respectively trace the motion trajectory to estimate a motion vector as a candidate.

[0055] By going over the available motion vectors in the reference frame to build a motion field for the current frame, motion vector candidates can be determined for predicting the motion vectors at the coding block level. Even going over all the available motion vectors in the reference frames for the current frame to build the motion vector candidate list, there are inevitably many blocks in the current frame that do not have a motion vector projectedonto them, which leaves holes in the motion field. It is possible to simply treat these holes as unavailable and not use them for motion vector prediction. Improving prediction quality for motion vectors can be achieved by performing an additional “hole filling” process after the initial motion field is built.

[0056] FIG. 6 is a flowchart diagram of an example of a technique 600 (process, method, etc.) for encoding or decoding using projected motion field hole filling for motion vector referencing. The technique 600 can be implemented, for example, as a software program that may be executed by computing devices such as the transmitting station 102 or the receiving station 106. For example, the software program can include machine-readable instructions that may be stored in a memory such as the memory 204 or the secondary storage 214, and that, when executed by a processor, such as the processor 202, may cause the computing device to perform the technique 600. The technique 600 can be implemented using specialized hardware or firmware. As explained above, some computing devices may have multiple memories or processors, and the operations described in the technique 600 can be distributed using multiple processors, memories, or both.

[0057] For simplicity of explanation, the technique 600 is depicted and described as a series of steps or operations. However, the steps or operations in accordance with this disclosure can occur in various orders and / or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement a technique in accordance with the disclosed subject matter.

[0058] At operation 602, a motion trajectory is determined. The motion trajectory is a trajectory of motion between video frames of a video sequence including a current video block to encode or decode. The motion trajectory can, for example, show the motion of an object from a first reference frame to a current video frame including the current video block, and then from the current video frame to a second reference frame, such as where the first reference frame is a past frame with respect to the current video frame (e.g., such that the second reference frame comes before the current video frame in a display order) or a future frame with respect to that current video frame (e.g., such that the second reference frame comes after the current video frame in the display order).

[0059] The motion trajectory indicates how one or more objects move or other motion occurs between the video frames based on a path of the motion from one of the video frames to one or more of the others. As such, the motion trajectory may be based on or otherwise indicative of a display order of the video sequence. The display order of the video sequence isknown to an encoder performing the technique 600 based on the input video stream received by the encoder. However, a decoder performing the technique 600 does not receive an input video stream and may not be aware of the display order of the video sequence it decodes. Instead, the decoder receives an encoded bitstream indicating a coding order of the video sequence.

[0060] The display order and the coding order for a video sequence are typically different. For example, a display order for a video sequence including eight frames may be 1, 2, 3, 4, 5, 6, 7, and 8. However, a coding order for that video sequence may instead be, for example, 1, 5, 4, 6, 3, 7, 8, and 2, such as where a hierarchical framework is used for encoding and decoding the frames of the video sequence. The encoder and the decoder may both know which reference frames of the video sequence are usable for encoding or decoding the current video block; however, without further information being provided to the decoder, the decoder cannot leverage information about the display order of the video sequence to decode the current video block, such as leveraging the distance between frames in the display order to properly identify motion between frames for motion vector referencing.

[0061] Further, certain video frames included in the coding order may not be included in the display order. Such video frames, referred to as alternative reference frames, are decoded, but not displayed as part of an output video sequence. Instead, alternative reference frames remain in the reference frame buffer for use in predicting motion in video blocks of video frames that are output for display. An encoder may encode one or more syntax elements to a bitstream to indicate the locations of alternative reference frames in a coding order for a video sequence. A decoder can decode those one or more syntax elements to determine to use an alternative reference frame from the reference frame buffer rather than use a frame for display.

[0062] Data usable to determine the display order of the video sequence may be received by a decoder to determine the display order of the video sequence to decode the current video block. That data may include one or more frame offset values that indicate respective locations of video frames in a display order of a video sequence.

[0063] A frame offset value for a video frame is determined during encoding based on a frame index for that video frame and a number of packet stamps after which the video frame will be displayed. The frame index of a video frame reflects the location of that video frame in a display order, as known to the encoder (e.g., based on an input video stream received for encoding by the encoder). During an encoding operation, the encoder determines the frame indexes of the video frames in the video sequence being encoded. The first frame in thedisplay order of the video sequence has an index of 1, and the Nth frame in that display order has an index of N. However, whereas the first frame in the video sequence is the first video frame to be encoded, the next video frame to be encoded may be a video frame located in the middle of the video sequence.

[0064] When a video frame is encoded to a bitstream, the data indicative of the encoded video frame can be stored in a packet. A packet includes information about one or more video frames of a video sequence. One of those video frames is a displayable video frame to be displayed subsequent to decoding using a decoder. The others of those video frames are reference frames used to decode the video frame to be displayed.

[0065] The number of jumps for a packet reflects the number of video frames that have to be decoded by a decoder in order to display the displayable video frame, such as by outputting the displayable video frame to an output video stream. For example, when a decoder receives a packet including N video frames, the decoder decodes the first N-l video frames and uses those as reference frames for decoding the Nth video frame, which Nth video frame is then output for display. The number of packet stamps for a packet indicates the total number of video frames displayed after that packet has been decoded. The encoder can determine the number of packet stamps based on the number of reference frames used to encode a given video frame.

[0066] During encoding, subsequent to determining a frame index and a number of packet stamps for a video frame, the frame offset value is calculated as the difference between the frame index and the number of packet stamps. For example, for a video frame having a frame index of 1 and 1 packet stamp, the frame offset value is 0. In another example, for a video frame having a frame index of 8 and 2 packet stamps, the frame offset value is 6. In yet another example, for a video frame having a frame index of 2 and 2 packet stamps, the frame offset value is 0. Examples of relationships between frame indexes, numbers of packet stamps, and frame offset values are described below with respect to FIG. 7.

[0067] The motion trajectory is determined for the current video frame based on a reference motion vector used for coding one or more reference frames of the current video frame. For example, the reference motion vector may be a motion vector indicating or otherwise used to predict motion between one of the reference frames usable to predict motion of the current video block and a reference frame used to encode or decode one or more video blocks in that one of the reference frames. For example, the order of those reference frames for predicting motion therebetween can be indicated to an encoder based onan input video stream and to a decoder based on frame offset values. Based on that order, a linear interpolation can be performed using the reference motion vector against a reference frame usable for encoding or decoding the current video block and a reference frame used for encoding or decoding that reference frame.

[0068] Performing the linear interpolation against those frames can include projecting a group of pixels of one reference frame from a location of that reference frame to a location within another reference frame. For example, a 4x4 or 8x8 block of pixels located at a position within a first reference frame can be projected to a position within a second reference frame. The first reference frame is a reference frame used to encode or decode one or more video blocks of the second reference frame, and the second reference frame is a reference frame usable to predict motion of a portion of the current video frame. For example, the first reference frame may be a past frame and the second reference frame may be a future frame. There may be one or more other reference frames intermediate to the first reference frame and the current video frame. Examples of motion trajectories are described below with respect to FIGS. 8 and 9.

[0069] Determining the reference motion vector can include processing regions of a first reference frame usable to predict motion within the current video block. For example, processing the regions of the first reference frame can include comparing different groups of pixel values (e.g., 4x4 blocks, 8x8 blocks, or the like) of the first reference frame to same- dimensioned groups of pixel values in a second reference frame used to predict the motion of the first reference frame. That is, in contrast to the description above, the second reference frame is a reference frame in this example used to encode or decode one or more video blocks of the first reference frame, and the first reference frame is a reference frame usable to prediction motion of the current video frame. In either example, encoding or decoding a current frame may first include processing some or all 4x4, 8x8, or other-dimensioned blocks within some or all reference frames usable to predict motion within that current frame.

[0070] After determining the reference motion vector, the reference motion vector (RefMV_row, RefMV_col) is projected to the current video frame to determine where the motion trajectory fits within the current video frame. The location of the current video frame at which the motion trajectory fits the current video frame is the location of the projected block position (Block_row, Block_col) as calculated below continuing with the latter example above (i.e., where the second reference frame is a reference frame used to encode or decode one or more video blocks of the first reference frame, and the first reference frame is a reference frame usable to prediction motion of the current video frame).

[0071] Block_row = RefMV.row * (IstRefFrame - CurrFrame) / (IstRefFrame - 2ndRefFrame)

[0072] Block_col = RefMV.col * (IstRefFrame - CurrFrame) / (IstRefFrame - 2ndRefFrame).

[0073] In the above, the variable IstRefFrame represents the frame index of the first reference frame, the variable CurrFrame represents the frame index of the current video frame, and the variable 2ndRefFrame represents the frame index of the second reference frame.

[0074] Thereafter, the motion trajectory is determined for the projected block position (Block_row, Block_col) by estimating its motion vector (MV) towards one or more reference frames, such as each of the available reference frames for the current frame (also called the current coding frame). Estimating the motion vector includes performing a linear interpolation against a reference frame and the current video frame based on the location of the projected block within the current video frame. More specifically, using the same notation as above, the motion vector is estimated towards the first reference frame (MV_l.row, MV_l.col) according to the following equations.

[0075] MV_l.row = RefMV.row * (CurrFrame - IstRefFrame) / (IstRefFrame - 2ndRefFrame)

[0076] MV_l.col = RefMV.col * (CurrFrame - IstRefFrame) / (IstRefFrame - 2ndRefFrame).

[0077] Multiple motion vectors for the current projected block can be determined in this way by determining the reference motion vector of a respective reference frame and applying these equations to project the resulting motion vector towards the frames individually. Desirably, a defined sequence of reference frames is used, such as starting with the nearest frame in the display order to determine the projected motion vector for the block. The projected motion vector may be, for example, the first reference frame in the test sequence that returns a reference motion vector.

[0078] At operation 604, a query is made as to whether any further blocks of the reference frame (in this example, the first reference frame) are present. If further blocks are present, the technique 600 can return to operation 602 to process the next block to determine a motion trajectory based on a new reference motion vector, if any, determined as described above.

[0079] In summary, all the blocks within one or more reference frames may be used to estimate the motion field for the current frame in the unit of the blocks (e.g., a 4x4 block).Even so, there will most often exist at least one block in the current frame that does not have any motion vectors estimated / projected back towards a reference frame. Blocks in the current frame that have a motion vector projected towards the reference frame may be used as a proper subset of blocks of the current frame to determine the motion field for the current frame. Blocks without a motion vector projected towards the reference frame may be considered holes in the motion field for the current frame. As one example, this can occur when there is no reference motion vector in the reference frame (e.g., the block in the reference frame is intra predicted).

[0080] Holes in the motion field may be treated as unavailable and not used for motion vector prediction. Alternatively, filling the holes in the motion field can increase the number of blocks in a frame that have motion vector candidates determined using the motion field, improving prediction quality. Thus, the technique 600 fills the holes in the motion field at operation 606. The holes in the motion field may be filled by reusing the motion vector projected towards the reference frame for a spatially neighboring block, for example, the spatially nearest available projected motion vector. The spatially nearest available projected motion vector may be determined using Manhattan distance. If there are multiple reference spots that share the same distance, a predetermined pattern (a defined order shared with the decoder) can be used such that whichever comes first may be used as the projected motion vector to fill the hole. According to one example, the pattern is top-left — top — top-right — left — right — bottom-left — bottom — bottom-right.

[0081] After filling the holes in the motion field, an optional filtering process may be applied to the motion field. Once the motion field is complete, it can be used for motion vector referencing, i.e., for determining a motion vector candidate list and predicting the motion vectors at the coding block level as discussed in more detail below.

[0082] At operation 608, the technique 600 determines whether to apply filtering. If filtering is to be applied, the technique 600 filters the motion field at operation 610. Filtering the motion field may depend on whether the motion vector of a block of the current coding frame is obtained from projection or from hole filling. For example, where the motion vector for a block is determined at operation 606 (i.e., obtained by filling a hole in the motion vector field), filtering includes determining an average motion vector by averaging the projected motion vector with motion vectors from surrounding neighbor blocks and replacing the motion vector with the average motion vector. If the motion vector for the block is that determined at operation 602 (i.e., obtained through estimation / projection of a reference motion vector), the motion vector remains unchanged.

[0083] Whether to apply filtering or not in response to the operation 608 is performed differently at the encoder than at a decoder. At an encoder, for example the encoder 400, the encoder can decide whether to apply filtering by filtering the motion field and comparing it to the unfiltered motion field. For example, motion vectors can be selected from a motion vector candidate list for each of the filtered motion field and the unfiltered motion field for encoding respective motion vectors of the blocks of the current coding frame. The motion field that minimizes the differences between the motion vectors of the blocks of the current coding frame and the motion vector candidates is determined, and the encoder can send a bit to the decoder to indicate to apply filtering if the filtered motion field minimizes the differences or to not apply filtering if the unfiltered motion field minimizes the differences. At the encoder, the differences can be determined as the sum of absolute differences (SAD), the sum of squared errors (SSR), or some other difference measure. Calculations can be reduced by only determining the differences for those blocks that will change based on the filtering, i.e., the filled holes. The bit can be sent to the decoder in, for example, a frame header. At a decoder, for example the decoder 500, the bit can be read from the encoded bitstream to determine whether to apply filtering at operation 608.

[0084] Whether the motion field is filtered or not at operation 610, the process advances to use the motion field to predict the motion vectors at the coding block level. This includes, at operation 612, determining the motion vector candidate list for predicting motion within a current video block using the motion field. The motion vector candidate list can include one or more motion vector candidates determined based on the motion field. The motion vector candidate list can include one or more motion vectors from a collocated block (i.e., a block from the same location in a previously coded frame), although desirably motion vectors from collocated blocks can be omitted in favor of the motion vectors determined from the motion field.

[0085] In some implementations, the technique 600 includes one or more spatial motion vector candidates based on one or more neighbor blocks of the current block in the motion vector candidate list. For example, the one or more spatial motion vector candidates can include one or more motion vectors determined using intra prediction. For example, where the blocks of the video frame including the current video block are scanned in a raster order, a spatial motion vector candidate can be determined based on motion predicted using an above-neighbor block of the current video block, a left-neighbor block of the current video block, or the like, or a combination thereof.

[0086] The one or more spatial motion vector candidates may be organized into two different categories. A first category includes spatial motion vector candidates determined using one or more neighbor blocks immediately adjacent to the current video block. A second category includes spatial motion vector candidates determined using one or more neighbor blocks surrounding, but not immediately adjacent to, the current video block. For example, a spatial motion vector candidate in the second category may be derived using a block that is two blocks away from the current video block.

[0087] Spatial motion vectors in the first category may be prioritized over those in the second category. For example, if two different spatial motion vector candidates may both result in the same performance for predicting motion in the current video block, but each was from a different one of the two categories, the spatial motion vector candidate of the first category would be selected over the other.

[0088] Separate lists may be generated for each reference frame, or one list may be generated for a block that does not depend upon the reference frame(s) used for inter prediction of the block. The technique used for generating the motion vector candidate list is not critical so long as the same technique is used at the encoder and decoder.

[0089] At operation 614, a reference motion vector to use for encoding or decoding the motion vector for the current coding block is selected. The reference motion vector may be selected at an encoder by determining rate-distortion values resulting from predicting motion within the current video block using motion vector candidates of the motion vector candidate list. A rate-distortion value refers to a ratio that balances an amount of distortion (e.g., a loss in video quality) with rate (e.g., a number of bits) for coding a block or other video component. The rate-distortion values may, for example, be determined by performing a ratedistortion analysis against ones of the motion vector candidates of the motion vector candidate list. As a result of the rate-distortion analysis, the one of the motion vector candidates used to determine a lowest one of the rate-distortion values is identified, such as by comparing respective ones of the rate-distortion values. Stated differently, the motion vector candidate that most efficiently encodes the motion vector is selected as the reference motion vector.

[0090] At operation 618, the current coding block is encoded or decoded using the reference motion vector. The motion vector for the block may be a motion vector determined before the technique 600 through rate-distortion processing at an inter / intra prediction stage of the encoder. After the processing at operation 612, the motion vector for the block may be the reference motion vector. For example, the reference motion vector may replace apreviously generated motion vector for the block if the reference motion vector closely matches that original motion vector. Encoding or decoding can include generating a prediction block as a prediction residual based on a difference between pixel values of the current video block and pixel values of the prediction block obtained using the motion vector. For example, during operations for encoding the current coding block, the prediction residual can be generated using operations performed by the encoder 400 shown in FIG. 4 (e.g., at the intra / inter prediction stage 402). In another example, during operations for decoding the current video block, the prediction residual can be generated using operations performed by the decoder 500 shown in FIG. 5 (e.g., at the intra / inter prediction stage 508).

[0091] Thereafter, operations for encoding the current video block can include transforming, quantizing, and entropy encoding the prediction residual to an encoded bitstream (e.g., the compressed bitstream 420 shown in FIG. 4). In another example, operations for decoding the current video block can include reconstructing the video block based on the prediction residual, filtering the reconstructed video block, and outputting the filtered video block to a video stream (e.g., the output video stream 516 shown in FIG. 5). The motion vector, which is required to reconstruct the current block at a decoder is encoded into the bitstream by, for example, including an index to the motion vector candidate list that identifies the reference motion vector. Where the reference motion vector is used to differentially code the motion vector instead of being used as the motion vector itself, operations for encoding may include determining a residual motion vector as the difference between the two motion vectors and entropy encoding the residual motion vector to the bitstream. Operations for decoding may include reading the index indicating the reference motion vector (e.g., from the block header), selecting the reference motion vector from the motion vector candidate list, and, if necessary, reconstructing the motion vector from the motion vector residual and the reference motion vector.

[0092] In some implementations, the technique 600 includes encoding one or more further syntax elements to a bitstream to which the current block is encoded. For example, when the technique 600 is performed during operations for encoding a current video block, the one or more syntax elements encoded to the bitstream may indicate a display order of a video sequence including the current frame and the one or more reference frames. The one or more syntax elements may, for example, be encoded to a frame header of the current frame including the current video block. For example, the one or more syntax elements used to indicate the display order of the video sequence can include four, five, or some other number of bits for indicating frame offset values of the video frames of the video sequence.

[0093] Although an index is described as the syntax element to identify the reference motion vector, one or more syntax elements used to indicate the selected motion vector can include a number of bits for indicating a location within a reference frame of a block of pixels corresponding to the reference motion vector. The number of bits for the index can be based on the total number of motion vector candidates included in the motion vector candidate list. For example, when the motion vector candidate list includes two candidates, one bit can be used to indicate the reference motion vector (e.g., where each of the motion vector candidates is associated with one of the values of the bit, or by using a value of 1 to indicate the location of the block of pixels including the motion indicated by the selected motion vector). In another example, when the motion vector candidate list includes three candidates, two bits can be used.

[0094] In yet another example, one or more syntax elements used to indicate the selected motion vector can indicate context information associated with the reference motion vector. For example, when the reference motion vector is a spatial motion vector, the one or more syntax elements may indicate whether that reference motion vector was derived from an above-neighbor block, a left-neighbor block, or a block surrounding but not immediately adjacent to the current coding block. In another example, the one or more syntax elements may indicate whether the reference motion vector was derived from a first category of spatial motion vector candidates or a second category of spatial motion vector candidates.

[0095] At least one of the one or more syntax elements indicates a frame offset value for the current frame, which frame offset value indicates a location of the current frame in a video sequence. For example, the syntax elements indicating the frame offset value can include a number of bits based on the number of look-ahead frames used by the encoder and decoder. For example, when the number of look- ahead frames is 16, four bits may be used to indicate the frame offset value. In another example, when the number of look-ahead frames is 32, five bits may be used to indicate the frame offset value.

[0096] In some implementations, the technique 600, when performed during operations for decoding a current video block, includes decoding one or more syntax elements from a bitstream including an encoded frame to decode. For example, the one or more syntax elements decoded from the bitstream may indicate a frame offset value for the encoded video frame. In another example, the one or more syntax elements decoded from the bitstream may indicate a motion vector selected for encoding the encoded block during operations for encoding the block.

[0097] FIG. 7 is an illustration of an example of data used to indicate a display order of video frames of a video sequence. In particular, a table 700 includes a coding order, a frame index, a number of packet stamps, and a frame offset value for each of a plurality of video frames of a video sequence to encode or decode. The coding order includes spaces between codings of frames, such as between 5 and 6 and after 8. These spaces indicate when an alternative reference frame is encoded or decoded, as applicable. The frame index indicates the alternative reference frame used for those spaces. For example, the video frame located 3rd in the display order of the video sequence is encoded or decoded, as applicable, between 5 and 6 in the coding order. There is no frame offset value for these spaces since there is no video frame to display.

[0098] FIG. 8 is a diagram of an example of using a motion trajectory for a current frame 800 to predict motion of a current block 802. The motion trajectory shows motion between the current frame 800 and three reference frames, a first reference frame 804, a second reference frame 806, and a third reference frame 808. The motion trajectory is determined based on a reference motion vector 810, which indicates motion between the second reference frame 806 and the first reference frame 804. For example, the second reference frame 806 may be a reference frame used to predict motion of one or more blocks of the first reference frame 804. After the reference motion vector 810 is determined, the motion trajectory is determined as the trajectory corresponding to the direction of the reference motion vector 810.

[0099] The motion trajectory identifies the current block 802 as the location of the current frame 800 intersected by the motion trajectory. A first temporal motion vector candidate 812 may then be determined as indicating motion between the first reference frame 804 and the current frame 800. A second temporal motion vector candidate 814 may be determined as indicating motion between the third reference frame 808 and the current frame 800. One or more of the reference motion vector 810, the first temporal motion vector candidate 812, or the second temporal motion vector candidate 814 may be included in a motion vector candidate list from which a motion vector is selected for predicting motion of the current block 802.

[0100] The second reference frame 806 and the third reference frame 808 are shown as past frames with respect to the current frame 800. The first reference frame 804 is shown as a future frame with respect to the current frame 800. However, other numbers of past or future reference frames may be used. For example, a motion trajectory can be determined where there is one past reference frame and one future reference frame. In another example, amotion trajectory can be determined where there is one past frame and two future reference frames. In another example, a motion trajectory can be determined where there are two or more past reference frames and two or more future reference frames.

[0101] FIG. 9 is a diagram of an example of using multiple motion trajectories to predict motion of a current block. The motion trajectory shown in and described with respect to FIG. 8 has a constant slope from the past-most reference frame to the future-most reference frame and thus indicates that the motion between the video frames thereof occurs at a constant rate. However, this may not always be the case, such as where motion speeds up or slows down between video frames.

[0102] A current frame 900 includes a current block 902 to encode or decode. When the motion is very slow, a collocated block 904 in a first reference frame 906, that precedes the current frame 900, may be used to predict motion of the current block 902. However, if the motion is not slow enough, the corresponding motion trajectory may not intersect the current block 902 such that the collocated block 904 may not be used. For example, a first motion trajectory 908 from a second reference frame 910, that precedes the first reference frame 906, to a third reference frame 912, that succeeds the current frame 900, intersects the current frame 900 slightly above the current block 902.

[0103] In such a case, a motion field estimate for the current block 902 can be used, such as to identify motion from the collocated block 904 within an extended region of the current block 902 (e.g., a group of one or more 4x4, 8x8, or other-dimensioned blocks surrounding the current block 902 within the current frame 900). For example, the extended region may be defined based on the locations within the current frame 900 of objects from one or more of the first reference frame 906, the second reference frame 910, or the third reference frame 912, such as where the motion of those objects is indicated using one or more temporal motion vector candidates.

[0104] The motion field estimate may, for example, reflect the locations of all motion within the current frame 900 with respect to the available reference frames. For example, the motion field estimate may be used to determine a second motion trajectory 914 from the second reference frame 910 to the third reference frame 912. The second motion trajectory 914 intersects the current frame 900 at the current block 902. As such, a motion vector candidate associated with the second motion trajectory 914 may more accurately predict motion of the current block 902 itself than a motion vector candidate associated with the first motion trajectory 908.

[0105] Implementations of this disclosure include using a (e.g., linear) projection model to create a motion field estimation that can estimate different temporal motion trajectories, including those with high velocity. The resulting motion vectors are used as motion vector candidates in a dynamic motion vector referencing system to augment any motion vector candidates from spatial neighbors and optionally from a collocated block to the current block being coded (also called the current coding block). However, many coding blocks of the frame will not have a motion vector within the motion field, which reduces the usefulness of the motion vector referencing for the frame as a whole. The techniques herein describe projected motion field hole filling, which can increase the number of blocks of a frame that benefit from motion vector candidates determined using motion field estimation and hence improve motion vector referencing.

[0106] The techniques allow the codec to better track complex motion activities than spatial neighbors, which may fail to keep synchronized. As a result, compression performance can be improved compared to using spatial motion vectors from spatial neighbors alone or using those from spatial neighbors and a collocated block.

[0107] The aspects of encoding and decoding described above illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding, as those terms are used in the claims, could mean compression, decompression, transformation, or any other processing or change of data.

[0108] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, use of the word “example” is intended to present concepts in a concrete fashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise or clearly indicated otherwise by the context, the statement “X includes A or B” is intended to mean any of the natural inclusive permutations thereof. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more,” unless specified otherwise or clearly indicated by the context to be directed to a singular form. Moreover, use of the term “an implementation” or the term “one implementation” throughout this disclosure is not intended to mean the same embodiment or implementation unless described as such.

[0109] Implementations of the transmitting station 102 and / or the receiving station 106 (and the algorithms, methods, instructions, etc., stored thereon and / or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0110] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general-purpose computer or general-purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms, and / or instructions described herein. In addition, or alternatively, for example, a special purpose computer / processor can be utilized that can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein.

[0111] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can be implemented on a device separate from the server, such as a handheld communications device. In this instance, the transmitting station 102, using an encoder 400, can encode content into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communications device, and / or a device including an encoder 400 may also include a decoder 500.

[0112] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport the programfor use by or in connection with any processor. The medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0113] The above-described embodiments, implementations, and aspects have been described to facilitate easy understanding of this disclosure and do not limit this disclosure. On the contrary, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation as is permitted under the law to encompass all such modifications and equivalent arrangements.

Claims

What is claimed is:

1. A method for coding a current block of a current frame, the method comprising: determining a motion field for the current frame using a proper subset of blocks of the current frame, the motion field comprising a respective motion vector projecting onto a reference frame for each block of the proper subset of blocks; for a block of the current frame without a motion vector in the motion field, reusing the respective motion vector for a spatially neighboring block of the proper subset of blocks as a projected motion vector for the block within the motion field; determining a motion vector candidate list using the motion field; selecting a reference motion vector for the current block from the motion vector candidate list; and coding the current block using the reference motion vector.

2. The method of claim 1, wherein coding the current block comprises encoding the current block into an encoded bitstream using the reference motion vector.

3. The method of claim 2, comprising: encoding a syntax element into the encoded bitstream identifying the reference motion vector.

4. The method of one of claim 2 or claim 3, comprising: encoding a syntax element into the encoded bitstream identifying whether to filter the motion field before determining the motion vector candidate list.

5. The method of claim 1, wherein coding the current block comprises decoding the current block from an encoded bitstream using the reference motion vector.

6. The method of claim 5, comprising: decoding a syntax element from the encoded bitstream identifying the reference motion vector.

7. The method of one of claim 5 or claim 6, comprising:decoding a syntax element in the encoded bitstream identifying whether to filter the motion field before determining the motion vector candidate list.

8. The method of any one of claims 1 to 7, wherein reusing the respective motion vector for the spatially neighboring block comprises reusing the respective motion vector for a spatially nearest available neighboring block determined using Manhattan distance.

9. The method of any one of claim 1 to 8, comprising: filtering the motion field before determining the motion vector candidate list.

10. The method of claim 9, wherein filtering the motion field comprises: replacing the projected motion vector with an average motion vector generated from multiple motion vectors; and leaving the respective motion vector for each block of the proper subset of blocks unchanged.

11. The method of claim 10, wherein the multiple motion vectors comprise the projected motion vector and the respective motion vector for at least one spatially neighboring block.

12. An apparatus for coding a current block of a current frame, comprising: a processor configured to perform the method of any one of claims 1 to 11.

13. An apparatus for coding a current block of a current frame, comprising: a processor; and a non-transitory, computer-readable storage medium storing instructions for causing the processor to perform the method of any one of claims 1 to 11.

14. A non-transitory, computer-readable storage medium storing an encoded bitstream comprising an encoded residual for a current block, the encoded residual encoded by determining a motion field for multiple blocks of the current frame using a proper subset of blocks of the current frame, the motion field comprising a respective motion vector projecting onto a reference frame for each block of the multiple blocks projecting onto a reference frame proper subset of blocks, for a block of the current frame without a motionvector in the motion field, reusing the respective motion vector for a spatially neighboring block of the proper subset of blocks as a projected motion vector for the block within the motion field, determining a motion vector candidate list using the motion field, selecting a reference motion vector for the current block from the motion vector candidate list, and encoding the current block into the encoded bitstream using the reference motion vector.

15. The non-transitory, computer-readable storage medium of claim 14, wherein the encoded bitstream comprises: a first syntax element identifying the reference motion vector; and a second syntax element identifying whether to filter the motion field before determining the motion vector candidate list.

Images