Projected motion field hole filling for motion vector reference

Abstract

Projected motion field hole filling includes determining a motion field of a current frame using a block true subset of the current frame, the motion field including, for each block in the block true subset, a respective motion vector projected onto a reference frame. For a block in the current frame that does not have a motion vector in the motion field, a respective motion vector of a spatial domain neighboring block of the proper subset of the block is reused as a projected motion vector of the block within the motion field. A list of motion vector candidates may be determined using the motion field. A reference motion vector for the current block may be selected from the motion vector candidate list. The current block may be encoded into an encoded bitstream or decoded from an encoded bitstream using the reference motion vector.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to U.S. Provisional Patent Application No. 63 / 535,276, filed August 29, 2024, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Digital video streams can be represented using sequences of frames or still images. Digital video can be used in a variety of applications, including video conferencing, high-definition video entertainment, video advertising, or the sharing of user-generated videos. Digital video streams can contain large amounts of data and consume significant computational or communication resources of computing devices for processing, transmitting, or storing the video data. Various methods have been proposed to reduce the amount of data in video streams, including compression techniques in encoding and decoding. Summary of the Invention

[0004] One aspect taught in this paper is a method for encoding and decoding a current block of a current frame. The method includes: determining a motion field of the current frame using a proper subset of blocks in the current frame, the motion field including corresponding motion vectors projected onto a reference frame for each block in the proper subset; for blocks in the current frame that do not have motion vectors in the motion field, reusing the corresponding motion vectors of spatially neighboring blocks of the proper subset of the block as projected motion vectors for that block within the motion field; determining a candidate list of motion vectors using the motion field; selecting a reference motion vector for the current block from the candidate list; and encoding and decoding the current block using the reference motion vector, such as encoding the current block into an encoded bitstream using the reference motion vector.

[0005] Another aspect taught in this paper is a method for encoding and decoding a current block of a current frame. The method includes: determining a motion field of the current frame using a proper subset of blocks in the current frame, the motion field including corresponding motion vectors projected onto a reference frame for each block in the proper subset; for blocks in the current frame that do not have motion vectors in the motion field, reusing the corresponding motion vectors of spatially neighboring blocks in the proper subset of the block as projected motion vectors for that block within the motion field; determining a candidate list of motion vectors using the motion field; selecting a reference motion vector for the current block; and encoding and decoding the current block using the reference motion vector, such as decoding the current block from an encoded bitstream using the reference motion vector.

[0006] When encoding or decoding the current block involves encoding the current block itself, syntax elements identifying reference motion vectors can be encoded into the encoded bitstream. Syntax elements indicating whether the motion field is filtered before determining the motion vector candidate list can also be encoded into the encoded bitstream.

[0007] When decoding the current block, syntax elements identifying reference motion vectors can be decoded from the encoded bitstream. Syntax elements indicating whether the motion field is filtered before determining the candidate list of motion vectors can also be decoded from the encoded bitstream.

[0008] Reusing the corresponding motion vector of a neighboring block in the airspace may include reusing the corresponding motion vector of the nearest available neighboring block in the airspace, determined using Manhattan distance.

[0009] The motion field can be filtered before determining the candidate list of motion vectors. Filtering the motion field may include: replacing the projected motion vector with an average motion vector generated from multiple motion vectors; and keeping the corresponding motion vector of each block in the proper subset of the block unchanged. In these implementations, the multiple motion vectors may include the projected motion vector and the corresponding motion vector of at least one spatially neighboring block.

[0010] Another aspect of the teachings herein is a device for encoding or decoding a current block of a current frame, the device including a processor. The processor is configured to perform any of the methods described herein. For example, a memory may store instructions that, when executed, cause the processor to perform any of the methods described herein.

[0011] Another aspect of the teachings herein is a non-transitory computer-readable storage medium storing instructions for performing any of the methods described herein.

[0012] Another aspect of the teachings herein is a non-transitory computer-readable storage medium storing an encoded bitstream. The encoded bitstream includes an encoded residual of a current block, which is encoded by: determining a motion field for a plurality of blocks of the current frame using a proper subset of blocks of the current frame, the motion field including a corresponding motion vector projected onto the reference frame for each of the plurality of blocks projected onto the proper subset of blocks of the reference frame; for blocks in the current frame that do not have a motion vector in the motion field, reusing the corresponding motion vector of the spatially neighboring blocks of the plurality of proper subsets of blocks as the projected motion vector of that block within the motion field; determining a motion vector candidate list using the motion field; selecting a reference motion vector for the current block from the motion vector candidate list; and encoding the current block into the encoded bitstream using the reference motion vector.

[0013] The encoded bitstream may include: a first syntax element that identifies the reference motion vector and / or a second syntax element that identifies whether the motion field is filtered before determining the candidate list of motion vectors.

[0014] These and other aspects of this disclosure are made public in the following detailed description of the embodiments, the appended claims, and the accompanying drawings. Attached Figure Description

[0015] The description herein refers to the accompanying drawings, wherein the same reference numerals are used throughout several views to refer to the same parts.

[0016] Figure 1 This is a schematic diagram of a video encoding and decoding system.

[0017] Figure 2 This is a block diagram of an example computing device that can implement a sending station or a receiving station.

[0018] Figure 3 This is a diagram of a typical video stream that is to be encoded and subsequently decoded.

[0019] Figure 4 This is a block diagram of an encoder according to the implementation of this disclosure.

[0020] Figure 5 This is a block diagram of a decoder implemented according to this disclosure.

[0021] Figure 6 This is a flowchart illustrating an example of a technique for encoding or decoding using projected motion field hole filling for motion vector reference.

[0022] Figure 7 This is an illustration of an example of data used to indicate the display order of video frames in a video sequence.

[0023] Figure 8 This is an example of using the motion trajectory of the current frame to predict the motion of the current block.

[0024] Figure 9 This is an example of using multiple motion trajectories to predict the motion of the current block. Detailed Implementation

[0025] Video compression schemes may include breaking down a corresponding image or video frame into smaller parts such as video blocks, and using techniques to limit the information included in each video block to generate an encoded bitstream. The encoded bitstream can be decoded to reconstruct the source image based on the limited information. Encoding or decoding a video block may include, for example, predicting motion within that video block relative to one or more other video blocks in the same or different video frames.

[0026] Predicting motion within a video block becomes complex due to variations in motion speed, such as across multiple video frames in a video sequence. For example, an object moving between a series of consecutive frames might move faster in the first half of the series and slower in the second half. Information indicating the display order of the series of video frames can help determine which frames are available for predicting motion within a given video block; however, this information is typically not provided to the decoder. Furthermore, multiple distinct motion directions may exist between those frames, making it often difficult to efficiently identify the possible motion vectors for predicting the video block's motion and select the optimal one for prediction.

[0027] Implementations of this disclosure include using (e.g., linear) projection models to create motion field estimates that can estimate different temporal motion trajectories, including those with high velocities. The resulting motion vectors are used as motion vector candidates in a dynamic motion vector reference system to augment any motion vector candidates from spatial neighbors, and optionally from co-located blocks that are being encoded (also referred to as the current codec block). However, many codec blocks in the frame will lack motion vectors within the motion field, reducing the usefulness of the motion vector reference for the frame as a whole. The techniques described herein describe projected motion field hole filling, which can increase the number of blocks in the frame that benefit from motion vector candidates determined using the motion field estimates, thereby improving the motion vector reference. Motion vector candidates can be used to encode the motion vectors of the current codec block, as described in more detail below after the initial reference to the system that implements the teachings herein.

[0028] Figure 1 This is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 may have, for example, such as... Figure 2 The computer described has an internal hardware configuration. However, other implementations of the sending station 102 are possible. For example, the processing of the sending station 102 can be distributed among multiple devices.

[0029] Network 104 can connect sending station 102 and receiving station 106 for encoding and decoding of video streams. Specifically, the video stream can be encoded in sending station 102 and decoded in receiving station 106. Network 104 can be, for example, the Internet. Network 104 can also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular telephone network, or any other component that transmits video streams from sending station 102 to (in this example) receiving station 106.

[0030] In one example, receiving station 106 could be such as Figure 2The described computer has an internal hardware configuration. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0031] Other implementations of the video encoding and decoding system 100 are possible. For example, network 104 may be omitted in the implementation. In another implementation, the video stream may be encoded and then stored for later transmission to receiving station 106 or any other device with memory. In one implementation, receiving station 106 receives (e.g., via network 104, a computer bus, and / or some communication path) the encoded video stream and stores it for later decoding. In an example implementation, the encoded video is transmitted via network 104 using the Real-Time Transport Protocol (RTP). In another implementation, a transport protocol different from RTP may be used, such as a video streaming protocol based on the Hypertext Transfer Protocol (HTTP).

[0032] When used in a video conferencing system, for example, sending station 102 and / or receiving station 106 may include the capability to both encode and decode video streams as described below. For example, receiving station 106 may be a video conferencing participant who receives an encoded video bitstream from a video conferencing server (e.g., sending station 102) for decoding and viewing, and further encodes his or her own video bitstream and transmits it to the video conferencing server for other participants to decode and view.

[0033] Figure 2 This is a block diagram illustrating an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement... Figure 1 One or both of the transmitting station 102 and the receiving station 106. The computing device 200 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device (e.g., a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.).

[0034] The processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device or multiple devices that exist now or are developed later and are capable of manipulating or processing information. For example, although the disclosed implementation can be practiced with a single processor (e.g., processor 202) shown, advantages in speed and efficiency can be achieved by using more than one processor.

[0035] In this implementation, the memory 204 in the computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. However, other suitable types of storage devices may be used as memory 204. Memory 204 may include code and data 206 accessed by the processor 202 using bus 212. Memory 204 may also include an operating system 208 and an application program 210, which includes at least one program that permits the processor 202 to execute the techniques described herein. For example, application program 210 may include applications 1 to N, which may also include video encoding / decoding applications that execute the techniques described herein. The computing device 200 may also include secondary storage 214, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may contain a considerable amount of information, it may be stored, in whole or in part, in secondary storage 214 and loaded into memory 204 as needed for processing.

[0036] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 218 may be coupled to the processor 202 via a bus 212. In addition to or as an alternative to the display 218, other output devices may be provided that allow a user to program or otherwise use the computing device 200. When the output device is a display or includes a display, the display may be implemented in various ways, including via a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light-emitting diode (LED) display (such as an organic LED (OLED) display).

[0037] The computing device 200 may also include an image sensing device 220, such as a camera or any other existing or later-developed image sensing device 220 capable of sensing images (such as images of a user operating the computing device 200), or communicating with such image sensing device. The image sensing device 220 may be positioned such that it is pointed toward the user operating the computing device 200. In an example, the position and optical axis of the image sensing device 220 may be configured such that the field of view includes an area directly adjacent to and visible from the display 218.

[0038] The computing device 200 may also include a sound sensing device 222, such as a microphone or any other sound sensing device that can sense the present or future presence of sound in the vicinity of the computing device 200, or communicate with such sound sensing device. The sound sensing device 222 may be positioned such that it is directed toward a user operating the computing device 200, and may be configured to receive sounds, such as speech or other words, emitted by the user when the user operates the computing device 200.

[0039] although Figure 2 The processor 202 and memory 204 of computing device 200 are depicted as integrated into a single unit, but other configurations may be utilized. The operation of processor 202 can be distributed across multiple machines (where individual machines may have one or more processors), which may be directly coupled or coupled across a local area network or other network. Memory 204 can be distributed across multiple machines, such as network-based memory or memory in multiple machines performing the operations of computing device 200. Although depicted herein as a single bus, bus 212 of computing device 200 may consist of multiple buses. Furthermore, auxiliary storage 214 may be directly coupled to other components of computing device 200 or accessible via a network, and may include integrated units (such as memory cards) or multiple units (such as multiple memory cards). Therefore, computing device 200 can be implemented in a wide variety of configurations.

[0040] Figure 3 This is an illustration of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes multiple adjacent frames 304. Although three frames are depicted as adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. Adjacent frames 304 can then be further subdivided into individual frames, such as frame 306. At the next level, frame 306 can be divided into a series of planes or segments 308. For example, segment 308 can be a subset of frames that allow for parallel processing. Segment 308 can also be a subset of frames that can separate video data into individual colors. For example, frame 306 of color video data can include a luminance plane and two chroma planes. Segment 308 can be sampled at different resolutions.

[0041] Regardless of whether frame 306 is divided into segments 308, frame 306 can be further subdivided into blocks 310, which can contain data corresponding to, for example, 16×16 pixels in frame 306. Block 310 can also be arranged to include data from one or more segments 308 of pixel data. Block 310 can also be any other suitable size, such as 4×4 pixels, 8×8 pixels, 16×8 pixels, 8×16 pixels, 16×16 pixels, or larger. Unless otherwise stated, the terms block and macroblock are used interchangeably herein.

[0042] Figure 4 This is a block diagram of an encoder 400 according to an implementation of this disclosure. As described above, the encoder 400 may be implemented in a transmitting station 102, such as by providing a computer software program stored in memory (e.g., memory 204). The computer software program may include machine instructions that, when executed by a processor (such as processor 202), cause the transmitting station 102 to... Figure 4 The video data is encoded in the manner described herein. The encoder 400 can also be implemented as dedicated hardware included, for example, in the transmitting station 102. In a particularly desirable implementation, the encoder 400 is a hardware encoder.

[0043] Encoder 400 has the following stages for performing various functions in the forward path (shown by solid connecting lines) to produce an encoded or compressed bitstream 420 using video stream 300 as input: intra / inter-frame prediction stage 402, transform stage 404, quantization stage 406, and entropy coding stage 408. Encoder 400 may also include a reconstruction path (shown by dashed connecting lines) for reconstructing frames for encoding future blocks. Figure 4 In the encoder 400, the following stages are used to perform various functions in the reconstruction path: dequantization stage 410, inverse transform stage 412, reconstruction stage 414, and loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0044] When video stream 300 is presented for encoding, corresponding adjacent frames 304, such as frame 306, can be processed in blocks. At the intra / inter-frame prediction level 402, the corresponding blocks can be encoded using intra-frame prediction or inter-frame prediction. In either case, prediction blocks can be formed. In the case of intra-frame prediction, prediction blocks can be formed from samples that have been previously encoded and reconstructed in the current frame. In the case of inter-frame prediction, prediction blocks can be formed from samples in one or more previously constructed reference frames.

[0045] Next, the prediction block can be subtracted from the current block at the intra / inter-frame prediction stage 402 to produce a residual block (also known as the residual). The transform stage 404 uses a block-based transform to transform the residual into transform coefficients, for example, in the frequency domain. The quantization stage 406 uses a quantizer value or quantization level to convert the transform coefficients into discrete quantum values, referred to as quantized transform coefficients. For example, the transform coefficients can be divided by the quantizer value and truncated.

[0046] The quantized transform coefficients are then entropy encoded by entropy coding level 408. The entropy-encoded coefficients, along with other information for decoding the block (which may include, for example, syntax elements indicating the prediction type, transform type, motion vector, quantizer values, etc.), are then output to a compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable-length codec (VLC) or arithmetic codec. The compressed bitstream 420 may also be referred to as an encoded video stream or an encoded video bitstream, and the terms will be used interchangeably herein.

[0047] The reconstruction path (shown by the dashed connection line) can be used to ensure encoder 400 and (see below for details) Figure 5 The decoder 500 (described below) uses the same reference frame to decode the compressed bitstream 420. The reconstruction path is performed in accordance with (as described below regarding...) Figure 5 Similar functionalities occur during the decoding process (described below), including dequantizing the quantized transform coefficients at dequantization stage 410 and performing an inverse transform on the dequantized transform coefficients at inverse transform stage 412 to produce a derived residual block (also referred to as the derived residual). At reconstruction stage 414, the predicted block predicted at intra / inter-frame prediction stage 402 is added to the derived residual to create a reconstructed block. Loop filtering stage 416 can be applied to the reconstructed block to reduce distortion, such as blocking artifacts.

[0048] Other variations of encoder 400 can be used to encode the compressed bitstream 420. In some implementations, for certain blocks or frames, a non-transform-based encoder can directly quantize the residual signal without the transform stage 404. In some implementations, the encoder may have a quantization stage 406 and a dequantization stage 410 combined in a common stage.

[0049] Figure 5 This is a block diagram of a decoder 500 according to an implementation of this disclosure. The decoder 500 can be implemented in a receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program may include machine instructions that, when executed by a processor (such as processor 202), cause the receiving station 106 to... Figure 5The video data is decoded in the manner described herein. The decoder 500 can also be implemented in hardware included in, for example, a transmitting station 102 or a receiving station 106.

[0050] Similar to the reconstruction path of encoder 400 discussed above, in one example, decoder 500 includes the following stages to perform various functions to produce output video stream 516 from compressed bitstream 420: entropy decoding stage 502, dequantization stage 504, inverse transform stage 506, intra / inter-frame prediction stage 508, reconstruction stage 510, loop filtering stage 512, and deblocking filtering stage 514. Other structural variations of decoder 500 can be used to decode compressed bitstream 420.

[0051] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a quantized set of transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and the inverse transform stage 506 performs an inverse transform on the dequantized transform coefficients to produce a derived residual, which can be the same as the derived residual created by the inverse transform stage 412 in the encoder 400. Using the header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter-frame prediction stage 508 to create the same prediction block as that created in the encoder 400 (e.g., at the intra / inter-frame prediction stage 402).

[0052] At reconstruction stage 510, the predicted block can be added to the derived residual to create a reconstructed block. Loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filters can be applied to the reconstructed block. In this example, deblocking filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as output video stream 516. Output video stream 516 can also be referred to as decoded video stream, and the terms will be used interchangeably herein. Other variations of decoder 500 can be used to decode the compressed bitstream 420. In some implementations, decoder 500 can produce output video stream 516 without deblocking filtering stage 514.

[0053] As described above, one technique for compressing the amount of data to be transmitted in a bitstream is to use inter-frame prediction, which creates a corresponding prediction block for the current codec block using one or more motion vectors. This prediction block(s) can be used to generate a residual to replace the original codec block for encoding and decoding. Motion vectors (or identifiers of motion vectors) for the inter-frame predicted blocks must also be sent. To reduce the number of bits required to send motion vectors, a list of motion vector candidates can be generated based on the block, and these candidates can be used to encode and decode the motion vectors used to predict the block. For example, the motion vector candidate can be used as the motion vector for predicting the block, or it can be used to generate a difference from the motion vector, also called a motion vector residual, which is then encoded and decoded in a manner similar to a residual block, replacing the original motion vector.

[0054] Higher-quality motion vector candidates (i.e., motion vector candidates that closely match the motion vectors of the current codec block) minimize the amount of data that must be transmitted at the decoder to reconstruct the motion vectors. Motion vector candidates can be obtained from the motion vectors used to predict intra-frame spatial neighboring blocks, based on the premise that the motion of such blocks will be similar to that of the current codec block.

[0055] Temporal motion vector correlation can be utilized by using motion vectors(s) from co-occurring blocks in previously reconstructed frames as reference motion vectors. While useful for tracking consistent and slow-moving activity, these motion vectors typically do not provide further compression gain relative to the motion vectors of spatially neighboring blocks. In contrast, high-velocity motion trajectories are not easily captured by conventional techniques. Motion field estimation can be used to linearly project available motion vectors from the reference frame to construct a set of motion trajectories traversing the current codec frame. Codec block locations within the codec frame can be used to trace motion trajectories individually to estimate motion vectors as candidates.

[0056] By constructing a motion field for the current frame by traversing the available motion vectors in the reference frames, motion vector candidates can be identified for motion vector prediction at the codec block level. Even after traversing all available motion vectors in the reference frames to construct a motion vector candidate list, many blocks in the current frame inevitably still lack motion vectors projected onto them, leaving holes in the motion field. These holes can simply be considered unusable and not used for motion vector prediction. The prediction quality of motion vectors can be improved by performing an additional "hole-filling" process after constructing the initial motion field.

[0057] Figure 6This is a flowchart illustrating an example of technique 600 (process, method, etc.) for encoding or decoding using projected motion field hole filling for motion vector reference. Technique 600 can be implemented as, for example, a software program executable by a computing device such as transmitting station 102 or receiving station 106. For example, the software program may include machine-readable instructions that can be stored in memory such as memory 204 or auxiliary storage 214, and when executed by a processor such as processor 202, can cause the computing device to execute technique 600. Technique 600 can be implemented using dedicated hardware or firmware. As explained above, some computing devices may have multiple memories or processors, and the operations described in technique 600 can be distributed using multiple processors, memories, or both.

[0058] For the sake of simplicity, technique 600 is depicted and described as a series of steps or operations. However, the steps or operations according to this disclosure may occur in various orders and / or concurrently. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all of the shown steps or operations may be necessary to implement the technique according to the disclosed subject matter.

[0059] In operation 602, a motion trajectory is determined. This motion trajectory is the movement between video frames in a video sequence that includes the current video block to be encoded or decoded. For example, the motion trajectory could show the movement of an object from a first reference frame to the current video frame that includes the current video block, and then from the current video frame to a second reference frame, such as when the first reference frame is a past frame relative to the current video frame (e.g., such that the second reference frame is before the current video frame in display order) or a future frame relative to the current video frame (e.g., such that the second reference frame is after the current video frame in display order).

[0060] The motion trajectory, based on a motion path from one video frame to one or more other video frames, indicates how one or more objects move or how other motion occurs between video frames. Thus, the motion trajectory can be based on or indicate the display order of the video sequence. The encoder of technique 600 knows the display order of the video sequence based on the input video stream it receives. However, the decoder of technique 600 does not receive an input video stream and may not know the display order of the video sequence it decodes. Instead, the decoder receives an encoded bitstream indicating the encoding / decoding order of the video sequence.

[0061] The display order and encoding / decoding order of a video sequence are often different. For example, a video sequence consisting of eight frames can be displayed in the order 1, 2, 3, 4, 5, 6, 7, and 8. However, the encoding / decoding order of the same video sequence can also be, for example, 1, 5, 4, 6, 3, 7, 8, and 2, such as when using a hierarchical frame structure to encode and decode the frames of a video sequence. Both the encoder and decoder know which reference frames of the video sequence are available for encoding or decoding the current video block; however, without further information provided to the decoder, it cannot utilize information about the display order of the video sequence to decode the current video block, such as using the distances between frames in the display order to correctly identify motion between frames for motion vector reference.

[0062] Furthermore, some video frames included in the encoding / decoding order may not be included in the display order. These video frames (called alternative reference frames) are decoded but not displayed as part of the output video sequence. Instead, the alternative reference frames are retained in a reference frame buffer to predict motion within video blocks of the video frames to be output for display. The encoder can encode one or more syntax elements into the bitstream to indicate the position of the alternative reference frames in the encoding / decoding order of the video sequence. The decoder can decode these one or more syntax elements to determine which alternative reference frames to use, rather than the frames intended for display.

[0063] The decoder may receive data that can be used to determine the display order of the video sequence in order to decode the current video block. This data may include one or more frame offset values ​​indicating the corresponding position of the video frame in the display order of the video sequence.

[0064] The frame offset of a video frame is determined during encoding based on the frame index of that video frame and the number of packet markers that will follow it in the display sequence. The frame index of a video frame reflects its position in the display order, as known to the encoder (e.g., based on the input video stream received by the encoder for encoding). During the encoding operation, the encoder determines the frame index of the video frames in the video sequence being encoded. The first frame in the display order of the video sequence has an index of 1, and the Nth frame in the display order has an index of N. However, while the first frame in the video sequence is the first video frame to be encoded, the next video frame to be encoded could be a video frame located in the middle of the video sequence.

[0065] When video frames are encoded into a bitstream, data indicating the encoded video frames can be stored in packets. A packet contains information about one or more video frames in the video sequence. One of those video frames is the displayable video frame to be displayed after being decoded using a decoder. The other video frames are reference frames used to decode the video frame to be displayed.

[0066] The number of hops in a packet reflects the number of video frames that the decoder must decode to display that displayable video frame (such as by outputting the displayable video frame to an output video stream). For example, when the decoder receives a packet containing N video frames, the decoder decodes the first N-1 video frames and uses them as reference frames for decoding the Nth video frame, which is then output for display. The number of packet markers in a packet indicates the total number of video frames displayed after that packet has been decoded. The encoder can determine the number of packet markers based on the number of reference frames used to encode a given video frame.

[0067] During encoding, after the frame index and packet tag number of the video frame are determined, the frame offset is calculated as the difference between the frame index and the packet tag number. For example, for a video frame with frame index 1 and 1 packet tag, the frame offset is 0. In another example, for a video frame with frame index 8 and 2 packet tags, the frame offset is 6. In yet another example, for a video frame with frame index 2 and 2 packet tags, the frame offset is 0. An example of the relationship between frame index, packet tag number, and frame offset is shown below. Figure 7 describe.

[0068] A motion trajectory for the current video frame is determined based on reference motion vectors from one or more reference frames used for encoding or decoding the current video frame. For example, the reference motion vectors may be motion vectors indicating or otherwise used to predict the motion between one of the reference frames that can be used to predict the motion of the current video block and reference frames used to encode or decode one or more video blocks within that one reference frame. For example, the order of those reference frames used to predict the motion between them may be indicated to the encoder based on the input video stream and to the decoder based on frame offset values. Based on this order, linear interpolation can be performed using the reference motion vectors on the reference frames that can be used to encode or decode the current video block and the reference frames used to encode or decode that reference frame.

[0069] Performing linear interpolation on these frames can include projecting a set of pixels from a location in one reference frame to a location within another reference frame. For example, a 4×4 or 8×8 pixel block located at a location in the first reference frame can be projected to a location within the second reference frame. The first reference frame is a reference frame used to encode or decode one or more video blocks of the second reference frame, and the second reference frame is a reference frame that can be used to predict motion in a portion of the current video frame. For example, the first reference frame could be a past frame, and the second reference frame could be a future frame. There may be one or more other reference frames between the first reference frame and the current video frame. Examples of motion trajectories are referenced below. Figure 8 and Figure 9 Describe it.

[0070] Determining the reference motion vector may include processing a region of a first reference frame that can be used to predict motion within the current video block. For example, processing the region of the first reference frame may include comparing different groups of pixel values ​​(e.g., 4×4 blocks, 8×8 blocks, etc.) of the first reference frame with groups of pixel values ​​of the same dimension in a second reference frame used to predict the motion of the first reference frame. That is, contrary to the above description, in this example, the second reference frame is a reference frame used to encode or decode one or more video blocks of the first reference frame, and the first reference frame is a reference frame that can be used to predict the motion of the current video frame. In either example, encoding or decoding the current frame may first include processing some or all of the 4×4, 8×8, or other dimensional blocks in some or all of the reference frames that can be used to predict motion within the current frame.

[0071] After determining the reference motion vector, the reference motion vector (RefMV_row, RefMV_col) is projected onto the current video frame to determine the position of the motion trajectory fit within the current video frame. The current video frame position of the motion trajectory fit is the position of the projection block position (Block_row, Block_col), which is calculated in the following manner, continuing from the latter example above (i.e., where the second reference frame is a reference frame used to encode or decode one or more video blocks of the first reference frame, and the first reference frame is a reference frame that can be used to predict the motion of the current video frame).

[0072] Block_row = RefMV.row (1stRefFrame - CurrFrame) / (1stRefFrame -2ndRefFrame)

[0073] Block_col = RefMV.col (1stRefFrame - CurrFrame) / (1stRefFrame -2ndRefFrame).

[0074] In the above formula, variable 1stRefFrame represents the frame index of the first reference frame, variable CurrFrame represents the frame index of the current video frame, and variable 2ndRefFrame represents the frame index of the second reference frame.

[0075] Subsequently, the motion trajectory for the projected block position is determined by estimating the motion vector (MV) from the projected block position (Block_row, Block_col) to one or more reference frames (such as the current frame, also known as the current codec frame, for each available reference frame). Estimating this motion vector involves performing linear interpolation on the reference frames and the current video frame based on the position of the projected block within the current video frame. More specifically, using the same notation as above, the motion vector (MV_1.row, MV_1.col) is estimated towards the first reference frame according to the following equation: MV_1.row = RefMV.row (CurrFrame - 1stRefFrame) / (1stRefFrame -2ndRefFrame) MV_1.col = RefMV.col (CurrFrame - 1stRefFrame) / (1stRefFrame -2ndRefFrame). Multiple motion vectors for the current projected block can be determined in this way by identifying reference motion vectors for the corresponding reference frames and applying these equations to project the resulting motion vectors individually onto each frame. Ideally, a defined sequence of reference frames, such as starting with the most recent frame in the display order, is used to determine the projected motion vectors for that block. For example, the projected motion vectors could be the first reference frame in the test sequence that returns the reference motion vectors.

[0076] In operation 604, it is checked whether there is a reference frame (the first reference frame in this example) for any further blocks. If there are further blocks, technique 600 can return to operation 602 to process the next block, thereby determining the motion trajectory based on the new reference motion vector (if any) determined as described above.

[0077] In summary, all blocks within one or more reference frames can be used to estimate the motion field of the current frame on a block-by-block basis (e.g., 4×4 blocks). Even so, there will most likely still be at least one block in the current frame that does not have any motion vectors estimated / projected onto the reference frame. Blocks in the current frame with motion vectors projected onto the reference frame can be used as a proper subset of the blocks in the current frame to determine the motion field of the current frame. Blocks without motion vectors projected onto the reference frame can be considered as holes in the motion field of the current frame. This can occur, for example, when there are no reference motion vectors in the reference frame (e.g., the blocks in the reference frame are intra-frame predicted).

[0078] Holes in the motion field can be considered unusable and not used for motion vector prediction. Alternatively, filling holes in the motion field can increase the number of blocks in a frame with motion vector candidates determined using that motion field, thereby improving prediction quality. Therefore, technique 600 fills holes in the motion field at operation 606. Holes in the motion field can be filled by reusing motion vectors projected onto a reference frame from spatially neighboring blocks (e.g., the nearest available projected motion vector in the spatial domain). The nearest available projected motion vector in the spatial domain can be determined using Manhattan distance. If multiple reference points share the same distance, a predetermined pattern (defined in order shared with the decoder) can be used such that the first one that appears is available as a projected motion vector to fill the hole. According to one example, this pattern is: top left → top → top right → left → right → bottom left → bottom → bottom right.

[0079] After filling the holes in the motion field, optional filtering processes can be applied to the motion field. Once the motion field is complete, it can be used as a motion vector reference, i.e., to determine a candidate list of motion vectors and predict motion vectors at the codec block level, as discussed in more detail below.

[0080] In operation 608, technique 600 determines whether filtering should be applied. If filtering is to be applied, technique 600 filters the motion field in operation 610. Filtering the motion field may depend on whether the motion vector of the block in the current codec frame is obtained from projection or from hole filling. For example, if the block's motion vector is determined in operation 606 (i.e., obtained by filling holes in the motion vector field), filtering involves determining an average motion vector by averaging the projected motion vector with motion vectors from surrounding neighboring blocks, and replacing the original motion vector with this average motion vector. If the block's motion vector is determined in operation 602 (i.e., obtained by estimating / projecting a reference motion vector), the motion vector remains unchanged.

[0081] The execution of whether to apply filtering in response to operation 608 differs between the encoder and decoder. At the encoder, such as encoder 400, the encoder determines whether to apply filtering by filtering the motion field and comparing it to an unfiltered motion field. For example, for each of the filtered and unfiltered motion fields, motion vectors can be selected from a list of motion vector candidates to encode the individual motion vectors of the current encoded / decoded frame's blocks. The motion field that minimizes the difference between the motion vectors of the current encoded / decoded frame's blocks and the motion vector candidates is determined, and the encoder can send a bit to the decoder indicating whether filtering is applied if the filtered motion field minimizes the difference, or not if the unfiltered motion field minimizes the difference. At the encoder, the difference can be determined as a sum of absolute differences (SAD), a sum of squared errors (SSR), or some other measure of difference. Computational complexity is reduced by determining the difference only for blocks that will change based on filtering (i.e., the holes filled). This bit can be sent to the decoder, for example, in the frame header. At the decoder, such as decoder 500, the bit can be read from the encoded bitstream to determine whether filtering is applied in operation 608.

[0082] Regardless of whether the motion field was filtered in operation 610, the process continues to use the motion field to predict motion vectors at the codec block level. This includes, in operation 612, using the motion field to determine a candidate list of motion vectors for predicting motion within the current video block. This candidate list may include one or more motion vector candidates determined based on the motion field. The candidate list may include one or more motion vectors from co-bit blocks (i.e., blocks at the same position in previously codec frames), although ideally, motion vectors from co-bit blocks could be omitted, with priority given to motion vectors determined from the motion field.

[0083] In some implementations, technique 600 includes one or more spatial motion vector candidates based on one or more neighboring blocks of the current block in the motion vector candidate list. For example, the one or more spatial motion vector candidates may include one or more motion vectors determined using intra-frame prediction. For example, in the case of scanning blocks of a video frame that includes the current video block in raster order, spatial motion vector candidates may be determined based on motion predicted using the upper neighboring block of the current video block, the left neighboring block of the current video block, or a combination thereof.

[0084] The one or more spatial motion vector candidates can be organized into two distinct categories. The first category includes spatial motion vector candidates determined using one or more neighboring blocks that are immediately adjacent to the current video block. The second category includes spatial motion vector candidates determined using one or more neighboring blocks that surround, but are not immediately adjacent to, the current video block. For example, spatial motion vector candidates in the second category can be derived using blocks that are two blocks away from the current video block.

[0085] Spatial motion vectors in the first category can be preferred over those in the second category. For example, if two different spatial motion vector candidates may both produce the same performance in predicting motion in the current video block, but each comes from a different category, the spatial motion vector candidate from the first category will be preferred over the other.

[0086] A separate list can be generated for each reference frame, or a single list can be generated for a block that does not depend on the inter-frame predictions used for the block. The technique used to generate the motion vector candidate list is not critical, as long as the same technique is used at both the encoder and decoder.

[0087] In operation 614, a reference motion vector is selected for encoding or decoding the motion vectors of the current codec block. This reference motion vector can be selected at the encoder by determining the rate-distortion value produced by predicting motion within the current video block using motion vector candidates from a motion vector candidate list. The rate-distortion value refers to the ratio of the amount of distortion (e.g., video quality loss) to the bitrate (e.g., number of bits) used to encode or decode the block or other video components. For example, the rate-distortion value can be determined by performing rate-distortion analysis on each motion vector candidate in the motion vector candidate list. The result of the rate-distortion analysis is that the motion vector candidate with the lowest rate-distortion value among the motion vector candidates is identified, for example, by comparing their respective rate-distortion values. In other words, the motion vector candidate that most effectively encodes the motion vector is selected as the reference motion vector.

[0088] In operation 618, the current codec block is encoded or decoded using a reference motion vector. The motion vector for this block may be a motion vector determined prior to technique 600 through rate-distortion processing at the inter-frame / intra-frame prediction level of the encoder. After the processing in operation 612, the motion vector for this block may be the reference motion vector. For example, if the reference motion vector closely matches the original motion vector, the reference motion vector may replace the motion vector previously generated for this block. Encoding or decoding may include generating a prediction block as a prediction residual based on the difference between the pixel values ​​of the current video block and the pixel values ​​of the prediction block obtained using that motion vector. For example, during the operation of encoding the current codec block, a reference motion vector may be used... Figure 4The operation performed by the encoder 400 shown (e.g., in intra / inter-frame prediction level 402) generates prediction residuals. In another example, during the decoding operation of the current video block, the following can be used: Figure 5 The operations performed by the decoder 500 shown (e.g., in intra / inter-frame prediction level 508) generate prediction residuals.

[0089] Subsequently, the encoding operations for the current video block may include performing predictive residual transformation, quantization, and entropy encoding into an encoded bitstream (e.g., Figure 4 The compressed bitstream 420 is shown. In another example, decoding the current video block may include reconstructing the video block based on the prediction residual, filtering the reconstructed video block, and outputting the filtered video block to the video stream (e.g., Figure 5 The output video stream shown is 516. The motion vectors required to reconstruct the current block in the decoder are encoded into the bitstream, for example, by including an index (identifying a reference motion vector) pointing to a list of motion vector candidates. Where the reference motion vector is used for differential encoding / decoding of the motion vectors rather than as the motion vector itself, the encoding operation may include determining a residual motion vector as the difference between two motion vectors and encoding the entropy of this residual motion vector into the bitstream. The decoding operation may include reading the index indicating the reference motion vector (e.g., from the block header), selecting the reference motion vector from the list of motion vector candidates, and, if necessary, reconstructing the motion vector from the motion vector residual and the reference motion vector.

[0090] In some implementations, technique 600 includes encoding one or more further syntax elements into the bitstream to which the current block is encoded. For example, when technique 600 is performed during the encoding of the current video block, the one or more syntax elements encoded into the bitstream may indicate the display order of a video sequence including the current frame and one or more reference frames. For example, the one or more syntax elements may be encoded into the frame header of the current frame of the current video block. For example, the one or more syntax elements indicating the display order of the video sequence may include four, five, or other numbers of bits indicating frame offset values ​​for the video frames of the video sequence.

[0091] Although the index is described as a syntax element used to identify a reference motion vector, one or more syntax elements used to indicate the selected motion vector may include a number of bits indicating the position of the pixel block corresponding to the reference motion vector within the reference frame. The number of bits in the index may be based on the total number of motion vector candidates included in the motion vector candidate list. For example, when the motion vector candidate list includes two candidates, one bit may be used to indicate the reference motion vector (e.g., each motion vector candidate is associated with one of the values ​​of that bit, or the position of the pixel block including the motion indicated by the selected motion vector may be indicated by using a value of 1). In another example, when the motion vector candidate list includes three candidates, two bits may be used.

[0092] In yet another example, one or more syntax elements used to indicate the selected motion vector can indicate contextual information associated with the reference motion vector. For example, when the reference motion vector is a spatial motion vector, the one or more syntax elements can indicate whether the reference motion vector is derived from the upper neighboring block, the left neighboring block, or a block surrounding but not immediately adjacent to the current codec block. In another example, the one or more syntax elements can indicate whether the reference motion vector is derived from a first category of spatial motion vector candidates or from a second category of spatial motion vector candidates.

[0093] At least one of the syntax elements indicates the frame offset value of the current frame, which indicates the position of the current frame in the video sequence. For example, the syntax element indicating the frame offset value may include a number of bits based on the number of foreground frames used by the encoder and decoder. For example, when the number of foreground frames is 16, four bits can be used to indicate the frame offset value. In another example, when the number of foreground frames is 32, five bits can be used to indicate the frame offset value.

[0094] In some implementations, during the decoding operation of the current video block, technique 600 includes decoding one or more syntax elements from a bitstream including the encoded frame to be decoded. For example, the one or more syntax elements decoded from the bitstream may indicate a frame offset value of the encoded video frame. In another example, the one or more syntax elements decoded from the bitstream may indicate a motion vector selected for encoding the encoded block during the encoding operation of the block.

[0095] Figure 7This is an illustration of an example of data used to indicate the display order of video frames in a video sequence. Specifically, Table 700 includes the encoding / decoding order, frame index, number of packet tags, and frame offset value for each video frame in a video sequence to be encoded or decoded. The encoding / decoding order includes intervals between frame encoding / decoding, such as between 5 and 6 and after 8. These intervals indicate when alternative reference frames are encoded or decoded (if applicable). The frame index indicates the alternative reference frame used for those intervals. For example, the video frame that is the 3rd frame in the display order of this video sequence is encoded or decoded (if applicable) between 5 and 6 in the encoding / decoding order. These intervals do not have frame offset values ​​because there are no video frames to be displayed.

[0096] Figure 8 This is a schematic diagram illustrating an example of using a motion trajectory for the current frame 800 to predict the motion of the current block 802. The motion trajectory shows the motion between the current frame 800 and three reference frames (first reference frame 804, second reference frame 806, and third reference frame 808). The motion trajectory is determined based on a reference motion vector 810, which indicates the motion between the second reference frame 806 and the first reference frame 804. For example, the second reference frame 806 could be a reference frame used to predict the motion of one or more blocks of the first reference frame 804. After determining the reference motion vector 810, the motion trajectory is determined as a trajectory relative to the direction of the reference motion vector 810.

[0097] The motion trajectory identifies the current block 802 as the location of the current frame 800 intersecting with the motion trajectory. Then, a first temporal motion vector candidate 812 can be determined to indicate the motion between the first reference frame 804 and the current frame 800. A second temporal motion vector candidate 814 can be determined to indicate the motion between the third reference frame 808 and the current frame 800. One or more of the reference motion vector 810, the first temporal motion vector candidate 812, or the second temporal motion vector candidate 814 can be included in a motion vector candidate list from which a motion vector is selected to predict the motion of the current block 802.

[0098] The second reference frame 806 and the third reference frame 808 are shown as past frames relative to the current frame 800. The first reference frame 804 is shown as a future frame relative to the current frame 800. However, other numbers of past or future reference frames can be used. For example, with one past reference frame and one future reference frame, a motion trajectory can be determined. In another example, with one past frame and two future reference frames, a motion trajectory can be determined. In yet another example, with two or more past reference frames and two or more future reference frames, a motion trajectory can be determined.

[0099] Figure 9This is an illustration of an example of using multiple motion trajectories to predict the motion of the current block. About Figure 8 The motion trajectory shown and described has a constant slope from the farthest past reference frame to the farthest future reference frame, thus indicating that motion occurs at a constant rate between video frames. However, this may not always be the case, such as in cases where motion accelerates or decelerates between video frames.

[0100] The current frame 900 includes the current block 902 to be encoded or decoded. When the motion is very slow, the co-occurrence block 904 in the first reference frame 906 preceding the current frame 900 can be used to predict the motion of the current block 902. However, if the motion is not slow enough, the corresponding motion trajectory may not intersect the current block 902, making it impossible to use the co-occurrence block 904. For example, the first motion trajectory 908 from the second reference frame 910 preceding the first reference frame 906 to the third reference frame 912 following the current frame 900 intersects the current frame 900 slightly above the current block 902.

[0101] In this context, motion field estimation of the current block 902 can be used, such as to identify the motion of co-located block 904 within an extended region from the current block 902 (e.g., a set of one or more 4×4, 8×8, or other dimensional blocks surrounding the current block 902 within the current frame 900). For example, this extended region can be defined based on the positions of objects within the current frame 900 from one or more of the first reference frame 906, the second reference frame 910, or the third reference frame 912, such as using one or more temporal motion vector candidates to indicate the motion of these objects.

[0102] For example, the motion field estimate can reflect the position of all motions within the current frame 900 relative to available reference frames. For instance, the motion field estimate can be used to determine a second motion trajectory 914 from a second reference frame 910 to a third reference frame 912. This second motion trajectory 914 intersects the current frame 900 at the current block 902. Thus, motion vector candidates associated with the second motion trajectory 914 can predict the motion of the current block 902 itself more accurately than motion vector candidates associated with the first motion trajectory 908.

[0103] Implementations of this disclosure include using (e.g., linear) projection models to create motion field estimates that can estimate different temporal motion trajectories, including those with high velocities. The resulting motion vectors are used as motion vector candidates in a dynamic motion vector reference system to augment any motion vector candidates from spatial neighbors, and optionally from co-located blocks co-located with the currently being encoded block (also referred to as the current codec block). However, many codec blocks in the frame will lack motion vectors within the motion field, reducing the usefulness of the motion vector reference for the frame as a whole. The techniques described herein describe projected motion field hole filling, which can increase the number of blocks in the frame that benefit from motion vector candidates determined using the motion field estimate, thereby improving the motion vector reference.

[0104] These techniques allow codecs to better track complex motion activities compared to spatially neighboring blocks that may not be able to maintain synchronization. Therefore, compression performance can be improved compared to using spatial motion vectors from spatially neighboring blocks alone, or using spatial motion vectors from both spatially neighboring and co-located blocks.

[0105] The above-described aspects of encoding and decoding illustrate some examples of encoding and decoding techniques. However, it should be understood that when those terms are used in the claims, encoding and decoding may mean compressing data, decompressing data, transforming data, or any other processing or alteration of data.

[0106] The word “example” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as an “example” is not necessarily to be construed as preferred or advantageous over other aspects or designs. Rather, the use of the word “example” is intended to present a concept in a specific manner. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clearly indicated in the context, the statement “X comprises A or B” is intended to mean either of its natural inclusive arrangements. That is, if X comprises A; X comprises B; or X comprises both A and B, then “X comprises A or B” is satisfied under any of the above examples. Additionally, unless otherwise specified or clearly indicated in the context for the singular form, the article “a / an” used in this application and the appended claims should generally be interpreted as meaning “one or more.” Furthermore, the use of the terms “implementation” or “an implementation” throughout this disclosure is not intended to refer to the same embodiment or implementation, unless so described.

[0107] The implementation of transmitting station 102 and / or receiving station 106 (and the algorithms, methods, instructions, etc. stored thereon and / or executed thereon (including by encoder 400 and decoder 500)) can be implemented in hardware, software, or any combination thereof. Hardware may include, for example, a computer, intellectual property (IP) core, application-specific integrated circuit (ASIC), programmable logic array, optical processor, programmable logic controller, microcode, microcontroller, server, microprocessor, digital signal processor, or any other suitable circuitry. In the claims, the term "processor" should be understood to cover any of the foregoing hardware individually or in combination. The terms "signal" and "data" are used interchangeably. Furthermore, portions of transmitting station 102 and receiving station 106 do not necessarily have to be implemented in the same manner.

[0108] Furthermore, in one aspect, for example, transmitting station 102 or receiving station 106 may be implemented using a general-purpose computer or general-purpose processor having a computer program that, when executed, performs any of the corresponding methods, algorithms, and / or instructions described herein. Alternatively or alternatively, for example, a special-purpose computer / processor may be utilized, which may include additional hardware for performing any of the methods, algorithms, or instructions described herein.

[0109] Transmitting station 102 and receiving station 106 can be implemented, for example, on a computer in a video conferencing system. Alternatively, transmitting station 102 can be implemented on a server, and receiving station 106 can be implemented on a separate device (such as a handheld communication device). In this example, transmitting station 102 can use encoder 400 to encode content into an encoded video signal and transmit the encoded video signal to the communication device. The communication device can then use decoder 500 to decode the encoded video signal. Alternatively, the communication device can decode content stored locally on the communication device (e.g., content not transmitted by transmitting station 102). Other suitable transmission and reception implementations are available. For example, receiving station 106 can be a generally fixed personal computer instead of a portable communication device, and / or the device including encoder 400 can also include decoder 500.

[0110] Furthermore, all or part of the implementations of this disclosure may take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium may be any means that can, for example, tangibly contain, store, communicate, or transmit a program for use by or in conjunction with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media are also available.

[0111] The embodiments, implementations, and aspects described above are intended to facilitate easy understanding of this disclosure and are not intended to limit it. Rather, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which should be given the broadest interpretation permitted under law to cover all such modifications and equivalent structures.

Claims

1. A method for encoding and decoding the current block of the current frame, the method comprising: The motion field of the current frame is determined using a block proper subset of the current frame, the motion field including a corresponding motion vector projected onto a reference frame for each block of the block proper subset; For a block in the current frame that has no motion vector in the motion field, the corresponding motion vector of the spatial neighboring block of the proper subset of the block is reused as the projected motion vector of the block in the motion field; The motion field is used to determine a candidate list of motion vectors; Select the reference motion vector of the current block from the list of motion vector candidates; as well as The current block is encoded and decoded using the reference motion vector.

2. The method as described in claim 1, wherein, Encoding and decoding the current block includes encoding the current block into an encoded bitstream using the reference motion vector.

3. The method of claim 2, comprising: The syntax elements that identify the reference motion vector are encoded into the encoded bitstream.

4. The method of any one of claims 2 or 3, comprising: Syntax elements indicating whether the motion field is filtered before determining the candidate list of motion vectors are encoded into the encoded bitstream.

5. The method of claim 1, wherein, Encoding and decoding the current block includes decoding the current block from the encoded bitstream using the reference motion vector.

6. The method of claim 5, comprising: Decode the syntax elements that identify the reference motion vector from the encoded bitstream.

7. The method of any one of claims 5 or 6, comprising: Decode the syntax element in the encoded bitstream that indicates whether the motion field was filtered before determining the candidate list of motion vectors.

8. The method according to any one of claims 1 to 7, wherein, Reusing the corresponding motion vector of the spatial neighboring block includes reusing the corresponding motion vector of the nearest available neighboring block in the spatial domain, determined using Manhattan distance.

9. The method according to any one of claims 1 to 8, comprising: The motion field is filtered before the candidate list of motion vectors is determined.

10. The method of claim 9, wherein, Filtering the sports field includes: Replace the projected motion vector with an average motion vector generated from multiple motion vectors; and The corresponding motion vector of each block in the true subset of the blocks remains unchanged.

11. The method of claim 10, wherein, The plurality of motion vectors includes the projected motion vector and the corresponding motion vector of at least one spatial neighboring block.

12. An apparatus for encoding and decoding the current block of the current frame, comprising: A processor configured to perform the method as described in any one of claims 1 to 11.

13. An apparatus for encoding and decoding the current block of the current frame, comprising: processor; as well as A non-transitory computer-readable storage medium storing instructions for causing the processor to perform the method as described in any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing an encoded bitstream, the encoded bitstream including an encoded residual of a current block, the encoded residual being encoded by: determining a motion field of a plurality of blocks of the current frame using a proper subset of blocks of the current frame, the motion field including a corresponding motion vector projected onto the reference frame for each of the plurality of blocks projected onto the proper subset of blocks of the reference frame; for blocks in the current frame that do not have motion vectors in the motion field, reusing the corresponding motion vectors of spatially neighboring blocks of the proper subset of blocks as projected motion vectors of the blocks within the motion field; determining a motion vector candidate list using the motion field; selecting a reference motion vector for the current block from the motion vector candidate list; and encoding the current block into the encoded bitstream using the reference motion vector.

15. The non-transitory computer-readable storage medium of claim 14, wherein, The encoded bitstream includes: The first syntax element that identifies the reference motion vector; and A second syntax element that indicates whether the motion field is filtered before determining the candidate list of motion vectors.

Images