Predictive video coding that employs virtual reference frames generated by Direct Video Projection (DMVP).

Abstract

The present invention provides a video encoding and decoding apparatus and method for generating a virtual reference frame that can be used for predicting input video frames. [Solution] A video coding method includes predictive coding of an input frame, decoding the coded data of a reference frame if the coded input frame is designated as a reference frame, storing the decoded reference frame data for use as a predictive reference for an input frame to be coded later, and generating data for a virtual reference frame from a pair of stored reference frames, wherein the predictive coding of the input frame includes a predictive search between the reference frame data and the virtual reference frame data.

Claims

1. A video encoding method, Predictive coding of input frames, If a coded input frame is specified as a reference frame, the coded data of the reference frame is decoded, The decoded reference frame data is stored for use as a predictive reference for the input frame to be coded later. This includes generating data for a virtual reference frame from a pair of stored reference frames, A method for predictive coding of the input frame, comprising predictive searching between the reference frame data and virtual reference frame data.

2. The method according to claim 1, further comprising outputting to the decoder data representing the virtual reference frame and data representing the input frame that has been predictively coded with reference to the virtual reference frame, if a virtual reference frame is selected by the predictive search.

3. The method according to claim 1, wherein if a virtual reference frame is not selected by predictive search, the data representing the virtual reference frame is not output to the decoder.

4. The method according to claim 1, wherein the pair of first reference frames has a time position on the first side of the time position of the virtual reference frame, and the pair of second reference frames has a time position on the second side of the time position of the virtual reference frame.

5. The method according to claim 1, wherein the content of the virtual reference frame is generated by the projection of motion vectors extending from the pair of first reference frames to the pair of second reference frames.

6. The method according to claim 1, wherein the content of the virtual reference frame is generated by the projection of motion vectors extending from the pair of first reference frames to the pair of second reference frames across a plurality of reference frames.

7. The method according to claim 1, wherein when the current pixel block of an input frame is predictively coded against a virtual reference frame, the motion vector of the pixel block is derived by referring to the motion vectors of other pixel blocks adjacent to the current pixel block, which use a common set of reference frames for prediction as the current pixel block.

8. The method according to claim 1, further comprising providing a channel with data representing the virtual reference frame, including a time interpolation mode identifier indicating the use of a decoder for the virtual reference frame.

9. The aforementioned time interpolation mode identifier is in the following state: A first state indicating that the decoder should use the virtual reference frame as a reference frame, A second state indicating that the decoder should output the virtual reference frame for display, The method according to claim 8, wherein the decoder takes one of a third state indicating that it should output the virtual reference frame for a display enhanced by additional information supplied by the encoder.

10. The method according to claim 1, further comprising providing data that identifies the mode of predictive coding of the input frame.

11. The aforementioned predictive coding mode information is in the following state: The No_Skip state indicates that the predictive coding generates block-level motion information and residual information for the coded input frame content. The Full_Skip state indicates that the predictive coding uses direct motion vector interpolation without using auxiliary coding data, The method according to claim 10, wherein the predictive coding takes one of the following states: that the predictive coding uses direct motion vector interpolation and that the Semi_Skip state includes auxiliary coding data.

12. The method according to claim 1, further comprising: predictive coding of a pixel block of the input frame with reference to the virtual reference frame, and transmitting coded data of the pixel block along with a syntax element that identifies the motion vector having a value of 0 if the motion vector obtained from the predictive coding is less than a threshold.

13. The method according to claim 1, further comprising the fact that the predictive search of the pixel blocks of the input frame is constrained to a predetermined search window for the collated locations of the virtual reference frame.

14. The method according to claim 1, wherein the content of the virtual reference frame is generated by an optical flow motion vector refinement technique.

15. A computer-readable medium for storing program instructions, wherein when the program instructions are executed by a processing device, the processing device... The input frame is used for predictive coding. When a coded input frame is specified as a reference frame, the coded data of the reference frame is decoded. The decoded reference frame data is stored to be used as a predictive reference for the input frame to be coded later. Generate data for a virtual reference frame from a pair of stored reference frames. A computer-readable medium in which the predictive coding of the input frame includes a predictive search between the reference frame data and virtual reference frame data.

16. An encoding terminal, A video encoder having an input for source video, A video decoder having an input for coded video from the aforementioned video encoder, A reference picture buffer for storing the decoded reference frame output from the video decoder, A virtual reference picture generator having input to a reference frame from the aforementioned reference picture buffer, A terminal including a virtual reference picture buffer having input for a virtual reference frame output by the virtual reference picture generator.

17. The terminal according to claim 16, further comprising a predictor having inputs for reference frames from the reference picture buffer and inputs for virtual reference frames from the virtual reference picture buffer.

18. The terminal according to claim 16, wherein the pair of first reference frames has a time position on the first side of the time position of the virtual reference frame, and the pair of second reference frames has a time position on the second side of the time position of the virtual reference frame.

19. The terminal according to claim 16, wherein the virtual reference picture generator generates the content of the virtual reference frame by projecting motion vectors extending from the pair's first reference frame to the pair's second reference frame.

20. The terminal according to claim 16, wherein the virtual reference picture generator generates the content of the virtual reference frame by projecting motion vectors extending from the pair's first reference frame to the pair's second reference frame across a plurality of reference frames.

21. The terminal according to claim 16, wherein when the current pixel block of an input frame is predictively coded with respect to a virtual reference frame, the video encoder derives the motion vector of the pixel block by referencing the motion vectors of other pixel blocks adjacent to the current pixel block, which use a common set of reference frames for prediction as the current pixel block.

22. A video decoding method, Predictive decoding of the coded frame according to coding parameters provided along with the coded frame, If the coded frame is designated as a reference frame, the decoded reference frame is stored for use as a predictive reference for subsequent decoded frames. If the coding parameter identifies a coded frame as one coded by referring to a virtual reference frame, The content of the virtual reference frame is derived from a pair of stored reference frames, A method comprising decoding the coded frame using the virtual reference frame as a predictive reference.

23. The method according to claim 22, wherein the pair of first reference frames has a time position on the first side of the time position of the virtual reference frame, and the pair of second reference frames has a time position on the second side of the time position of the virtual reference frame.

24. The method according to claim 22, wherein the content of the virtual reference frame is generated by the projection of motion vectors extending from the pair of first reference frames to the pair of second reference frames.

25. The method according to claim 22, wherein the content of the virtual reference frame is generated by the projection of motion vectors extending from the pair of first reference frames to the pair of second reference frames across a plurality of reference frames.

26. The method according to claim 22, wherein when the current pixel block of an input frame is predictively coded with respect to a virtual reference frame, the motion vector of the pixel block is derived by referring to the motion vectors of other pixel blocks adjacent to the current pixel block, which use a common set of reference frames for prediction as the current pixel block.

27. A computer-readable medium for storing program instructions, wherein when the program instructions are executed by a processing device, the processing device... The coded frame is predicted and decoded according to the coding parameters provided along with the coded frame. If the coded frame is designated as a reference frame, the decoded reference frame is stored to be used as a predictive reference for subsequent decoded frames. If the coding parameter identifies a coded frame as one coded by referring to a virtual reference frame, The content of the virtual reference frame is derived from a pair of stored reference frames. A computer-readable medium that uses the aforementioned virtual reference frame as a predictive reference to decode the coded frame.

28. Decryption terminal, A video decoder having an input for coded video data, A reference picture buffer for storing the decoded reference frame output from the video decoder, A virtual reference picture generator having input to a reference frame from the aforementioned reference picture buffer, A terminal including a virtual reference picture buffer having input for a virtual reference frame output by the virtual reference picture generator.

29. The terminal according to claim 28, further comprising a predictor having inputs for reference frames from the reference picture buffer and inputs for virtual reference frames from the virtual reference picture buffer.

30. The terminal according to claim 28, wherein the pair of first reference frames has a time position on the first side of the time position of the virtual reference frame, and the pair of second reference frames has a time position on the second side of the time position of the virtual reference frame.

31. The terminal according to claim 28, wherein the virtual reference picture generator generates the content of the virtual reference frame by projecting motion vectors extending from the pair's first reference frame to the pair's second reference frame.

32. The terminal according to claim 28, wherein the virtual reference picture generator generates the content of the virtual reference frame by projecting motion vectors extending from the pair's first reference frame to the pair's second reference frame across a plurality of reference frames.

Images