Local hash-based motion estimation for screen remoting scenarios

Abstract

Innovations in motion estimation adapted for screen remoting scenarios are described. For example, a video encoder calculates a hash value for a current block in a current picture. The video encoder searches, subject to a spatial constraint, for a matching block in a reference picture (e.g., the previous picture in display order) based at least in part on the hash value for the current block. Thespatial constraint defines a search area in the reference picture within which hash values for candidate blocks in the reference picture may be compared to the hash value for the current block. By using a spatial constraint to limit the range of the local hash-based motion estimation, the video encoder can speed up the motion estimation process while still considering the candidate blocks in the reference picture that are most likely to match the current block.

Description

Background technique [0001] Video is delivered digitally when it is streamed over the Internet and played back through a web browser or media player. Digital video is also used when video is provided through many broadcast services, satellite services, and cable television services. Live video conferencing typically uses digital video, and it also uses digital video during video capture on most smartphones, webcams, and other video capture devices. [0002] Digital video can consume a very large number of bits. The number of bits used per second to represent video content is called the bit rate. Engineers use compression (also known as source coding or source coding) to reduce the bit rate of digital video. Compression reduces the cost of storing and transmitting video information by converting the information to a lower bit rate form. Decompression (which is also called decoding) reconstructs a version of the original information from the compressed form. A "codec" is an...

AI Technical Summary

Problems solved by technology

However, in some coding scenarios, the encoder-side decision about motion estimation is not effectively implemented

In particular, when encoding screen capture content for remote screen rendering (which is also known as "screen remoting"), motion estimation decisions cannot be efficiently made in various situations

For example, when a screen capture video shows a user vertically or horizontally scrolling through a text document or web page (which has text, embedded images, and / or other content), traditionally used for 16x16 blocks, 8x8 blocks, 4x4 blocks etc. block-based motion estimation is usually complex and time-consuming

Similarly, when screen capture video shows a user dragging a window (which has text, embedded images, and / or other content) around a GUI, traditional block-based motion estimation is often complex and time-consuming

In addition to using a lot of processing resources (which is problematic for low-complexity devices), traditional block-based motion estimation can add latency, which is problematic for real-time screen remoting

Also, block-based motion estimation is often unable to detect large scrolling and window moving activities in screen capture videos

Furthermore, block-based motion estimation is often unable to detect matches for new text (newly exposed text from scrolling or newly typed text), even if the new text includes (letter) patterns that match other patterns elsewhere on the screen

When scrolling activity and window movement activity are not encoded efficiently, or when new text is not encoded efficiently, overall compression efficiency suffers, which is especially problematic in low-bandwidth scenarios

Images