Efficient video skimmer

Abstract

Disclosed are a system, method, apparatus, and computer readable media containing instructions for displaying video files for rapid searching. In two different types of exemplary embodiments, a standalone video skimming system, and a video skimming system includes a server and a client system are disclosed, where the video file may be locally or remotely stored, or can be obtained from a live feed. The system displays many small windows simultaneously, in which different parts of the video chosen by the user are shown at the same time to shorten the skimming time. The video file is encoded using layered encoding to display smaller versions using lower layers, and without needing any processing to generate smaller versions of the video from the original full screen version. A video extractor is described for extracting the necessary bitstreams from a local video database containing layered encoded video files according to user specified window sizes, and distributing the signals over the electronic communications network channel. The system also includes a skimming control logic which can receive control commands from clients and invoke the video extractor to extract appropriate audio-visual signals there from for each command.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS[0001]This application claims the benefit of priority to U.S. Provisional Application Ser. No. 61,172,355, filed Apr. 24, 2009, which is hereby incorporated by reference herein in its entirety.BACKGROUND[0002]1. Technical Field[0003]The disclosed invention relates to techniques for searching for content in a compressed digital video file accessed from local storage or over a network such as the Internet. In particular, it relates to the use of layered video coding technology in connection with content searching for retrieving and displaying selected video segments.[0004]2. Background Art[0005]Subject matter related to the present application can be found in co-pending U.S. patent application Ser. Nos. 12 / 015,956, filed Jan. 17, 2008 and entitled “System And Method For Scalable And Low-Delay Videoconferencing Using Scalable Video Coding,” 11 / 608,776, filed Dec. 8, 2006 and entitled “Systems And Methods For Error Resilience And Random Access In V...

AI Technical Summary

Benefits of technology

[0053]The term “codec” is equally used herein to describe techniques for encoding and decoding, and for implementations of these techniques. An encoder converts input media data into a bitstream or a packet stream, and a decoder converts an input bitstream or packet stream into a media representation suitable for presentation to a user, for example digital or analog video ready for presentation through a monitor, or digital or analog audio ready for presentation through loudspeakers. A transcoder converts an input bitstream or packet stream compressed using a compression technique into its original media representation suitable for presentation to a user and then re-converts into an input bitstream or packet stream using another type of compression technique. Encoders and decoders can be dedicated hardware devices or building blocks of a software-based implementation running on a general purpose CPU.

[0054]Set-top-boxes and personal computers (PCs) can be built such that many encoders or decoders may run in parallel or quasi-parallel. For hardware encoders or decoders, one way to support multiple encoders/decoders is to integrate multiple of their instances in the set-top-box or PC. For software implementations, similar mechanisms can be employed.

[0055]Traditional video codecs used in video distribution systems provide only a single bit stream at a given bitrate, and no layers. As explained above, when a lower temporal or spatial resolution is required from a full length video file (such as for fast forwarding or for display at a smaller spatial size in a MBW), first, the full resolution file must be decoded to regenerate the raw (uncompressed) video, which then needs to be sub-sampled in temporal and/or spatial dimension, as the case may be, to produce a lower spatio-temporal resolution appropriate for the MBW. This process wastes significant bandwidth (if the full length video file is in a remote location and needs to be transported over a network), time, and computational resources. However, support for lower resolutions is beneficial in the video skimmer to enable display of many video chapters simultaneously, and without consuming processing time and power to generate them. The network

Problems solved by technology

Currently, these videos are often of small size and low resolution.

Even after applying digital video compression techniques, high resolution video results in large file sizes.

“Skimming” video, alternatively known as browsing, has been a technical challenge for a long time.

a. Fast forwarding: fast forwarding (also known as increasing the video playback speed) shortens the video viewing time. However, speeding up the video rate distorts the video information and may cause elimination of short events. This method has been the most popular browsing technique so far. Fast forwarding is discussed in more detail below.

b. Text Based Queries: This refers to a querying of metadata associated with the full length video or video chapters for specific textual information. For example, a text based query may be in the form of “scene with George falling off the bridge”. Text based queries today require the video to be annotated, mostly a manual process, before the video can be queried. Although text-based video query has been in existence for a long time, only few applications can afford the required intense human effort needed to intelligently categorize and annotate the videos. One example of video content that contains metadata which enables text based queries is medical records used in some systems.

c. Automatic Indexing: In the academic literature [for example, Cees G. M. Snoek and Marcel Worring, “Multimodal Video Indexing: A Review of the State-of-the-art,” Multimedia Tools and Applications, Volume 25, Number 1/January, 2005, Springer], techniques have been proposed to automatically index video for browsing representations based on information within the video. These indexing systems can use, for example, any of the following information aspects to generate video chapters:Motion of the video;Scene changes;Image statistics—such as color and shape;Audio information; and/orSpecific object types in the video.

Today, when using any form of automated video skim generation, it is unfortunately quite frequent that a certain scene, in which a user may be interested, stays unidentified by the skimming process.

In summary, the automated context-sensitive generation of video skims, despite the significant research conducted over the past decade, has remained a task that is difficult, requiring high computational complexity and involving human interaction such as filtering and processing.

(1) The search may still take a long time depending on where the specific video segment of interest is located within the full length video sequence (particularly if it is located towards the end).

(2) The video segment of interest may be made unnoticeable or totally l

Images