Video/audio signal processing method and video/audio signal processing apparatus

Abstract

A metadata extraction unit has a feature point selection and motion estimation unit 62 for extracting at least one feature point representing characteristics of the video / audio signals in a compressed domain of the video / audio signals. Thus, reduction of time or cost for processing can be realized and it makes it possible to process effectively.

Description

TECHNICAL FIELD [0001] This invention relates to a video / audio signal processing method and a video / audio signal processing apparatus, and it provides a computationally efficient method for this to facilitate applications like (but not restricted to) camera motion extraction and video summarization from MPEG compressed domain. BACKGROUND ART [0002] With the increasing capacity of video storage devices, the need emerges for structuring and summarization of video content for convenient browsing by the user. Video browsing is enabled by metadata (i.e. data about data), which is preferably extracted automatically. [0003]FIG. 1 depicts the prior art of motion related metadata extraction from MPEG (Moving Picture Experts Group) compressed video in the pel domain. Full decoding of MPEG video into the pel domain is performed by an MPEG decoding unit 11. A motion estimation unit 12 (based on optical flow calculation or blockmatching which is known to the skilled in the art) calculates motion...

AI Technical Summary

Problems solved by technology

However, this method may produce unreliable results in case of other camera motion types than the modeled ones.

However, the camera motion estimation algorithms developed for the pel domain (as listed above) are not directly applicable to the MPEG compressed domain.

Therefore time consuming decoding of the MPEG compressed bitstream is required and as well a computational demanding motion estimation in the pel domain and a camera motion estimation has to be performed (FIG. 1).

However, this basic model is not able to detect camera zoom and rotation.

However, using MPEG motion vectors for camera motion estimation has several drawbacks.

Further, the performance of using MPEG motion vectors for camera motion estimation depends significantly of MPEG's Group of Picture (GOP) structure, the video sampling rate (e.g., 5 . . . 30 frames per second) and other factors, and is therefore not reliable for exact camera motion estimation.

More over, MPEG motion vectors (especially small ones) are often significantly influenced by noise and may be not reliable.

Therefore the algorithms based on employing MPEG motion vectors are not applicable here.

Therefore the camera motion estimation algorithms based on motion vectors contained in the compressed stream are not applicable to these cases.

Moreover, interpolation of motion vectors for I-frames from B-frames fails in case of rapid camera or object motion, where new image content occurs.

Images