Computer Method and Apparatus for Processing Image Data

Abstract

A data compression method and apparatus that includes detecting a portion of a signal comprising a sequence of video frames that uses a disproportionate amount of bandwidth compared to other portions of the signal. The detected portion of the signal result in determined components of interest. Relative to certain variance, these components of interest are normalized to generate an intermediate form, which represents the components of interest reduced in complexity by the certain variance and enables a compressed form of the signal that maintains saliency. The detecting includes any of:(i) analyzing image gradients across frames where image gradient is a first derivative model and gradient flow is a second derivative,(ii) integrating finite differences of pels temporally / spatially to form a derivative model,(iii) analyzing an illumination field across frames, and(iv) predictive analysis, to determine bandwidth consumption, which is used to determine the components of interest.

Description

RELATED APPLICATIONS[0001]This application is a divisional of U.S. application Ser. No. 12 / 522,322, filed Jul. 7, 2009, which is the U.S. National Stage of International Application No. PCT / US2008 / 000090, filed on Jan. 4, 2008, which designates the U.S., published in English, claims the benefit of U.S. Provisional Application No. 60 / 881,966, filed on Jan. 23, 2007. U.S. application Ser. No. 12 / 522,322 is also a continuation-in-part of U.S. application Ser. No. 11 / 396,010, filed Mar. 31, 2006, which is a continuation-in-part of U.S. application Ser. No. 11 / 336,366 filed Jan. 20, 2006, which is a continuation-in-part of U.S. application Ser. No. 11 / 280,625 filed Nov. 16, 2005 which is a continuation-in-part of U.S. application Ser. No. 11 / 230,686 filed Sep. 20, 2005 which is a continuation-in-part of U.S. application Ser. No. 11 / 191,562 filed Jul. 28, 2005, now U.S. Pat. No. 7,158,680. U.S. application Ser. No. 11 / 396,010 also claims priority to U.S. Provisional Application No. 60 / 667...

AI Technical Summary

Benefits of technology

[0024]Generating normalized pel data and further reducing it to the encoded pel data provides a data representation of the appearance of the pel data in the original video frame. This representation can be useful in and of itself, or as input for other processing. The encoded data may be compact enough to provide an advantageous compression ratio over conventional compression without further processing.

[0025]The encoded data may be used in place of the “transform coefficients” in conventional video compression algorithms. In a conventional video compression algorithm, the pel data is “transform encoded” using a Discrete Cosine Transform (DCT). The resulting “transform coefficients” are then further processed using quantization and entropy encoding. Quantization is a way to lower the precision of the individual coefficients. Entropy encoding is a lossless compression of the quantized coefficients and can be thought of in the same sense as zipping a file. The present invention is generally expected to yield a more compact encoded vector than DCT, thereby allowing a higher compression ratio when used in a conventional codec algorithm.

[0026]In a preferred embodiment, components of interest (i.e., the interesting portions of a video signal) are determined as a function of disproportionate bandwidth consumption and image gradients over time. The components of interest (determined portions of video signal) are normalized relative to global structure, global motion and pose, local deformation and / or illumination. Such normalization reduces the complexity of the components of interest in a manner that enables application of geometric data analysis techniques with increased effectiveness.

Problems solved by technology

In general, PCA cannot be effectively applied to the original video frames.

But, once the frames have been segmented and further normalized, the variation in the appearance of the pels in those frames no longer has the interference of background pels or the spatial displacements from global motion.

Images