Distortion estimation for quantized data

Abstract

A method and an apparatus for estimating distortion due to quantization of data are described. A histogram with multiple frequency bands may be obtained for a set of coefficients to be quantized. Distortion due to quantization of the set of coefficients may be estimated based on the histogram and average distortions for the frequency bands of the histogram. The number of coefficients in each frequency band may be multiplied with an average distortion for the frequency band to obtain a per-frequency band distortion. The per-frequency band distortions for all of the frequency bands may be accumulated and scaled with a correction factor to obtain the estimated distortion. Each of the multiple frequency bands is corresponding to a unique quantization step of a set of quantization steps. Meanwhile, the starting value and the ending value of each frequency band are unique based on the respective quantization step correspondingly.

Description

technical field [0001] This disclosure relates generally to data processing, and more particularly, to techniques for estimating distortion due to data quantization. Background technique [0002] Encoding systems are widely used to process data before transmission or storage. An encoding system may quantize input data to obtain quantized data and may further process the quantized data to generate output data for transmission or storage. Quantized data can be represented as a set of discrete values, and the step size between successive discrete values ​​can be a selectable parameter. Larger quantization step sizes may result in more coarsely represented quantized data, which may result in larger errors or more distortion between the input data and the quantized data. However, a larger quantization step size may allow the quantized data to be represented with fewer bits and / or may result in quantized data containing more zeros, both of which may allow sending at a lower rate...

AI Technical Summary

Problems solved by technology

A larger quantization step size may result in a coarser representation of the quantized data, which may result in larger errors or more distortion between the input data and the quantized data

However, a larger quantization step size may allow the quantized data to be represented with fewer bits and / or may result in quantized data containing more zeros, both of which may allow sending at a lower rate and / or using less memory store output data

Conversely, a smaller quantization step size can result in a finer representation of the quantized data, which can lead to smaller errors or less distortion between the input data and the quantized data

However, a smaller quantization step size can also result in quantized data that is represented by more bits and / or contains fewer zeros, both of which can result in output data being sent at a higher rate and / or stored in more memory

Images