Pictorial digital image processing incorporating adjustments to compensate for dynamic range differences

Abstract

Image-dependent tone and color reproduction processing of digital images is accomplished by creating a spatially blurred and sub-sampled version of the original image, and applying knowledge of the capture device physical characteristics to obtain statistics related to the scene or original captured. The statistics are used for image-dependent linearization of the captured image data based on an OECF model, and in conjunction with information about the output medium, to calculate image-specific preferred tone reproduction curves based on a preferred reproduction model. The image data can then be processed for reproduction on output media with different density ranges and color gamuts. All the processing described can be accomplished automatically, but the access to accurate scene information afforded by the image-dependent linearization, and to the perceptually intuitive parameters controlling the calculation of the preferred reproduction curves, also allows for simple, intuitive manual adjustment.

Description

[0001] This application is a Division of copending allowed parent application Ser. No. 09 / 771,735, filed Jan. 29, 2001, by Jack M. Holm, entitled PICTORIAL DIGITAL IMAGE PROCESSING INCORPORATING IMAGE AND OUTPUT DEVICE MODIFICATIONS, which parent application is a Division of application Ser. No. 08 / 822,053, filed Mar. 24, 1997, by Jack M. Holm, entitled STRATEGY FOR PICTORIAL DIGITAL IMAGE PROCESSING, now Pat. No. 6,249,315. The parent and grandparent applications are hereby incorporated by reference.[0002] This invention relates generally to the processing of digital images to produce desired tone and color reproduction characteristics. Specifically, this invention makes use of capture and output device information, in conjunction with opto-electronic conversion function (OECF) and preferred reproduction models, to determine image-specific processing based on statistics derived from the image data. The processing determined may be applied automatically or with user input. Backgroun...

AI Technical Summary

Benefits of technology

[0041] 7. How to implement user adjustments that produce preferred reproduction with maximum simplicity and intuitiveness.

Problems solved by technology

Manual processing is time consuming and must be done by individuals with significant expertise, partly because the controls found in currently available software packages make it difficult to achieve desired tone and color reproduction.

Simple controls tend to vary the reproduction in ways that miss the optimum, and complex controls offer too many degrees of freedom.

With digital cameras in particular, this is frequently not the case.

Other difficulties in determining scene physical characteristics have resulted from a lack of standard, accurate measurement approaches.

When basic flaws are present in a measurement approach, such as the omission of flare considerations and the fact that the spectral characteristics of the detector preclude calorimetric information from being obtained, attempts to calculate scene values inevitably produce erroneous results.

These errors reduce accuracy expectations and mask other error sources, seriously degrading the correlation between captured data and scene characteristics.

It is clear that the factors limiting tone and color reproduction quality in digital images stem from an incomplete and sometimes inappropriate global strategy.

1. Inaccurate and non-standard device measurements.

a) Flare and other non-linearities in both capture devices and measuring instruments are frequently not considered, or are measured for a particular condition, and the resulting values are erroneously assumed to be applicable to other conditions.

d) Measurement precision is frequently specified in linear space, resulting in perceptually large errors for darker image areas.

These areas are also most affected by flare, making dark area measurements particularly inaccurate.

e) Measurement geometries and lighting may be inconsistent or inappropriate to the device use conditions.

All of these effects compound to produce device characterization measurements which can be quite inaccurate.

A common perception of these inaccuracies is that it is practically impossible to obtain stable measurements and, therefore, measurement accuracy need not be too high.

These techniques routinely produce repeatable measurements with several orders of magnitude higher accuracy than those obtained for digital systems, which is one of the reasons the less flexible conventional systems are able to outperform current automatic digital systems.

Unfortunately, the reason these techniques are so accurate is because they have been refined specifically for conventional photographic materials.

2. Difficulties in communicating device information due to a lack of standard data types, terms, and data formats.

The measurement methods described in the standards mentioned above tell how to measure digital photography system characteristics using various test targets, but they do not deal with methods for estimating image-dependent capture non-linearities.

4. Lack of specification of standard, optimal methods for transforming image data from the capture device spectral space to standard color spaces.

Many of these methods have merit in particular circumstances, but in many cases are applied inappropriately.

The lack of accurate characterization data compounds the problem in that it is difficult to tell if the cause of low quality transformed data is an inappropriate transformation method, inaccurate characterization data, or both.

Another difficulty has been that, until recently, the only standard color spaces used for digital photography were those defined by the CIE (Commission Internationale de L'clairage or International Commission on Illumination) based on the human visual system (HVS).

For various reasons, it is generally impractical to design digital photography systems that mimic the HVS.

Device-independent color is an appropriate goal in some cases, but frequently falls short.

This is certainly not desirable, and consequently a number of ICC member (and other) companies are now creating "ICC profiles" that produce colors from the same image data which vary between devices.

Appearance models may be able to predict how something will look under certain conditions, but they give no information about observer preferences for tone and color reproduction in a particular photograph.

Unfortunately, flare is present in all capture devices that form an image of the scene or original with a lens (as opposed to contact type input devices, like drum scanners).

a) ICC input profiles only allow transformations to CIE color spaces, yet transformations to this type of color space are valid only if the capture device sensitivities are color matching functions, or colorants found in the scene or original are known, and are spanned by spectral basis functions not greater in number than the number of device spectral capture channels. These conditions are almost never met when digital cameras are used to capture natural scenes.

b) The appropriate ICC input profile for a particular device and / or set of colorants to be captured is generally assumed to be invariant with the content of the scene or original. This assumption is not valid with the many capture devices that have significant amounts of flare, such as digital cameras and area array scanners.

c) The measurement techniques used to determine ICC profiles are variable, and the profiles constructed are usually not optimal. Frequently, profiles are not linearized correctly, neutrals are not preserved, and incorrect assumptions are made about the colorants in the scene or original. These inaccuracies are masked by and compounded with the inaccuracies mentioned in a and b.

d) While it is recognized that different output media must produce different colorimetric renderings of the same image data for the results to be acceptable, there is no standard methodology for determining how to render images based on the dynamic range of the scene or original, as compared to the output medium.

The ICC efforts have resulted in a significant improvement over doing nothing to manage colors, but in their current manifestation are not viewed as a substitute for manual processing.

As stated previously, current manual processing software tends to be overly complicated and offers too many degrees of freedom or is incapable of producing optimal results.

Images