Method and system for estimating the accuracy of inference algorithms using the self-consistency methodology

Abstract

The present invention provides a method for measuring the self-consistency of inference algorithms. The present invention provides a method for measuring the accuracy of an inference algorithm that does not require comparison to ground truth. Rather, the present invention pertains to a method for measuring the accuracy of an inference algorithm by comparing the outputs of the inference algorithm against each other. Essentially, the present invention looks at how well the algorithm applied to many of the different observations gives the same answer. In particular, the present invention provides a method that is not time and labor intensive and is cost effective.

Description

[0001]The invention was made with Government support under contract number F33615-97-C-1023 awarded by the Defense Advance Research Projects Agency. The Government has certain rights in this invention.FIELD OF INVENTION[0002]The present invention relates to the field of inference algorithms.BACKGROUND OF THE INVENTION[0003]An inference algorithm is any algorithm that takes input observations from the world and makes inferences about the causes of those observations. Different types of inference algorithms are stereo algorithms and computer vision algorithms. Inference algorithms are used in several areas including three-dimensional imaging and voice recognition and natural language understanding. As technology utilizing inference algorithms advances, methods for determining the accuracy of these algorithms is essential. Accuracy determinations are based on statistical characterizations of the performance of the algorithm.[0004]One possible statistical characterization of the perform...

AI Technical Summary

Benefits of technology

[0007]The present invention provides a method for measuring the self-consistency of inference algorithms. The present invention provides a method for measuring the accuracy of an inference algorithm that does not require comparison to ground truth. Rather, the present invention pertains to a method for measuring the accuracy of an inference algorithm by comparing the outputs of the inference algorithm against each other. Essentially, the present invention looks at how well the algorithm applied to many of the different observations gives the same answer. In particular, the present invention provides a method that is not time and labor intensive and is cost effective.

[0010]In one embodiment of the present invention, the internal parameters of the algorithm are adjusted after each comparison to make the inference algorithm more and more consistent. This is useful for fine-tuning an algorithm to increase its effectiveness and accuracy.

Problems solved by technology

These models are expected to be incomplete and partially incorrect.

This requires at least 3 images per point in time, but is less reliable than (a) when there are fewer than about 5 images per point in time.

It should also be noted that resampling theory is less reliable than self-consistency methodology when there are fewer than about 5 images per point in time.

There is not enough data in these conditions to compute the self-consistency distribution.

The problem with the traditional sum-of-squared-differences (SSD) score is that it is ambiguous.

However, Cj can only be strongly negative if these costs are large enough, that is, if there is enough texture for a reliable match.

Images