Method and apparatus for significance testing and confidence interval construction based on user-specified distributions

Abstract

A computer and computer implemented method and program product for analyzing statistical data in which the data to be analyzed need not be transformed into a "Normal" distribution, thus avoiding introduction of error. Generally, the computer first determines a test statistic (formula) and associated null hypothesis. Then the distribution from which the original data arose, consistent with the null hypothesis, is defined. The computer then produces numerous randomly-generated data sets of the identical size and dimensions of the original statistical data set, according to the distribution defined above. A numerical value of the test statistic is computed from the test statistic formula for each randomly generated data set and stored in a vectored array. The numerical value of the test statistic computed from the original statistical data is then compared with the array and the associated percentile determined. With this information, the significance of the numerical value of the test statistic derived from the original data can be determined and the null hypothesis may be rejected, and if so, at what level of significance. Embodiments of the invention may likewise be used in alternative statistical applications, including computation of confidence intervals and likelihood ratios.

Description

[0001] This application is a continuation of U.S. patent application Ser. No. 09 / 594,144, filed on Jun. 15, 2000, the content of which is hereby incorporated by reference in its entirety.[0002] 1. Field of the Invention[0003] The present invention relates to the analysis of statistical data, preferably on a computer and using a computer implemented program. The invention more specifically relates to a method and apparatus that accurately analyzes statistical data when that data is not "normally distributed," by which is meant, as used herein, that the data set does not correspond to a "normal probability distribution" or does not show a bell-shaped curve.[0004] 2. Description of the Prior Art[0005] Conventional data analysis involves the testing of statistical hypotheses for validation. The usual method for testing these hypotheses, in most situations, is based on the well known "General Linear Model," which produces valid results only if the data are either normally distributed or ...

AI Technical Summary

Benefits of technology

[0010] It is yet another object of the invention to analyze said data without transforming the naturally occurring distribution of the original data into a Normal distribution, thereby avoiding errors which transformation may introduce into the analysis, said transformation preceding traditional data analysis techniques.

[0013] It is an additional object of the present invention to provide a method and apparatus of statistical analysis which enable the user to construct new test statistics, rather than rely on those test statistics with distributions that have already been determined. The subject invention removes this restriction so that any function of the data may be used as a test statistic.

[0014] It is a further object of the present invention to provide a method and apparatus for statistical analysis that enables the user to make inferences on multiple parameters simultaneously. The instant invention will permit all aspects of more than one distribution to be tested one against the other in a single analysis and determine significant differences, if any exist.

[0015] Yet another object of the present invention is to provide a mehod and apparatus that enables a user to perform sensitivity analysis on the underlying data.

[0017] The invention achieves the above objects by providing a technique to analyze empirical data within its original distribution rather than transforming it to a Normal distribution. It is preferably implemented using a digital processing computer, and therefore a computer, as well as a method and program to be executed by a digital processing computer. The technique comprises, in part, the computer generating numerous random data bases of the same size and distribution of the original database to provide comparisons to numerical relationships arising purely by chance. The best mode of the invention requires input from the user defining a number of options, although alternative modes of the invention would involve the computer determining options at predetermined stages in the analysis. The method and program disclosed herein is superior to prior art in that it allows data to be analyzed more accurately and efficiently, permits the data to be analyzed in accordance with any distribution (including the distribution which generated the data), avoids the errors which may be introduced by data transformation, and facilitates sensitivity analysis.

Problems solved by technology

More recent research has demonstrated, however, that the practice of non-linear transformation actually introduces unintended and significant error into the analysis.

Images