Method and system for predictive modeling

Abstract

A method (100) for carrying out a predictive analysis is provided which generates a predictive model (Padj (Y|X)) based on two separate pieces of information, namely - a set of original training data (Dorig), and - a "true" distribution of indicators (Ptrue(X)). The method (100) begins by generating a base model distribution (Pgen(Y|X)) from the original training data set (Dorig) containing tuples (x, y) of indicators (x) and corresponding labels (y) (step 120). Using the "true" distribution (Ptrue(X)) of indicators, a random data set (D') of indicator records (x) is generated reflecting this "true" distribution (Ptrue(X)) (step 140).; Subsequently, the base model (Pgen(Y|X)) is applied to said random data set (D'), thus assigning a label (y) or a distribution of labels to each indicator record (x) in said random data set (D') and generating an adjusted training set (Dadj) (step 150). Finally, an adjusted predictive model (Padj (Y|X)) is trained based on said adjusted training set (Dadj) (step 160).

Description

technical field [0001] The present invention generally relates to predictive modeling. In particular, the invention relates to adapting existing predictive models generated from a training data set to additional information such as a given probability distribution of features. Background technique [0002] Predictive modeling is the process by which a model is created or selected in an attempt to best predict the probability of an outcome. Typically, the model is chosen on a detection theory basis to try to guess the probability of an outcome given a set amount of input data (eg: given an email to determine how likely it is to be spam). Thus, it is assumed that the features (indicators) X A predefined set of , the goal of predictive modeling is to predict the probability P(Y| X ). This task can be thought of as for the "true" probability distribution P(Y| X ), however, this is not directly observable. Instead, one has to try to generate the best distribution, which sho...

AI Technical Summary

Problems solved by technology

However, this scheme requires heavy computational effort and, in most cases, involves considerable response time, which renders the task non-interactive and does not allow the user to interactively experiment with information about the "true" distribution P( X ) with different assumptions

Also, this presents a security risk organizationally, since everyone who takes the model and adapts it to new applications will need to gain access to the actual source data

Images