Artificial intelligence prediction method based on reasoning enhancement

Abstract

The invention discloses an artificial intelligence prediction method based on reasoning enhancement. And the selective deviation correction effectively reduces the influence of the selective deviation, thereby improving the prediction accuracy. Compared with a traditional regression method which assumes that data distribution is consistent, the method can effectively eliminate false correlation and recognize a real causal effect, so that higher prediction accuracy is shown in a high-deviation scene. And obvious robustness and stability are shown in a data feature missing scene. According to the method, a complete data set without missing items is obtained through complete condition specification multiple interpolation, the method is superior to traditional simple interpolation, interpolation deviation is reduced, the robustness of the model in a high missing data scene is remarkably improved, external factor information can be more comprehensively utilized, and the adaptability to a complex scene is improved. Explicit modeling is performed on external factors. By means of the special processing for the external factors, the influence of external information can be better captured in a complex environment, and the prediction effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing and prediction systems, and in particular to an artificial intelligence prediction method based on reasoning enhancement. Background Art

[0002] In practical applications, when dealing with complex scenarios, existing AI prediction methods often suffer from a significant decrease in the accuracy and stability of prediction results due to selective bias, missing data, and the influence of external factors. In the prediction of marketing campaign effects, customer characteristics (such as age, income, etc.) often affect both the allocation of marketing strategies (treatment variables) and the customer's response results (effect variables). This dual influence introduces selective bias into the data, making it impossible for the model to accurately capture the true causal relationship and may produce false associations, further weakening the reliability of the prediction. In addition, much customer information that is critical to the model (such as socioeconomic status or purchase intention) is usually hidden, cannot be directly observed, or is partially missing, which significantly increases the difficulty of modeling. At the same time, in actual marketing scenarios, the available data at the beginning of the campaign is usually incomplete, and the model must make predictions based on limited observational information, which places higher requirements on the robustness of the prediction system under high uncertainty conditions. Combined with the above problems, an effective prediction method not only needs to correct data bias, but also needs to have a strong ability to handle missing data and adapt to changes in external factors, so as to achieve high accuracy and stability in complex scenarios.

[0003] Currently, AI prediction methods have been widely studied in academia and industry, and some technologies have achieved remarkable results. For example, traditional regression prediction methods include Lasso regression [1], support vector regression (SVR) [2], and XGBoost [3], which train models with historical data and use them to predict target variables. These methods use the potential statistical patterns in the data to build models, can deal with nonlinear relationship problems well, and have been successfully applied in many fields (such as marketing and financial forecasting). However, these methods usually assume that the data distribution is consistent, and it is difficult to deal with the selective bias problem caused by the inconsistent distribution of characteristics of different subgroups in real scenarios. In addition, when faced with missing data, these methods often use simple interpolation techniques (such as mean interpolation or zero filling), which may ignore the uncertainty of the data and thus affect the accuracy of the prediction.

[0004] On the other hand, causal inference methods have become an important tool for solving the problem of prediction bias in recent years. This type of method is based on the potential outcome framework (POF) and estimates the effects of intervention measures by modeling causal relationships. Typical methods include TARNET[4] and Dragonnet[5], which can effectively correct the bias introduced by confounding variables, especially in the task of effect estimation. However, the core goal of these methods is to estimate the treatment effect (i.e., the impact of the intervention on the outcome) rather than predict future effects. When it is necessary to predict the effects of new scenarios or new customers, the performance of these methods may be insufficient. In addition, most causal inference methods assume that the confounding variables are fully observed. However, in actual business scenarios, many confounding variables are hidden and can only be inferred indirectly through proxy variables. At the same time, these methods usually assume that the data is complete and do not fully consider the prevalence and complexity of missing data in reality.

[0005] Although some deep learning techniques (such as variational autoencoders, VAE) attempt to introduce generative models into causal inference to model hidden confounding variables and handle complex data distributions, their application in data missing, external factor modeling, and causal explanation is still insufficient. In general, existing technologies are still limited in solving selective bias, missing data processing, and complex external factor modeling, especially in future-oriented prediction tasks. These technologies are difficult to directly apply to the complex scenarios mentioned in the article, such as the marketing campaign effect prediction task with multiple missing data and external factors.

[0006] The prior art has the following problems:

[0007] 1) It is difficult to effectively deal with selection bias

[0008] Traditional prediction methods assume that data distribution is uniform and cannot solve the problem of selective bias caused by inconsistent distribution of characteristics of different subgroups. Although causal inference methods can correct the bias caused by confounding variables, they are insufficient in predicting new scenarios.

[0009] 2) Limited ability to handle missing data

[0010] Existing technologies usually use simple interpolation methods (such as mean interpolation or zero filling) to deal with missing data, which ignores the uncertainty of the data and leads to reduced prediction accuracy. In addition, most causal inference methods assume that the data is complete and do not fully consider the prevalence and complexity of missing data.

[0011] 3) Insufficient modeling of complex external factors and hidden confounding variables

[0012] Most current causal inference methods assume that confounding variables are fully observed. When faced with hidden confounding variables, they need to rely on proxy variables for indirect inference, but the effect is limited. At the same time, existing technologies are less adaptable to changes in external factors and cannot maintain the robustness of predictions under high uncertainty conditions.

[0013] Prior art references: [1] Volker Roth. 2004. The generalized LASSO. IEEE transactions onneural networks15, 1 (2004), 16–28. [2] Mariette Awad and Rahul Khanna. 2015. Support vector regression.In Efficient learning machines. Springer, 67–80. [3] Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable treeboosting system.In Proceedings of the 22nd acm sigkdd internationalconference on knowledge discovery and data mining. 785–794. [4] Fredrik Johansson, Uri Shalit, and David Sontag. 2016. Learningrepresentations for counterfactual inference. In International conference onmachine learning.PMLR, 3020–3029. [5] Claudia Shi, David Blei, and Victor Veitch. 2019. Adapting neuralnetworks for the estimation of treatment effects. Advances in neuralinformation processing systems 32 (2019). Summary of the invention

[0014] In view of the shortcomings of the prior art, the present invention provides an artificial intelligence prediction method based on reasoning enhancement. The present invention is achieved through the above technical solutions:

[0015] The present invention discloses an artificial intelligence prediction method based on reasoning enhancement, comprising:

[0016] Obtain the definition of the variables of interest in the prediction system, define the effect variable Y, the treatment variable T, the confounding variable Z, the observable proxy variable X, the adjustment variable M, and the causal relationship between the variables;

[0017] According to the definition of variables, obtain the data set of relevant variables;

[0018] Based on the obtained data set of relevant variables, the missing data characteristics of the data in the data set were imputed using the fully conditional standardized multiple imputation FCSMI to obtain a complete data set without missing items;

[0019] According to the causal relationship between the variables, assuming that the confounding variable Z obeys the normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as input, and the expectation and variance of the normal distribution obeyed by Z as output;

[0020] According to the causal relationship between the variables, assuming that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output;

[0021] Set the value z of the confounding variable to obey the prior probability distribution;

[0022] Based on the two fully connected neural network models established above, a complete data set without missing items is used as the training data set, and the variational lower bound ELBO is used as the objective function to obtain the parameters of the neural network model through training;

[0023] Based on the parameters of the trained neural network model, calculate the conditional probability distribution P(y|z,m,t) of the effect variable Y;

[0024] Based on the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the effect variable Y and any given value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t, and the prior probability distribution obeyed by the value of the confounding variable z, the Bayesian formula is applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given.

[0025] As a further improvement, the definition of obtaining the variables of interest of the relevant prediction system described in the present invention is specifically to clarify the role and observable conditions of each relevant variable in the prediction system, that is:

[0026] The effect variable Y is the main target of the prediction system's prediction results;

[0027] The treatment variable T is the variable that mainly affects the effect variable Y;

[0028] The confounding variable Z is a variable that affects both the treatment variable T and the effect variable Y;

[0029] An observable proxy variable X is an observable variable X that is affected by a confounding variable Z;

[0030] The adjustment variable M is the external factor that affects the effect variable.

[0031] As a further improvement, according to the present invention, based on the obtained data set, the missing items in the data set are interpolated using the fully conditional canonical multiple imputation FCSMI to obtain a complete data set without missing items, specifically:

[0032] 1) Perform mean interpolation on the missing data features in the obtained data set of relevant variables to obtain the initial complete data set;

[0033] 2) Further adjust the data features in the initial complete data set, use other data features as input and the missing data features in the obtained data set of relevant variables as output, and use the initial complete data set as the training data set to establish and train a linear regression model or a logistic regression model;

[0034] 3) Use the linear regression model or logistic regression model obtained in the previous step to interpolate the missing data features in the data set of the obtained relevant variables. If it does not converge, return to step 2) and loop until convergence.

[0035] As a further improvement, according to the causal relationship between the variables, the present invention assumes that the confounding variable Z obeys the normal distribution under the given conditions of the processing variable T, the observable proxy variable X, and the adjustment variable M, and constructs a fully connected neural network model with the value of the processing variable t, the value of the proxy variable x, and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Z as output;

[0036] It is assumed that the confounding variable Z follows a normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M. The expectation and variance of its distribution are functions of the observable proxy variable X, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the value z of the confounding variable are finally output.

[0037] Including the expectation of the normal distribution and variance σ 2 :

[0038]

[0039] Among them, g is a fully connected neural network model.

[0040] As a further improvement, according to the causal relationship between the variables, the present invention assumes that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, and establishes a fully connected neural network model with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output;

[0041] It is assumed that the effect variable Y follows a normal distribution under the given conditions of the confounding variable Z, the treatment variable T, and the adjustment variable M. The expectation and variance of its distribution are functions of the confounding variable Z, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the effect variable value y are finally output.

[0042]

[0043] Including the expectation of the normal distribution and variance σ 2

[0044]

[0045] Among them, f is a fully connected neural network model with a nonlinear activation layer.

[0046] As a further improvement, the two fully connected neural network models established above described in the present invention use a complete data set without missing items as a training data set, and use the variational lower bound ELBO as the objective function. The parameters of the neural network model obtained by training, the variational lower bound ELBO as the objective function is specifically:

[0047] .

[0048] As a further improvement, the present invention provides a conditional probability distribution p(y|z,m,t) based on the conditional probability distribution of the effect variable Y and any given value z of the confounding variable, the value m of the adjustment variable, and the value t of the treatment variable, and the prior probability distribution obeyed by the value z of the confounding variable, and applies the Bayesian formula to obtain a prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given:

[0049] Using the prior distribution p(z) of the confounding variable z and the conditional probability distribution p(y|z,m,t) predicted by the trained neural network model, the predicted distribution of the outcome variable y under the given adjustment variable m and treatment variable t is calculated using the Bayesian formula:

[0050] .

[0051] The beneficial effects of the present invention are as follows:

[0052] 1) Selective bias correction, effectively reducing the impact of selective bias, thereby improving the accuracy of prediction. The present invention obtains the definition of the variables of interest in the prediction system, defines the effect variable Y, the processing variable T, the confounding variable Z, the observable proxy variable X, the adjustment variable M, the causal relationship between the variables, and establishes multiple neural network models and regression models based on the causal relationship between the variables to correct the selective bias caused by the confounding variables. Compared with the traditional regression method that assumes consistent data distribution, the present invention can effectively eliminate pseudo-correlations and identify true causal effects, thereby showing higher prediction accuracy in high-bias scenarios.

[0053] 2) Processing of missing data features, showing remarkable robustness and stability in scenarios with missing data features. This paper adopts fully conditional specification of multiple imputation (FCSMI) to obtain a complete data set without missing items. This method is superior to traditional simple interpolation (such as mean interpolation), not only reducing interpolation bias, but also significantly improving the robustness of the model in scenarios with high missing data.

[0054] 3) External factor modeling can make more comprehensive use of external factor information and improve the adaptability to complex scenarios. The present invention explicitly models external factors by defining adjustment variables M. This specialized treatment of external factors enables the present invention to better capture the impact of external information in complex environments and improve the prediction effect.

[0055] 4) Inference of confounding variables: by modeling the conditional probability distribution of confounding variables under given conditions of treatment variable T, observable proxy variable X, and adjustment variable M, the accuracy of prediction and the reliability of results are enhanced, and the versatility of the algorithm is enhanced. By using a neural network model, the present invention can infer hidden confounding variables from observed data and incorporate them into the prediction framework. This makes up for the deficiency of traditional causal inference methods that assume that confounding variables are fully observed, which not only enhances the prediction ability of the model and improves the reliability of the results, but also expands the scope of reasoning of the model.

[0056] These key technologies enable the present invention to significantly outperform traditional methods in selective bias correction, missing data processing, external factor modeling, and hidden confounding variable inference, providing a more accurate, robust, and reliable solution for complex prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is an algorithm flow chart of an artificial intelligence prediction method based on reasoning enhancement in the present invention. DETAILED DESCRIPTION

[0058] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention. Figure 1 It is an algorithm flow chart of an artificial intelligence prediction method based on reasoning enhancement in the present invention;

[0059] The present invention discloses an artificial intelligence prediction method based on reasoning enhancement, comprising: obtaining the definition of the variables concerned by the prediction system, defining the effect variable Y, the processing variable T, the confusion variable Z, the observable proxy variable X, the adjustment variable M, and the causal relationship between the variables; the effect variable Y is the main target of the prediction result of the prediction system;

[0060] The treatment variable T is the variable that mainly affects the effect variable Y;

[0061] The confounding variable Z is a variable that affects both the treatment variable T and the effect variable Y;

[0062] An observable proxy variable X is an observable variable X that is affected by a confounding variable Z;

[0063] The adjustment variable M is the external factor that affects the effect variable.

[0064] According to the definition of variables, obtain the data set of relevant variables;

[0065] According to the obtained data set of relevant variables, the missing data characteristics of the data in the data set are interpolated using the fully conditional standardized multiple imputation FCSMI to obtain a complete data set without missing items; specifically:

[0066] 1) Perform mean interpolation on the missing data features in the obtained data set of relevant variables to obtain the initial complete data set;

[0067] 2) Further adjust the data features in the initial complete data set, use other data features as input and the missing data features in the obtained data set of relevant variables as output, and use the initial complete data set as the training data set to establish and train a linear regression model or a logistic regression model;

[0068] 3) Use the linear regression model or logistic regression model obtained in the previous step to interpolate the missing data features in the data set of the obtained relevant variables. If it does not converge, return to step 2) and loop until convergence.

[0069] According to the causal relationship between the variables, assuming that the confounding variable Z obeys the normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as input, and the expectation and variance of the normal distribution obeyed by Z as output;

[0070] Assume that the confounding variable Z follows a normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M. The expectation and variance of its distribution are functions of the observable proxy variable X, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m. The feature representation of the input variables is learned through several hidden layers, and finally the parameters of the distribution of the value z of the confounding variable are output;

[0071] Including the expectation of the normal distribution and variance σ 2 :

[0072]

[0073] Among them, g is a fully connected neural network model.

[0074] According to the causal relationship between the variables, assuming that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output;

[0075] It is assumed that the effect variable Y follows a normal distribution under the given conditions of the confounding variable Z, the treatment variable T, and the adjustment variable M. The expectation and variance of its distribution are functions of the confounding variable Z, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the effect variable value y are finally output.

[0076]

[0077] Including the expectation of the normal distribution and variance σ 2：

[0078]

[0079] Among them, f is a fully connected neural network model with a nonlinear activation layer.

[0080] Set the value z of the confounding variable to obey the prior probability distribution;

[0081] Based on the two fully connected neural network models established above, a complete data set without missing items is used as the training data set, and the variational lower bound ELBO is used as the objective function to obtain the parameters of the neural network model through training;

[0082] The variational lower bound ELBO is the objective function:

[0083] .

[0084] Based on the parameters of the trained neural network model, calculate the conditional probability distribution P(y|z,m,t) of the effect variable Y;

[0085] Based on the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the effect variable Y and any given value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t, and the prior probability distribution obeyed by the value of the confounding variable z, the Bayesian formula is applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given;

[0086] Using the prior distribution p(z) of the confounding variable z and the conditional probability distribution p(y|z,m,t) predicted by the trained neural network model, the predicted distribution of the outcome variable y under the given adjustment variable m and treatment variable t is calculated using the Bayesian formula:

[0087] .

[0088] Further: The present invention accurately describes the definition of variables in the prediction system and their causal relationships, adopts a systematic method to clarify the role of related variables and their position in the prediction model, and at the same time, clarifies the causal relationship between variables. First, the following categories of key variables are defined, and their roles and functions are explained. The effect variable is the main goal of the prediction system, representing the final result that the system hopes to obtain through prediction. For example, in sales forecasting, the effect variable may be the sales volume of a certain future period. In the causal relationship, the effect variable is usually set as the target node, and the effects of all other variables will point to Y through direct or indirect paths. The processing variable T is the core factor that mainly affects the effect variable Y, and is the key variable that the prediction system hopes to study or optimize. For example, in the advertising delivery scenario, the processing variable T can be the amount of advertising delivery, and its impact on Y is usually modeled through a direct path. Confounding variables act on both the processing variable T and the effect variable Y, and their existence may cause deviations in the relationship between T and Y. For example, in consumer behavior research, seasonal factors may affect both advertising delivery (processing variables) and directly act on sales (effect variables). Therefore, accurately identifying confounding variables is an important step to ensure clear causal relationships. Since the confounding variable Z is often difficult to observe directly, the present invention introduces an observable proxy variable X to indirectly reflect the impact of Z by measuring X. For example, seasonal factors (confounding variables) can be reflected by temperature or precipitation (proxy variables). The adjustment variable M is a factor that produces external interference on the effect variable Y, such as the macroeconomic environment or policy changes. The existence of the adjustment variable needs to be included in the causal analysis to ensure the accuracy and reliability of the prediction model. Next, in order to more intuitively display the relationship between the variables and their role in the system, it is necessary to clarify the causal relationship between the variables by constructing a causal diagram. In the causal diagram, Figure 1 In the causal diagram, the treatment variable T has a direct impact on the effect variable Y, so the causal relationship is represented by the arrow T→Y in the diagram. At the same time, the confounding variable Z affects both T and Y through two paths, namely Z→T and Z→Y. In order to capture the impact of Z, the observable proxy variable X is included in the diagram and represented by the path Z→X. In addition, the adjustment variable M is used as an external interference variable to directly affect the effect variable Y, which is represented by M→Y in the causal diagram.

[0089] It is necessary to clarify the data source according to the definition of each variable, collect and integrate it, and obtain the data set of relevant variables in the prediction system.

[0090] For missing data features in the dataset of relevant variables in the prediction system, the fully conditional canonical multiple imputation (FCSMI) method is used to interpolate the missing data to generate a complete dataset without missing items. The specific method is described as follows. First, mean interpolation is performed on the missing data features in the dataset, replacing the missing items with the mean value of each corresponding feature. This step obtains an initial complete dataset, which provides a reasonable starting point for subsequent interpolation methods. Next, for each missing data feature, a regression model is established using other features as input variables and the missing data feature as the output variable. According to the data feature type, a linear regression model is used for continuous missing data features; a logistic regression model is used for categorical and logical missing data features. The initial complete dataset is used as the training dataset to train the established regression model to capture the relationship between features. Subsequently, the trained regression model is used to predict the missing features, and the missing values ​​are replaced with the predicted values ​​to generate a new complete data set. The change in the interpolation value between the new complete data set and the previous complete data set is checked to determine whether the interpolation has converged. If not, return to the previous step, update the regression model and re-interpolate, and repeat the process iteratively until the interpolation value converges. Through the above FCSMI interpolation method, a complete data set without missing items is finally obtained, providing a reliable data basis for subsequent analysis and modeling.

[0091] Based on the causal relationship between variables, causal reasoning and modeling are performed. A total of two neural network models need to be established to achieve the purpose of fitting the conditional probability distribution of related variables. The description of the first model is as follows. First, make a statistical assumption that the confounding variable Z obeys a normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, and the expectation and variance of its distribution are related to the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m. Subsequently, a fully connected neural network g (t, x, m) is established with the value of the treatment variable t, the value of the observable proxy variable x, and the value of the adjustment variable m as input and the variance and expectation of the normal distribution obeyed by Z as output. The specific structure of the network g should include a fully connected network from the input to the middle hidden layer, and two independent fully connected branches connected after the hidden layer, which respectively output the expectation and variance of the normal distribution of the confounding variable Z. The description of the second model is as follows. First, make a statistical assumption that the effect variable Y follows a normal distribution under the given conditions of the treatment variable T, the confounding variable Z, and the adjustment variable M, and the expectation and variance of its distribution are related to the value of the treatment variable t, the value of the confounding variable z, and the value of the adjustment variable m. Next, a fully connected neural network f (t, x, m) is established with the value of the treatment variable t, the value of the confounding variable z, and the value of the adjustment variable m as input and the variance and expectation of the normal distribution that Y follows as output. The specific structure of the network g should include a fully connected network from the input to the middle hidden layer, a nonlinear activation layer after the hidden layer, and two independent fully connected branches connected after the activation layer, which respectively output the expectation and variance of the normal distribution of the effect variable Y.

[0092] Set the prior probability distribution p(z) that the confounding variable Z obeys, generally using the multivariate normal distribution.

[0093] Based on the two fully connected neural network models established above, a complete data set without missing items is used as the training data set, and the parameters of the neural network model are learned by optimizing the variational lower bound (ELBO) objective function. During the training process, the neural network model g(t,x,m) is first used to estimate the distribution parameters (expectation and variance) of the confounding variable Z, and Z is sampled from the distribution. Then, the sampled Z is input into the neural network model f(t, z, m) together with the treatment variable T and the adjustment variable M to estimate the conditional distribution parameters (expectation and variance) of the effect variable Y. Based on this, the objective function ELBO is transformed into, During the optimization process, the negative value of ELBO is used as the loss function, and the model parameters are iteratively updated through optimization algorithms such as gradient descent until the objective function converges, thereby obtaining a trained neural network model that can effectively represent the causal relationship between each variable.

[0094] Finally, using the prior distribution p(Z) of the confounding variable Z and the conditional probability distribution p(Y|Z,M,T) predicted by the trained neural network model, the Bayesian formula can be used to calculate the predicted distribution p(Y|M,T) of the effect variable Y given the adjustment variable M and the treatment variable T, that is, the predicted target. Among them, p(y|z,m,t) is the conditional probability distribution learned by the neural network model f(t, z, m), which predicts the distribution parameters (expectation and variance) of Y given Z, M, and T. p(Z) is the prior distribution of the confounding variable Z, which is usually assumed to be a standard normal distribution. .

[0095] Implementation Cases:

[0096] Taking practical applications as an example, the evaluation of personalized advertising effectiveness is a typical problem that the prediction system attempts to solve, and it is also a typical problem of causal reasoning, because the advertising effect is not only affected by the advertising strategy, but may also be affected by many factors such as user purchasing habits, browsing behavior, and external time factors (such as weekends). Some of these factors are observable, while some (such as purchasing habits) are hidden confounding variables that cannot be directly observed. This method clearly defines the problem variables, fills in missing data, models the distribution of hidden variables and outcome variables, and ultimately achieves accurate prediction and optimization of advertising effectiveness through the following steps.

[0097] 1. Clearly define the variables that the prediction system focuses on. In the advertising delivery scenario, it is necessary to define the key variables in the system and clarify the causal relationship between them for subsequent modeling. The specific variables are as follows.

[0098] Effect variable Y: Advertising effect is the core prediction target of this system, which can usually be measured by advertising conversion rate (such as the proportion of users who purchase products after clicking on an ad) or sales.

[0099] Treatment variable T: advertising delivery strategy, which refers to the advertising delivery method selected by the platform, including delivery duration, display frequency, delivery budget, display channel, etc. These strategies will directly affect the advertising effect.

[0100] Hidden confounding variable Z: user's purchasing habits, which is a potential variable that cannot be directly observed, but will affect both the advertising strategy T and the advertising effect Y. For example, customers with good purchasing habits may be more likely to convert under high-frequency advertising, while customers with unclear purchasing habits may need other strategies.

[0101] Proxy variable X: The price of the products that the customer browsed within 10 days, such as the mean, highest price, standard deviation of the price, and other statistical characteristics. Proxy variables can indirectly reflect the user's purchasing habits, so they can be used as an observation substitute for the hidden confounding variable Z.

[0102] Adjustment variable M: ​​Weekend marker, which is an observable external factor that indicates whether the customer's behavior occurred on the weekend when the data was collected. Weekends usually affect customers' browsing and purchasing behaviors. For example, if the frequency of shopping increases on weekends, the advertising effect may increase accordingly.

[0103] Description of causal relationship: Advertising strategy T and hidden confounding variable Z jointly affect advertising effect Y. At the same time, proxy variable X and adjustment variable M are also associated with Z and T.

[0104] 2. Dataset acquisition and missing data feature interpolation (FCSMI method)

[0105] In practical applications, advertising data often contains missing data features, such as incomplete data on the price of a customer's browsed item, or missing records in the advertising strategy. To this end, the FCSMI (Fully Conditional Standardized Multiple Imputation) method is used for imputation to ensure data integrity and improve model training results.

[0106] Initial mean interpolation: For numerical features (such as the price X of the products viewed by customers within 10 days), calculate the mean of all observed values ​​and use this mean to preliminarily fill in the missing data features. For logical features (such as whether a certain type of link has been clicked), calculate the mode (the most frequently occurring category) of the feature and use the mode to fill in the missing values. This stage aims to quickly obtain an initial complete data set to provide a basis for subsequent modeling.

[0107] Then, for numerical features, a linear regression model is used with other observed variables as input to predict the missing values ​​of numerical features. For logical features, a logistic regression model is used with other observed variables as input to predict the probability of missing categories of logical features, and the category with the highest probability is selected for filling.

[0108] Iterative interpolation and convergence check: Use the model parameters obtained in the previous step to interpolate the data set and evaluate the data integrity and model convergence. If the model parameters have not converged, re-execute full conditional interpolation and model prediction until the model converges.

[0109] 3. Inference on the distribution of hidden confounding variable Z Assume that hidden confounding variable Z (customer purchasing habits) is normally distributed under the conditions of treatment variable T, proxy variable X and adjustment variable M. Establish a fully connected neural network model, input proxy variable X, treatment variable T, effect variable Y, and output the parameters of normal distribution, namely mean μ and variance σ 2 .

[0110] 4. Construct the conditional probability distribution model of the outcome variable Y Next, assume that the advertising effect Y follows a normal distribution under the conditions of hidden variable Z, treatment variable T and adjustment variable M. Build a fully connected neural network model, input adjustment variable M, treatment variable T, hidden variable Z, and output the distribution parameters of advertising effect Y (mean μ, variance σ 2 ).

[0111] 5. Train model parameters with variational lower bound (ELBO) as the objective function In this step, the two neural network models mentioned above are optimized through the variational inference framework. To this end, the variational lower bound (ELBO, Evidence Lower Bound) is selected as the objective function to jointly optimize the parameters of the two models. This process approximates the log-likelihood of the data by maximizing ELBO, thereby improving the predictive ability and generalization performance of the model. In probabilistic inference, the goal is to calculate the joint distribution and marginal probability distribution of observed data (such as advertising effects, delivery strategies) under latent variables (hidden confounding variables Z). However, due to the introduction of hidden variables, directly solving these distributions is often computationally complex or even unsolvable, so variational inference is needed to approximate the solution. .

[0112] 6. Apply the Bayesian formula to make predictions by setting the prior distribution of the hidden confounding variable Z, e.g. , combined with the conditional probability distribution of Y, the predicted distribution of advertising effect is finally obtained through Bayesian formula and integration .

[0113] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. An artificial intelligence prediction method based on reasoning enhancement, characterized in that: include: Obtain the definition of the variables of interest in the prediction system, define the effect variable Y, the treatment variable T, the confounding variable Z, the observable proxy variable X, the adjustment variable M, and the causal relationship between the variables; According to the definition of variables, obtain the data set of relevant variables; Based on the obtained data set of relevant variables, the missing data characteristics of the data in the data set were imputed using the fully conditional standardized multiple imputation FCSMI to obtain a complete data set without missing items; According to the causal relationship between the variables, assuming that the confounding variable Z obeys the normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as input, and the expectation and variance of the normal distribution obeyed by Z as output; According to the causal relationship between the variables, assuming that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output; Set the value z of the confounding variable to obey the prior probability distribution; Based on the two fully connected neural network models established above, a complete data set without missing items is used as the training data set, and the variational lower bound ELBO is used as the objective function to obtain the parameters of the neural network model through training; Based on the parameters of the trained neural network model, calculate the conditional probability distribution P(y|z,m,t) of the effect variable Y; Based on the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the effect variable Y and any given value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t, and the prior probability distribution obeyed by the value of the confounding variable z, the Bayesian formula is applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given.

2. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: The definition of the variables concerned in the prediction system is to clarify the role and observable conditions of each variable in the prediction system, namely: The effect variable Y is the main target of the prediction system's prediction results; The treatment variable T is the variable that mainly affects the effect variable Y; The confounding variable Z is a variable that affects both the treatment variable T and the effect variable Y; An observable proxy variable X is an observable variable X that is affected by a confounding variable Z; The adjustment variable M is the external factor that affects the effect variable.

3. The artificial intelligence prediction method based on reasoning enhancement according to claim 1, characterized in that: According to the obtained data set, the missing items in the data set are interpolated using the fully conditional standardized multiple imputation FCSMI to obtain a complete data set without missing items, specifically: 1) Perform mean interpolation on the missing data features in the obtained data set of relevant variables to obtain the initial complete data set; 2) Further adjust the data features in the initial complete data set, use other data features as input and the missing data features in the obtained data set of relevant variables as output, and use the initial complete data set as the training data set to establish and train a linear regression model or a logistic regression model; 3) Use the linear regression model or logistic regression model obtained in the previous step to interpolate the missing data features in the data set of the obtained relevant variables. If it does not converge, return to step 2) and loop until convergence.

4. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: According to the causal relationship between the variables, it is assumed that the confounding variable Z obeys the normal distribution under the given conditions of the processing variable T, the observable proxy variable X, and the adjustment variable M, and a fully connected neural network model is constructed with the value of the processing variable t, the value of the proxy variable x, and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Z as output; Wherein, it is assumed that the confounding variable Z obeys a normal distribution under the given conditions of the processing variable T, the observable proxy variable X, and the adjustment variable M, and the expectation and variance of its distribution are functions of the observable proxy variable X, the adjustment variable M, and the processing variable T. The input of the neural network is the value t of the processing variable, the value x of the proxy variable, and the value m of the adjustment variable. The feature representation learning of the input variables is performed through several hidden layers, and finally the parameters of the distribution of the value z of the confounding variable are output; ； Including the expectation of the normal distribution and variance σ 2 : ； Among them, g is a fully connected neural network model.

5. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: According to the causal relationship between the variables, it is assumed that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, and a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output; Wherein, it is assumed that the effect variable Y obeys the normal distribution under the given conditions of the confounding variable Z, the treatment variable T, and the adjustment variable M, and the expectation and variance of its distribution are functions of the confounding variable Z, the adjustment variable M, and the treatment variable T. The input of the neural network is the value z of the confounding variable, the value m of the adjustment variable, and the value t of the treatment variable. The feature representation learning of the input variables is performed through several hidden layers, and finally the parameters of the distribution of the value y of the effect variable are output; ； Including the expectation μ and variance σ of the normal distribution 2 : ； Among them, f is a fully connected neural network model with a nonlinear activation layer.

6. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: The two fully connected neural network models established above use a complete data set without missing items as a training data set, and use the variational lower bound ELBO as the objective function. The parameters of the neural network model obtained through training, the variational lower bound ELBO as the objective function are specifically: 。 7. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: The conditional probability distribution p(y|z,m,t) based on the conditional probability distribution of the effect variable Y and any given value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t and the prior probability distribution obeyed by the value of the confounding variable z are applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given: Using the prior distribution p(z) of the confounding variable z and the conditional probability distribution p(y|z,m,t) predicted by the trained neural network model, the predicted distribution of the outcome variable y under the given adjustment variable m and treatment variable t is calculated using the Bayesian formula: 。

Images