An artificial intelligence prediction method based on reasoning enhancement

Abstract

The present invention discloses an artificial intelligence prediction method based on reasoning enhancement. Selective bias correction effectively reduces the impact of selective bias, thereby improving the accuracy of prediction. Compared with the traditional regression method that assumes consistent data distribution, the present invention can effectively eliminate pseudo-correlation and identify the real causal effect, thereby showing higher prediction accuracy in high-bias scenarios. It shows significant robustness and stability in scenarios where data features are missing. The use of fully conditional standardized multiple interpolation to obtain a complete data set without missing items is better than traditional simple interpolation. It not only reduces the interpolation bias, but also significantly improves the robustness of the model in scenarios with high missing data, and can more comprehensively utilize external factor information and improve adaptability to complex scenarios. The present invention explicitly models external factors. This specialized treatment of external factors enables the present invention to better capture the impact of external information in complex environments and improve prediction results.

Description

Technical Field

[0001] The present invention relates to the field of data processing and prediction systems, and in particular to an artificial intelligence prediction method based on reasoning enhancement. Background Art

[0002] In practical applications, existing AI prediction methods often suffer from significant degradation in accuracy and stability when handling complex scenarios due to selection bias, missing data, and the influence of external factors. In predicting the effectiveness of marketing campaigns, customer characteristics (such as age and income) often influence both the allocation of marketing strategies (treatment variables) and the customer's response (effect variables). This dual influence introduces selection bias into the data, preventing the model from accurately capturing true causal relationships and potentially generating spurious correlations, further weakening the reliability of predictions. Furthermore, much customer information crucial to the model (such as socioeconomic status or purchase intention) is often hidden, unobservable, or partially missing, significantly increasing the difficulty of modeling. Furthermore, in real-world marketing scenarios, the data available at the start of a campaign is often incomplete, forcing models to make predictions based on limited observational information. This places higher demands on the robustness of the prediction system under high uncertainty. Given these challenges, an effective prediction method must not only correct for data bias but also possess robust capabilities for handling missing data and adapt to changes in external factors to achieve high accuracy and stability in complex scenarios.

[0003] Current AI prediction methods have been widely studied in academia and industry, and some technologies have achieved remarkable results. For example, traditional regression prediction methods include Lasso regression [1], support vector regression (SVR) [2], and XGBoost [3], which train models using historical data to predict target variables. These methods use the underlying statistical patterns in the data to build models, can effectively deal with nonlinear relationship problems, and have been successfully applied in many fields (such as marketing and financial forecasting). However, these methods usually assume that the data distribution is consistent, and it is difficult to deal with the selective bias caused by the inconsistent distribution of characteristics of different subgroups in real scenarios. In addition, when faced with missing data, these methods often use simple interpolation techniques (such as mean interpolation or zero filling), which may ignore the uncertainty of the data and thus affect the accuracy of the prediction.

[0004] On the other hand, causal inference methods have become an important tool for solving the problem of prediction bias in recent years. This type of method is based on the potential outcome framework (POF) and estimates the effects of intervention measures by modeling causal relationships. Typical methods include TARNET[4] and Dragonnet[5], which can effectively correct the bias introduced by confounding variables and perform well in effect estimation tasks. However, the core goal of these methods is to estimate the treatment effect (i.e., the impact of the intervention on the outcome) rather than predict future effects. When it is necessary to predict the effects of new scenarios or new customers, the performance of these methods may be insufficient. In addition, most causal inference methods assume that the confounding variables are fully observed. However, in actual business scenarios, many confounding variables are hidden and can only be inferred indirectly through proxy variables. At the same time, these methods usually assume that the data is complete and do not fully consider the prevalence and complexity of missing data in reality.

[0005] While some deep learning techniques (such as variational autoencoders (VAEs)) have attempted to incorporate generative models into causal inference to model hidden confounding variables and handle complex data distributions, their application to missing data, modeling external factors, and explaining causal relationships remains limited. Overall, existing techniques remain limited in addressing selection bias, handling missing data, and modeling complex external factors, particularly in future-oriented forecasting tasks. These techniques are difficult to directly apply to the complex scenarios mentioned in the article, such as predicting marketing campaign effectiveness where multiple missing data and external factors are jointly influential.

[0006] The existing technology has the following problems:

[0007] 1) Difficulty in effectively addressing selection bias

[0008] Traditional forecasting methods assume a uniform data distribution and are unable to address the problem of selective bias caused by inconsistent distributions of characteristics across subpopulations. While causal inference methods can correct for bias caused by confounding variables, they are insufficient when predicting novel scenarios.

[0009] 2) Limited ability to handle missing data

[0010] Existing technologies typically use simple interpolation methods (such as mean imputation or zero padding) to handle missing data. This ignores data uncertainty and leads to reduced prediction accuracy. Furthermore, most causal inference methods assume complete data and fail to fully consider the prevalence and complexity of missing data.

[0011] 3) Insufficient modeling of complex external factors and hidden confounding variables

[0012] Most current causal inference methods assume that confounding variables are fully observable. When faced with hidden confounding variables, they rely on indirect inference using proxy variables, but this approach is limited in effectiveness. Furthermore, existing technologies are less adaptable to changes in external factors and cannot maintain robust predictions under conditions of high uncertainty.

[0013] Prior art references:

[0014] [1] Volker Roth. 2004. The generalized LASSO. IEEE transactions onneural networks15, 1 (2004), 16–28.

[0015] [2] Mariette Awad and Rahul Khanna. 2015. Support vector regression.In Efficient learning machines. Springer, 67–80.

[0016] [3] Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable treeboosting system.In Proceedings of the 22nd acm sigkdd internationalconference on knowledge discovery and data mining. 785–794.

[0017] [4] Fredrik Johansson, Uri Shalit, and David Sontag. 2016. Learningrepresentations for counterfactual inference. In International conference onmachine learning.PMLR, 3020–3029.

[0018] [5] Claudia Shi, David Blei, and Victor Veitch. 2019. Adapting neuralnetworks for the estimation of treatment effects. Advances in neuralinformation processing systems 32 (2019). Summary of the Invention

[0019] In view of the shortcomings of the existing technology, the present invention provides an artificial intelligence prediction method based on reasoning enhancement. The present invention is achieved through the above technical solutions:

[0020] The present invention discloses an artificial intelligence prediction method based on reasoning enhancement, comprising:

[0021] Obtain the definition of the variables of interest in the prediction system, define the effect variable Y, treatment variable T, confounding variable Z, observable proxy variable X, adjustment variable M, and the causal relationship between the variables;

[0022] According to the definition of variables, obtain the data set of relevant variables;

[0023] Based on the obtained dataset of relevant variables, the missing data characteristics of the data in the dataset were imputed using fully conditional standardized multiple imputation (FCSMI) to obtain a complete dataset without missing items;

[0024] According to the causal relationship between the variables, assuming that the confounding variable Z obeys the normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Z as output;

[0025] According to the causal relationship between the variables, assuming that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output;

[0026] Set the prior probability distribution of the value z of the confounding variable;

[0027] Based on the two fully connected neural network models established above, the complete dataset without missing items is used as the training dataset, and the variational lower bound ELBO is used as the objective function to obtain the parameters of the neural network model through training;

[0028] Based on the parameters of the trained neural network model, calculate the conditional probability distribution P(y|z,m,t) of the effect variable Y;

[0029] Based on the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the effect variable Y and any given value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t, and the prior probability distribution obeyed by the value of the confounding variable z, the Bayesian formula is applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given.

[0030] As a further improvement, the definition of the variables of interest in the prediction system described in the present invention is specifically to clarify the role and observable conditions of each relevant variable in the prediction system, namely:

[0031] The effect variable Y is the main target of the prediction system's prediction results;

[0032] The treatment variable T is the variable that mainly affects the effect variable Y;

[0033] The confounding variable Z is a variable that affects both the treatment variable T and the effect variable Y;

[0034] An observable proxy variable X is an observable variable X that is affected by a confounding variable Z;

[0035] The adjustment variable M is the external factor that affects the effect variable.

[0036] As a further improvement, the present invention uses the fully conditional canonical multiple imputation (FCSMI) to interpolate missing items in the data set based on the obtained data set to obtain a complete data set without missing items, specifically:

[0037] 1) Perform mean interpolation on the missing data features in the obtained dataset of relevant variables to obtain the initial complete dataset;

[0038] 2) Further adjust the data features in the initial complete dataset. For the missing data features in the dataset of the obtained relevant variables, use other data features as input and the missing data features in the dataset of the obtained relevant variables as output, and use the initial complete dataset as the training dataset to establish and train a linear regression model or a logistic regression model;

[0039] 3) Use the linear regression model or logistic regression model obtained in the previous step to interpolate the missing data features in the obtained dataset of relevant variables. If convergence is not achieved, return to step 2) and repeat the loop until convergence is achieved.

[0040] As a further improvement, according to the present invention, based on the causal relationship between the variables, it is assumed that the confounding variable Z obeys a normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, and a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as inputs and the expectation and variance of the normal distribution obeyed by Z as outputs;

[0041] Here, it is assumed that the confounding variable Z follows a normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M. The expectation and variance of its distribution are functions of the observable proxy variable X, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the confounding variable value z are finally output.

[0042] Including the expectation of the normal distribution and variance σ 2 :

[0043]

[0044] Among them, g is a fully connected neural network model.

[0045] As a further improvement, according to the present invention, based on the causal relationship between the variables, it is assumed that the effect variable Y obeys a normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, and a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output;

[0046] Here, it is assumed that the effect variable Y obeys the normal distribution under the given conditions of the confounding variable Z, the treatment variable T, and the adjustment variable M. The expectation and variance of its distribution are functions of the confounding variable Z, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the effect variable value y are finally output.

[0047]

[0048] Including the expectation of the normal distribution and variance σ 2

[0049]

[0050] Among them, f is a fully connected neural network model with a nonlinear activation layer.

[0051] As a further improvement, the two fully connected neural network models established above described in the present invention use a complete data set without missing items as a training data set, and use the variational lower bound ELBO as the objective function. The parameters of the neural network model are obtained through training, and the variational lower bound ELBO is the objective function specifically as follows:

[0052] .

[0053] As a further improvement, the present invention applies the conditional probability distribution p(y|z,m,t) based on the conditional probability distribution of the effect variable Y and any given confounding variable value z, adjustment variable value m, and treatment variable value t, and the prior probability distribution obeyed by the confounding variable value z, and applies the Bayesian formula to obtain a prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given:

[0054] Using the prior distribution p(z) of the confounding variable z and the conditional probability distribution p(y|z,m,t) predicted by the trained neural network model, the predicted distribution of the outcome variable y under the given adjustment variable m and treatment variable t is calculated using the Bayesian formula:

[0055] .

[0056] The beneficial effects of the present invention are as follows:

[0057] 1) Selectivity bias correction effectively reduces the impact of selectivity bias, thereby improving prediction accuracy. This method obtains the definition of the variables of interest to the prediction system, defines the effect variable Y, the treatment variable T, the confounding variable Z, the observable proxy variable X, the adjustment variable M, and the causal relationship between the variables. Based on the causal relationship between the variables, multiple neural network models and regression models are established to correct the selectivity bias caused by confounding variables. Compared with traditional regression methods that assume consistent data distribution, this method can effectively eliminate spurious correlations and identify true causal effects, thereby demonstrating higher prediction accuracy in high-bias scenarios.

[0058] 2) Missing data feature processing: This method demonstrates remarkable robustness and stability in scenarios with missing data features. This method employs fully conditional specification of multiple imputation (FCSMI) to obtain a complete dataset without missing items. This method outperforms traditional simple imputation (such as mean imputation), not only reducing imputation bias but also significantly improving the model's robustness in scenarios with high levels of missing data.

[0059] 3) External factor modeling enables more comprehensive utilization of external factor information and improves adaptability to complex scenarios. This invention explicitly models external factors by defining an adjustment variable, M. This specialized treatment of external factors enables the invention to better capture the impact of external information in complex environments, improving prediction performance.

[0060] 4) Confounding variable inference: By modeling the conditional probability distribution of confounding variables under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, this method enhances the accuracy of predictions, the reliability of the results, and the versatility of the algorithm. By using a neural network model, the present invention can infer hidden confounding variables from observed data and incorporate them into the prediction framework. This overcomes the deficiency of traditional causal inference methods that assume complete observation of confounding variables, not only enhancing the model's predictive power and improving the reliability of the results, but also expanding the model's scope of reasoning.

[0061] These key technologies enable the present invention to significantly outperform traditional methods in selective bias correction, missing data processing, external factor modeling, and hidden confounding variable inference, providing more accurate, robust, and reliable solutions for complex prediction tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 This is an algorithm flow chart of an artificial intelligence prediction method based on reasoning enhancement in the present invention. DETAILED DESCRIPTION

[0063] In order to facilitate ordinary technicians in this field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention and are not used to limit the present invention. Figure 1 This is an algorithm flow chart of an artificial intelligence prediction method based on reasoning enhancement in the present invention;

[0064] The present invention discloses an artificial intelligence prediction method based on reasoning enhancement, comprising: obtaining definitions of variables of interest of a prediction system, defining effect variables Y, processing variables T, confusion variables Z, observable proxy variables X, adjustment variables M, and causal relationships between the variables; the effect variable Y is the main target of the prediction result of the prediction system;

[0065] The treatment variable T is the variable that mainly affects the effect variable Y;

[0066] The confounding variable Z is a variable that affects both the treatment variable T and the effect variable Y;

[0067] An observable proxy variable X is an observable variable X that is affected by a confounding variable Z;

[0068] The adjustment variable M is an external factor that affects the effect variable.

[0069] According to the definition of variables, obtain the data set of relevant variables;

[0070] Based on the obtained dataset of relevant variables, the missing data characteristics of the data in the dataset are imputed using fully conditional standardized multiple imputation (FCSMI) to obtain a complete dataset without missing items; specifically:

[0071] 1) Perform mean interpolation on the missing data features in the obtained dataset of relevant variables to obtain the initial complete dataset;

[0072] 2) Further adjust the data features in the initial complete dataset. For the missing data features in the dataset of the obtained relevant variables, use other data features as input and the missing data features in the dataset of the obtained relevant variables as output, and use the initial complete dataset as the training dataset to establish and train a linear regression model or a logistic regression model;

[0073] 3) Use the linear regression model or logistic regression model obtained in the previous step to interpolate the missing data features in the obtained dataset of relevant variables. If convergence is not achieved, return to step 2) and repeat the loop until convergence is achieved.

[0074] According to the causal relationship between the variables, assuming that the confounding variable Z obeys the normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M, a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Z as output;

[0075] Assume that the confounding variable Z follows a normal distribution under the given conditions of the treatment variable T, the observable proxy variable X, and the adjustment variable M. The expectation and variance of its distribution are functions of the observable proxy variable X, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the confounding variable value z are finally output.

[0076] Including the expectation of the normal distribution and variance σ 2 :

[0077]

[0078] Among them, g is a fully connected neural network model.

[0079] According to the causal relationship between the variables, assuming that the effect variable Y obeys the normal distribution under the given conditions of the treatment variable T, the confounding variable Z and the adjustment variable M, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z and the value of the adjustment variable m as input and the expectation and variance of the normal distribution obeyed by Y as output;

[0080] Here, it is assumed that the effect variable Y obeys the normal distribution under the given conditions of the confounding variable Z, the treatment variable T, and the adjustment variable M. The expectation and variance of its distribution are functions of the confounding variable Z, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t. The feature representation of the input variables is learned through several hidden layers, and the parameters of the distribution of the effect variable value y are finally output.

[0081]

[0082] Including the expectation of the normal distribution and variance σ 2：

[0083]

[0084] Among them, f is a fully connected neural network model with a nonlinear activation layer.

[0085] Set the prior probability distribution of the value z of the confounding variable;

[0086] Based on the two fully connected neural network models established above, the complete dataset without missing items is used as the training dataset, and the variational lower bound ELBO is used as the objective function to obtain the parameters of the neural network model through training;

[0087] The variational lower bound ELBO is the objective function specifically:

[0088] .

[0089] Based on the parameters of the trained neural network model, calculate the conditional probability distribution P(y|z,m,t) of the effect variable Y;

[0090] Based on the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the effect variable Y and any given value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t, and the prior probability distribution obeyed by the value of the confounding variable z, the Bayesian formula is applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the effect variable Y when the treatment variable value y and the adjustment variable value m are given;

[0091] Using the prior distribution p(z) of the confounding variable z and the conditional probability distribution p(y|z,m,t) predicted by the trained neural network model, the predicted distribution of the outcome variable y under the given adjustment variable m and treatment variable t is calculated using the Bayesian formula:

[0092] .

[0093] Furthermore, this invention accurately describes the definitions of variables in a prediction system and their causal relationships, employing a systematic approach to clarify the roles of relevant variables and their placement in the prediction model, while also clarifying the causal relationships between variables. First, the following key variables are defined and their roles and functions are clarified. The effect variable is the primary target of the prediction system, representing the desired end result. For example, in sales forecasting, the effect variable might be sales for a specific future period. In a causal relationship, the effect variable is typically set as the target node, and the effects of all other variables are directed towards Y through direct or indirect pathways. The treatment variable T is the core factor that primarily influences the effect variable Y and is the key variable that the prediction system aims to study or optimize. For example, in an advertising scenario, the treatment variable T might be the amount of advertising delivered, and its impact on Y is typically modeled through a direct pathway. Confounding variables act on both the treatment variable T and the effect variable Y, and their presence can lead to deviations in the relationship between T and Y. For example, in consumer behavior research, seasonality can influence both advertising delivery (the treatment variable) and directly affect sales (the effect variable). Therefore, accurately identifying confounding variables is a crucial step in ensuring clear causal relationships. Since the confounding variable Z is often difficult to observe directly, the present invention introduces an observable proxy variable X to indirectly reflect the influence of Z by measuring X. For example, seasonal factors (confounding variables) can be reflected by temperature or precipitation (proxy variables). The adjustment variable M is a factor that produces external interference on the effect variable Y, such as the macroeconomic environment or policy changes. The existence of the adjustment variable needs to be included in the causal analysis to ensure the accuracy and reliability of the prediction model. Next, in order to more intuitively show the relationship between the variables and their role in the system, it is necessary to clarify the causal relationship between the variables by constructing a causal diagram. In the causal diagram, Figure 1 In the causal diagram, the treatment variable T has a direct impact on the effect variable Y, so the causal relationship is represented by the arrow T→Y. Simultaneously, the confounding variable Z affects both T and Y through two paths: Z→T and Z→Y. To capture the influence of Z, an observable proxy variable X is included in the diagram and represented by the path Z→X. Furthermore, the manipulation variable M acts as an external confounding variable and directly affects the effect variable Y, represented in the causal diagram by M→Y.

[0094] It is necessary to clarify the data source according to the definition of each variable, collect and integrate it, and obtain the data set of relevant variables in the prediction system.

[0095] For missing data features in the dataset of relevant variables in the prediction system, the fully conditional canonical multiple imputation (FCSMI) method is used to impute the missing data to generate a complete dataset without missing items. The specific method is described below. First, mean imputation is performed on the missing data features in the dataset, replacing the missing items with the mean value of each corresponding feature. This step obtains an initial complete dataset, providing a reasonable starting point for subsequent imputation methods. Next, for each missing data feature, a regression model is established using other features as input variables and the missing data feature as the output variable. Depending on the data feature type, a linear regression model is used for continuous missing data features; a logistic regression model is used for categorical and logical missing data features. The initial complete dataset is used as the training dataset to train the established regression model to capture the relationship between features. Subsequently, the trained regression model is used to predict missing features and replace missing values ​​with predicted values ​​to generate a new complete dataset. The difference between the interpolated values ​​in the new complete dataset and the previous complete dataset is checked to determine whether the interpolation has converged. If not, the process returns to the previous step, updates the regression model, and re-interpolates, repeating this process until the interpolated values ​​converge. Using this FCSMI interpolation method, a complete dataset with no missing items is ultimately obtained, providing a reliable data foundation for subsequent analysis and modeling.

[0096] Causal inference and modeling are performed based on the causal relationships between variables. Two neural network models are required to fit the conditional probability distributions of the relevant variables. The first model is described below. First, a statistical assumption is made: the confounding variable Z follows a normal distribution given the treatment variable T, the observable proxy variable X, and the adjustment variable M. The expectation and variance of this distribution are related to the treatment variable value t, the proxy variable value x, and the adjustment variable value m. Subsequently, a fully connected neural network g(t, x, m) is constructed, taking the treatment variable value t, the observable proxy variable value x, and the adjustment variable value m as inputs and the variance and expectation of the normal distribution of Z as outputs. The specific structure of network g should include a fully connected network from the input to the intermediate hidden layer, and two independent fully connected branches after the hidden layer, which respectively output the expectation and variance of the normal distribution of the confounding variable Z. The second model is described below. First, make the statistical assumption that the effect variable Y follows a normal distribution given the treatment variable T, the confounding variable Z, and the adjustment variable M. The expectation and variance of this distribution are related to the treatment variable value t, the confounding variable value z, and the adjustment variable value m. Next, establish a fully connected neural network f(t, x, m) with the treatment variable value t, the confounding variable value z, and the adjustment variable value m as inputs and the variance and expectation of the normal distribution of Y as outputs. The specific structure of network g should include a fully connected network from the input to the intermediate hidden layer, a nonlinear activation layer after the hidden layer, and two independent fully connected branches after the activation layer, which respectively output the expectation and variance of the normal distribution of the effect variable Y.

[0097] Set the prior probability distribution p(z) that the confounding variable Z obeys, generally using the multivariate normal distribution.

[0098] Based on the two fully connected neural network models established above, a complete dataset without missing items is used as the training dataset, and the parameters of the neural network model are learned by optimizing the variational lower bound (ELBO) objective function. During the training process, the neural network model g(t,x,m) is first used to estimate the distribution parameters (expectation and variance) of the confounding variable Z, and Z is sampled from this distribution. Then, the sampled Z is input into the neural network model f(t, z, m) together with the treatment variable T and the adjustment variable M to estimate the conditional distribution parameters (expectation and variance) of the effect variable Y. Based on this, the objective function ELBO is transformed into, During the optimization process, the negative value of ELBO is used as the loss function, and the model parameters are iteratively updated through optimization algorithms such as gradient descent until the objective function converges, thereby obtaining a trained neural network model that can effectively represent the causal relationship between each variable.

[0099] Finally, using the prior distribution p(Z) of the confounding variable Z and the conditional probability distribution p(Y|Z,M,T) predicted by the trained neural network model, the predicted distribution p(Y|M,T) of the effect variable Y under the given adjustment variable M and treatment variable T can be calculated using the Bayesian formula, that is, the predicted target. Among them, p(y|z,m,t) is the conditional probability distribution learned by the neural network model f(t, z, m), which predicts the distribution parameters (expectation and variance) of Y under given Z, M, T conditions. p(Z) is the prior distribution of the confounding variable Z, which is usually assumed to be a standard normal distribution. .

[0100] Implementation Cases:

[0101] Taking a practical example, evaluating the effectiveness of personalized advertising is a typical problem that prediction systems attempt to solve, and it's also a classic problem for causal inference. This is because advertising effectiveness is influenced not only by the advertising strategy but also by multiple factors, such as user purchasing habits, browsing behavior, and external time factors (such as weekends). Some of these factors are observable, while others (such as purchasing habits) are hidden confounding variables that are not directly observable. This method uses the following steps to clearly define the problem variables, fill in missing data, model the distribution of the hidden variables and the outcome variable, and ultimately achieve accurate prediction and optimization of advertising effectiveness.

[0102] 1. Clearly define the variables that the prediction system focuses on. In the advertising delivery scenario, it is necessary to define the key variables in the system and clarify the causal relationships between them to facilitate subsequent modeling. The specific variables are as follows.

[0103] Effect variable Y: Advertising effect is the core prediction target of this system and can usually be measured by advertising conversion rate (such as the proportion of users who purchase products after clicking on an ad) or sales.

[0104] Treatment variable T: Ad placement strategy, which refers to the platform's chosen method of ad placement, including factors such as duration, display frequency, budget, and display channels. These strategies directly impact ad effectiveness.

[0105] Hidden confounding variable Z: User purchasing habits. This is a latent variable that cannot be directly observed, but it will affect both the advertising strategy T and the advertising effect Y. For example, customers with good purchasing habits may be more likely to convert with high-frequency advertising, while customers with less obvious purchasing habits may require other strategies.

[0106] Proxy variable X: The prices of items viewed by customers over the past 10 days, including statistical characteristics such as the mean, highest price, and standard deviation of the item price. Proxy variables can indirectly reflect a user's purchasing habits and thus serve as observational substitutes for the hidden confounding variable Z.

[0107] Adjustment variable M: ​​Weekend marker. This is an observable external factor that indicates whether customer activity occurred on weekends at the time of data collection. Weekends often influence customer browsing and purchasing behavior. For example, increased shopping frequency on weekends may lead to improved advertising effectiveness.

[0108] Causal relationship description: Advertising strategy T and hidden confounding variable Z jointly affect advertising effect Y. At the same time, proxy variable X and adjustment variable M are also associated with Z and T.

[0109] 2. Dataset Acquisition and Missing Data Feature Interpolation (FCSMI Method)

[0110] In real-world applications, advertising data often contains missing data, such as incomplete price data for items viewed by customers or missing records in advertising strategies. To address this, we use the FCSMI (Fully Conditional Specification Multiple Imputation) method to ensure data integrity and improve model training performance.

[0111] Initial mean imputation: For numerical features (e.g., the price X of items viewed by a customer over the past 10 days), the mean of all observed values ​​is calculated and used to preliminarily impute missing data. For logical features (e.g., whether a certain type of link has been clicked), the mode (the most frequently occurring category) of the feature is calculated and used to impute missing values. This stage aims to quickly obtain an initial, complete dataset, providing a foundation for subsequent modeling.

[0112] Then, for numerical features, a linear regression model is used with other observed variables as input to predict missing values ​​of the numerical features. For logistic features, a logistic regression model is used with other observed variables as input to predict the probability of missing classes of the logistic features, and the class with the highest probability is selected for filling.

[0113] Iterative interpolation and convergence check: Use the model parameters obtained in the previous training step to interpolate the dataset and evaluate data integrity and model convergence. If the model parameters have not converged, re-run full conditional interpolation and model prediction until the model converges.

[0114] 3. Infer the distribution of the hidden confounding variable Z. Assume that the hidden confounding variable Z (customer purchasing habits) follows a normal distribution under the conditions of the treatment variable T, the proxy variable X, and the adjustment variable M. Build a fully connected neural network model, input the proxy variable X, the treatment variable T, and the effect variable Y, and output the parameters of the normal distribution, namely the mean μ and variance σ. 2 .

[0115] 4. Construct a conditional probability distribution model for the outcome variable Y. Next, assume that the advertising effect Y follows a normal distribution under the conditions of the hidden variable Z, the treatment variable T, and the adjustment variable M. Build a fully connected neural network model, input the adjustment variable M, the treatment variable T, and the hidden variable Z, and output the distribution parameters of the advertising effect Y (mean μ, variance σ) 2 ).

[0116] 5. Train model parameters using the variational lower bound (ELBO) as the objective function. In this step, the two neural network models described above are optimized using a variational inference framework. To this end, the variational lower bound (ELBO) is selected as the objective function to jointly optimize the parameters of both models. This process approximates the log-likelihood of the data by maximizing the ELBO, thereby improving the model's predictive power and generalization performance. In probabilistic inference, the goal is to compute the joint distribution and marginal probability distribution of observed data (such as ad effectiveness and delivery strategy) under the influence of latent variables (hidden confounding variables Z). However, due to the introduction of hidden variables, directly solving for these distributions is often computationally complex or even impossible to analyze, so variational inference is required to approximate them. .

[0117] 6. Apply the Bayesian formula to make predictions by setting the prior distribution of the hidden confounding variable Z, e.g. , combined with the conditional probability distribution of Y, the predicted distribution of advertising effect is finally obtained through Bayesian formula and integration

[0118] .

[0119] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A method for predicting the effect of advertising delivery based on artificial intelligence with reasoning enhancement, characterized in that: include: Obtain the definitions of the variables of interest to the prediction system, defining the effect variable Y to represent the advertising effect, the processing variable T to represent the advertising delivery strategy, the confounding variable Z to represent the user's purchasing habits, the observable proxy variable X to represent the user's consumption and browsing behavior records, the adjustment variable M to represent the weekend external factors when the advertisement is delivered, and the causal relationship between the variables; According to the definition of variables, obtain the data set of relevant variables; Based on the obtained dataset of relevant variables, the missing data characteristics of the data in the dataset were imputed using fully conditional standardized multiple imputation (FCSMI) to obtain a complete dataset without missing items; Based on the causal relationship between the variables, we assume that the user's purchasing habits Z follow a normal distribution under the given conditions of the advertising strategy T, the user's consumption and browsing behavior record X, and the weekend external factor M during the advertising. We then construct a fully connected neural network model that takes the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as inputs and outputs the expectation and variance of the normal distribution that Z follows. Based on the causal relationship between the variables, assuming that the advertising effect Y follows a normal distribution under the given conditions of the advertising strategy T, the user's purchasing habits Z, and the weekend external factor M during the advertising, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z, and the value of the adjustment variable m as inputs, and the expectation and variance of the normal distribution followed by Y as outputs; Set the prior probability distribution of the user's purchasing habits z; Based on the two fully connected neural network models established above, a complete data set without missing items is used as the training data set, and the variational lower bound ELBO is used as the objective function. The parameters of the neural network model are obtained through training; based on the parameters of the trained neural network model, the conditional probability distribution P(y|z,m,t) of the advertising effect Y is calculated; based on the advertising effect Y and any given user's purchasing habits z, the weekend external factor m during advertising delivery, the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the advertising delivery strategy t and the prior probability distribution obeyed by the user's purchasing habits z, the Bayesian formula is applied to obtain the prediction conclusion about the advertising effect Y, that is, the conditional probability distribution P(y|m,t) of the advertising effect Y under the given advertising delivery strategy t and the weekend external factor m during advertising delivery.

2. The artificial intelligence prediction method based on reasoning enhancement according to claim 1, characterized in that: The definition of the variables of interest in the prediction system is specifically to clarify the role and observable conditions of each relevant variable in the prediction system, namely: The effect variable Y is the main target of the prediction result of the advertising effect prediction system, that is, the advertising effect; The treatment variable T is the variable that primarily affects the effect variable Y, i.e., the advertising delivery strategy; The confounding variable Z is a variable that affects both the treatment variable T and the effect variable Y, i.e., the user's consumption behavior habits; the observable proxy variable X is the observable variable X affected by the confounding variable Z, i.e., the user's consumption browsing behavior record; The adjustment variable M is the external factor that affects the effect variable, that is, the external factor on weekends when advertising is released.

3. The artificial intelligence prediction method based on reasoning enhancement according to claim 1, characterized in that: According to the obtained data set, the missing items in the data set are imputed using fully conditional canonical multiple imputation (FCSMI) to obtain a complete data set without missing items, specifically: 1) Perform mean interpolation on the missing data features in the obtained dataset of relevant variables to obtain an initial complete dataset; 2) Further adjusting the data features in the initial complete data set, using other data features as input and the missing data features in the obtained data set of relevant variables as output, and using the initial complete data set as the training data set to establish and train a linear regression model or a logistic regression model; 3) Use the linear regression model or logistic regression model obtained in the previous step to interpolate the missing data features in the obtained data set of relevant variables. If it does not converge, return to step 2) and loop until convergence.

4. The artificial intelligence prediction method based on reasoning enhancement according to claim 1, characterized in that: Based on the causal relationship between the variables, assuming that the user's purchasing habits Z follow a normal distribution under the given conditions of the advertising strategy T, the user's consumption and browsing behavior record X, and the weekend external factor M during the advertising delivery, a fully connected neural network model is constructed with the value of the treatment variable t, the value of the proxy variable x, and the value of the adjustment variable m as inputs, and the expectation and variance of the normal distribution followed by Z as outputs; Wherein, it is assumed that the confounding variable Z obeys the normal distribution under the given conditions of the processing variable T, the observable proxy variable X, and the adjustment variable M, and the expectation and variance of its distribution are functions of the observable proxy variable X, the adjustment variable M, and the processing variable T. The input of the neural network is the value of the processing variable t, the value of the proxy variable x, and the value of the adjustment variable m. The feature representation learning of the input variables is performed through several hidden layers, and the parameters of the distribution of the confounding variable value z are finally output; q(z|x,t,y)=N(μ,σ 2 ); Including the expectation μ and variance σ of the normal distribution 2 : m,s 2 =g(x,t,y) Among them, g is a fully connected neural network model.

5. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: Based on the causal relationship between the variables, assuming that the advertising effect Y follows a normal distribution under the given conditions of the advertising delivery strategy T, the user's purchasing habits Z, and the weekend external factor M during the advertising delivery, a fully connected neural network model is established with the value of the treatment variable t, the value of the confounding variable z, and the value of the adjustment variable m as inputs and the expectation and variance of the normal distribution followed by Y as outputs; Wherein, it is assumed that the effect variable Y obeys the normal distribution under the given conditions of the confounding variable Z, the treatment variable T, and the adjustment variable M, and the expectation and variance of its distribution are functions of the confounding variable Z, the adjustment variable M, and the treatment variable T. The input of the neural network is the value of the confounding variable z, the value of the adjustment variable m, and the value of the treatment variable t. The feature representation learning of the input variables is performed through several hidden layers, and the parameters of the distribution of the value y of the effect variable are finally output; p(y|z,m,t)=N(μ,σ 2 ) Including the expectation μ and variance σ of the normal distribution 2 : m,s 2 =f(z,m,t) Among them, f is a fully connected neural network model with a nonlinear activation layer.

6. The artificial intelligence prediction method based on reasoning enhancement according to claim 1, characterized in that: The two fully connected neural network models established above use a complete data set without missing items as a training data set, and use the variational lower bound ELBO as the objective function. The parameters of the neural network model are obtained through training. The variational lower bound ELBO is the objective function specifically:

7. The artificial intelligence prediction method based on reasoning enhancement according to claim 1 is characterized in that: Based on the advertising effect Y and any given user's purchasing habits z, the weekend external factor m during advertising delivery, the conditional probability distribution p(y|z,m,t) of the conditional probability distribution of the advertising delivery strategy t, and the prior probability distribution obeyed by the value of the confounding variable z, the Bayesian formula is applied to obtain the prediction conclusion about the effect variable Y, that is, the conditional probability distribution P(y|m,t) of the advertising effect Y when the treatment variable value t and the weekend external factor m during advertising delivery are given: Using the prior distribution p(z) of the confounding variable z and the conditional probability distribution p(y|z,m,t) predicted by the trained neural network model, the predicted distribution of the outcome variable y under the given adjustment variable m and treatment variable t is calculated using the Bayesian formula: p(y|m,t)=∫p(z)p(y|z,m,t)dz.

Images