Advertisement feedback optimization method and device based on machine learning

Abstract

The invention discloses an advertisement feedback optimization method and device based on machine learning, and relates to the technical field of digital advertisements. The method comprises the following steps of: firstly, constructing a causal graph containing user characteristics, advertisement exposure, click and conversion, eliminating selection deviation caused by confusion variables by utilizing a dual machine learning model, and accurately estimating a condition average processing effect of advertisement putting; secondly, performing multi-contact attribution in combination with a conditional average processing effect and a Shapley value to generate an initial putting strategy, and dynamically updating the strategy by using real-time data through an incremental causal forest algorithm; and finally, screening high conditional average processing effect material features, generating a new idea by using a generative adversarial network, and feeding back the new idea to a delivery engine. According to the method, the problems of correlation deviation, inaccurate attribution and delayed creative optimization in traditional advertisement putting are solved, and the advertisement putting accuracy, the real-time response capability and the return on investment are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of digital advertising, and in particular to an advertisement feedback optimization method and device based on machine learning. BACKGROUND

[0002] With the increasing complexity of the mobile internet ecosystem, programmatic advertising has become the mainstream mode of digital marketing. In the real-time bidding ecosystem, the advertisement delivery device needs to predict the conversion probability of the advertisement to the user within milliseconds based on user portraits, context environment and historical behavior data, so as to determine the bidding strategy and channel allocation.

[0003] The existing advertisement delivery technology mainly relies on supervised learning models with click rate or conversion rate prediction as the core. This kind of method usually assumes that there is a direct correlation between advertisement exposure and user conversion, and directly uses the correlation in the historical log for modeling. However, in the actual delivery scenario, there are a large number of "confounding variables", such as user's inherent preference, consumption level or instant context environment. These variables simultaneously affect whether the user will be exposed to the advertisement and whether the user will have a conversion behavior. The existing technology often ignores the selection bias brought by these confounding variables, making it difficult for the model to distinguish between "incremental conversion brought by the advertisement" and "natural conversion of the user". For example, some high-value users will purchase goods regardless of whether they see the advertisement, and if the system simply attributes this part of the conversion to the advertisement exposure, it will lead to waste of budget on unnecessary people, and it is difficult to evaluate the real causal effect of the advertisement. In addition, the existing multi-touch attribution model mostly uses statistical models based on rules (such as the last click) or static weights, lacking dynamic quantification of the causal contribution between different touches, which easily leads to neglect of early assist channels. In terms of model updating, the traditional scheme mostly adopts a T+1 day-level batch training mode, and in the face of sudden hot spots or rapid drift of data distribution, the model updating has significant lag, and cannot adjust the delivery strategy in real time. At the same time, the generation and delivery strategy of the advertisement are often separated, the generation of the advertisement mostly depends on the general generative adversarial network, lacks a directional optimization mechanism based on causal effect feedback, and the generated materials are difficult to accurately hit the user features with high conversion potential.

[0004] Therefore, there is an urgent need for an advertisement delivery method that can eliminate the influence of confounding variables, accurately quantify the causal effect of the advertisement, have real-time incremental learning ability, and can optimize the advertisement materials. SUMMARY

[0005] The present application provides an advertisement feedback optimization method and device based on machine learning, which solves the problems of correlation bias, inaccurate attribution and creative lag in the prior art.

[0006] In a first aspect, an advertisement feedback optimization method based on machine learning is provided, the method comprising:

[0007] Based on the full-volume historical interaction log of the target advertising system, a directed acyclic graph containing user feature vectors, advertising exposure states, user click behaviors, and conversion results is constructed, and a causal discovery algorithm is used to identify a set of potential confounding variables in the directed acyclic graph;

[0008] The historical interaction log is orthogonally processed by using a dual machine learning model, the selection bias caused by the set of confounding variables is eliminated by fitting a propensity score model and a result prediction model respectively, and the conditional average treatment effect of advertising delivery on target users is estimated;

[0009] The initial advertising delivery strategy containing bid coefficients and channel distribution ratios is generated by combining the conditional average treatment effect with the multi-touch attribution weight based on Shapley value, and the initial advertising delivery strategy is issued to the advertising delivery engine for execution;

[0010] During the advertising delivery process, incremental interaction data is collected in real time, and the feature splitting rule of the decision tree node is dynamically updated by using an incremental causal forest algorithm to correct the initial advertising delivery strategy to obtain a real-time delivery strategy;

[0011] The feature vector of the advertising material with high conditional average treatment effect is extracted, and the feature vector is input as a conditional constraint into a generative adversarial network to generate a new advertising material file and feed back to the advertising delivery engine.

[0012] Preferably, the step of using a dual machine learning model to orthogonally process the historical interaction log, eliminating the selection bias caused by the set of confounding variables by fitting a propensity score model and a result prediction model respectively, and estimating the conditional average treatment effect of advertising delivery on target users, includes:

[0013] Constructing a first machine learning regressor Based on the user feature vector Predicting the probability of advertising exposure state , and constructing a second machine learning regressor Based on the user feature vector Predicting the expected value of the conversion result ;

[0014] Calculate the residual of the advertising exposure state And the residual of the conversion result , to eliminate the direct influence of the user feature vector On And ;

[0015] Based on the residual And , a partial linear regression model is constructed, and a conditional average treatment effect function is estimated by minimizing the following loss function :

[0016]

[0017] wherein, denotes the number of samples, denotes the index of the th sample, denotes the causal effect value to be estimated for the user feature, denotes the conversion outcome residual of the th sample, denotes the exposure state residual of the th sample.

[0018] Preferably, the incremental causal forest algorithm is used to dynamically update the feature split rule of the decision tree node, including:

[0019] A preset time sliding window is set, and the incremental interaction data in the sliding window is periodically read and injected into each leaf node of the current causal forest;

[0020] For each leaf node, the information gain change rate before and after the incremental interaction data is introduced is calculated;

[0021] It is judged whether the information gain change rate is greater than a preset split threshold ;

[0022] If , the reconstruction operation of the current leaf node is triggered, and the optimal split feature and split threshold are recalculated using the incremental interaction data and the inventory data in the node to generate new child nodes;

[0023] If , the structure of the current leaf node is kept unchanged, and only the causal effect estimation value inside the node is updated;

[0024] The preset split threshold is set to 0.05.

[0025] Preferably, the method further comprises:

[0026] During the operation of the incremental causal forest algorithm, the mean of the prediction error in the last time windows is monitored in real time ;

[0027] The change rate of the prediction error is calculated ​​; wherein represents the mean of the prediction error of the current time window, represents the mean of the prediction error of the previous time window; if the rate of change exceeds a preset safety threshold, or a conceptual drift in data distribution is detected, the dynamic update of the incremental causal forest algorithm is suspended;

[0028] trigger a full retraining process, retrain the dual machine learning model and the causal forest model using all the accumulated historical data, and deploy the retrained model parameters as the new baseline model.

[0029] Preferably, the dual machine learning model and the incremental causal forest algorithm are deployed in a distributed architecture based on federated learning, and the method further comprises:

[0030] deploy a lightweight causal model locally on the user terminal device, and the size of the lightweight causal model is compressed to below a preset storage threshold;

[0031] train the lightweight causal model locally on the user terminal device using local interaction data, and calculate the gradient information of the model parameters ;

[0032] encrypt the gradient information using homomorphic encryption technology, and upload the encrypted gradient information to a cloud aggregation server;

[0033] aggregate the gradient information of each user terminal device on the cloud aggregation server through a federated averaging algorithm, update the global causal model parameters, and distribute the updated parameters to the user terminal device.

[0034] Preferably, the size of the lightweight causal model is less than 100KB, and the training process on the user terminal device only uses locally stored user behavior trajectory data, without cross-device transmission of raw data.

[0035] Preferably, the initial advertising strategy including bid coefficients and channel allocation ratios is generated by combining the conditional average treatment effect and the multi-touch attribution weight based on Shapley value, including:

[0036] regard each advertising touch point in the user conversion path as a participant in a cooperative game;

[0037] calculate the Shapley value of the first advertising touch point, and the calculation formula is:

[0038]

[0039] wherein, is a set of all advertisement touchpoints; is a subset of advertisement touchpoints (i.e., possible touchpoint combinations); and denote the number of elements in the set, respectively; is a subset of advertisement touchpoints ;

[0040] is the estimated conditional average treatment effect using a doubly robust machine learning model ; is the causal correction of the conversion contribution value , so that only reflects the incremental conversion caused by the causal advertisement touchpoint;

[0041] is the causal correction of the conversion contribution value ; is the budget allocation weight of the first advertisement touchpoint based on the corrected Shapley value .

[0042] Preferably, the feature vector of the advertisement material with a high conditional average treatment effect is extracted, and the feature vector is input as a conditional constraint into the generative adversarial network, including:

[0043] The high-quality material set with a high conditional average treatment effect is screened out from the historical advertisement material library; ;

[0044] The visual feature vector and semantic feature vector of the high-quality material set are extracted using a convolutional neural network; ;

[0045] The conditional generative adversarial network is constructed, and the visual feature vector and semantic feature vector are spliced as a conditional vector and input into a generator; ; ;

[0046] A new advertisement material image is generated based on random noise and the conditional vector using the generator; ; ;

[0047] The generated advertisement material image is discriminated for authenticity and condition consistency using a discriminator, and the generator and the discriminator reach Nash equilibrium until the generated advertisement material image is generated. ;

[0048] Preferably, a causal discovery algorithm is used to identify a set of potential confounding variables in a directed acyclic graph, including:

[0049] User features are defined as nodes​​​​ Ad exposure is defined as a node. The transformation result is defined as a node. ;

[0050] Conditional independence tests are performed on the full set of historical interaction data based on PC or FCI algorithms to construct an initial causal framework;

[0051] Identify nodes that are not directly observed but are simultaneously observed. and nodes Latent variables with causal paths exist And establish a path from the directed acyclic graph. point to and from point to The edge;

[0052] Using instrumental variables or proxy variables to handle latent variables Explicit modeling is performed, and the model is then incorporated into a set of confounding variables for subsequent dual machine learning correction.

[0053] Secondly, embodiments of the present invention provide an advertising feedback optimization device based on machine learning, the device comprising:

[0054] The causal graph construction module is used to construct a directed acyclic graph (DAG) containing user feature vectors, ad exposure status, user click behavior, and conversion results based on the full historical interaction logs of the target advertising system, and to identify the set of potential confounding variables in the DAG using a causal discovery algorithm.

[0055] The dual machine learning inference module is used to orthogonalize historical interaction logs using a dual machine learning model. By fitting a propensity score model and an outcome prediction model respectively, it eliminates selection bias caused by the set of confounding variables, thereby estimating the conditional average treatment effect of advertising on target users.

[0056] The strategy generation and execution module combines the conditional average processing effect with the multi-touchpoint attribution weight based on Shapley values ​​to generate an initial ad delivery strategy that includes bid coefficients and channel allocation ratios, and then sends the initial ad delivery strategy to the ad delivery engine for execution.

[0057] The incremental feedback update module is used to collect incremental interaction data in real time during the ad delivery process, and use the incremental causal forest algorithm to dynamically update the feature splitting rules of the decision tree nodes in order to correct the initial ad delivery strategy and obtain the real-time delivery strategy.

[0058] The creative generation linkage module is used for extracting a feature vector of an advertisement material with a high conditional average treatment effect, inputting the feature vector as a conditional constraint into a generative adversarial network, generating a new advertisement material file, and feeding back to an advertisement delivery engine.

[0059] Advantages:

[0060] By constructing a causal diagram and utilizing a double machine learning model to orthogonally process the historical interaction log, selection bias caused by potential confounding variables is effectively eliminated, and the conditional average treatment effect of advertisement delivery on target users can be accurately estimated. This enables the system to distinguish between real incremental conversions caused by advertisements and natural conversions of users, avoid unnecessary invalid delivery to users, and significantly improve the utilization efficiency of advertising funds. By combining the conditional average treatment effect with multi-touch attribution based on Shapley values, the attribution weight is corrected by the causal effect, solving the problem of inaccurate attribution of traditional models, and realizing the global optimal allocation of multi-channel budgets. Through the incremental causal forest algorithm, incremental data is processed in real time, and the model can quickly capture user interest drift or market changes, dynamically correct the delivery strategy, and significantly reduce the strategy response delay. Finally, the high causal effect feature is input as a constraint into the generative adversarial network, which connects the closed loop of effect evaluation and creative generation, realizes the directional optimization and automatic iteration of advertisement materials, and improves the click-through rate and conversion effect of advertisements. BRIEF DESCRIPTION OF DRAWINGS

[0061] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for purposes of illustration only and are not intended to limit the present application thereto. Moreover, the use of the same reference symbols in different drawings indicates similar or identical items. In the drawings:

[0062] Figure 1 An electronic device structure schematic diagram of a hardware running environment related to the embodiments of the present application;

[0063] Figure 2 A step flowchart of an advertisement feedback optimization method based on machine learning provided by the embodiments of the present application;

[0064] Figure 3 A functional module schematic diagram of an advertisement feedback optimization device based on machine learning provided by the embodiments of the present application. DETAILED DESCRIPTION

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the present application will be briefly introduced below in combination with the drawings and embodiments or prior art descriptions. Obviously, the following description of the drawings is only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor. It should be noted that the description of these embodiments is used to help understand the present application, but does not constitute a limitation on the present application. The technical solutions of the present application will be further described below in combination with the drawings.

[0066] Referring to Figure 1 , Figure 1 The electronic device structure schematic diagram of the hardware running environment involved in the embodiments of the present application.

[0067] As Figure 1 shown, the electronic device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between the components. The user interface 1003 can include a display, an input unit such as a keyboard, and an optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (WI-FI interface). The memory 1005 can be a high-speed random access memory (RAM) memory, or a stable non-volatile memory (NVM), such as a magnetic disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0068] Those skilled in the art can understand Figure 1 that the structure shown in the foregoing description does not constitute a limitation on the electronic device, and can include more or fewer components than the drawings, or combine certain components, or different component arrangements.

[0069] As Figure 1As shown, the memory 1005 as a storage medium can include an operation device, a data storage module, a network communication module, a user interface module and an electronic program. In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the application can be arranged in the electronic device, and the electronic device calls the machine learning-based advertisement feedback optimization device stored in the memory 1005 through the processor 1001, and executes the machine learning-based advertisement feedback optimization method provided by the embodiment of the application.

[0070] Reference Figure 2 The embodiment of the application provides a machine learning-based advertisement feedback optimization method, which is applied to an advertisement delivery device (such as a DSP demand side platform) or an associated data processing server. The method solves the effect evaluation deviation, inaccurate attribution and strategy updating lag caused by confounding variables in traditional advertisement delivery by introducing a causal inference framework and an incremental learning mechanism. The method flow described in the embodiment can specifically include the following steps S101 to S105:

[0071] S101, based on the full-amount historical interaction log of the target advertisement system, a directed acyclic graph containing a user feature vector, an advertisement exposure state, a user click behavior and a conversion result is constructed, and a potential confounding variable set in the directed acyclic graph is identified by using a causal discovery algorithm.

[0072] In the embodiment, S101 aims to establish a causal association model in the advertisement delivery device, and to separate the real causal structure from the statistical correlation. Specifically, the following sub-steps are included:

[0073] S1011, data preprocessing and node definition, extracting the full-amount historical interaction log from the data warehouse of the target advertisement system. The log data is usually stored in TB level or even PB level, and contains user behavior records in the past 6 months to 2 years. First, data cleaning is performed to remove outliers (such as crawler traffic and cheating clicks). The node variables of the causal graph are defined:

[0074] The user feature is defined as a node , is a high-dimensional vector containing static attributes and dynamic attributes. The static attributes include age, gender, region, device model, etc.; the dynamic attributes include the click rate in the past 7 days, the historical purchase average price, the APP active time distribution, etc. In order to deal with high-dimensional sparsity, the Embedding technology is used to map discrete features to low-dimensional dense vectors.

[0075] The advertisement exposure state is defined as a node , Usually a binary variable (0 or 1) or a continuous variable (exposure length / frequency). In this embodiment, not only whether exposure is considered, but also exposure intensity level is introduced.

[0076] The conversion result is defined as a node , The final business goal, such as purchase, activation or retention.

[0077] The user click behavior is defined as an intermediate node , As an intermediate path from exposure to conversion.

[0078] S1012, construct an initial causal skeleton based on PC algorithm or FCI algorithm, in order to discover causal structure from observed data, this embodiment adopts constraint interaction causal discovery algorithm.

[0079] Conditional independence test, use PC algorithm (Peter-Clark Algorithm) or FCI algorithm (Fast Causal Inference Algorithm) to test the conditional independence of the full amount of historical interaction data. For any two node variables, given a condition set , calculate the conditional mutual information or partial correlation coefficient. If independent at a certain significance level (such as ), delete the edge between them in the graph.

[0080] Construct an initial causal skeleton, remove irrelevant edges by traversing all node pairs, and obtain an undirected graph skeleton.

[0081] Direction orientation, based on V structure (V-structure) identification and direction propagation rule, the undirected edge is oriented to directed edge, forming directed acyclic graph (Directed Acyclic Graph, DAG).

[0082] S1013, identify the set of hidden variables and confounding variables, in actual advertising system, there are a large number of factors not directly observed but affecting the system (such as the user's current mood, unrecorded off-site social media influence, competitor's promotion activities, etc.).

[0083] Hidden variable identification , in the constructed DAG, if it is found that there is a backdoor path between node (advertising exposure) and node (conversion result) which cannot be blocked by observed variables , or FCI algorithm detects that there is a bidirectional edge (U ), it is inferred that there is a hidden variable .

[0084] establish causal paths, identify hidden variables that are not directly observed but simultaneously exist causal paths between nodes and nodes hidden variables that exist causal paths between and and between and and between and and , where is the confounder.

[0085] explicit modeling, use instrumental variable (IV) or proxy variable method to explicitly model the hidden variables .

[0086] instrumental variable method, find a variable (such as random experimental traffic marking in an advertising bidding system, or randomly jittered bid floors), which only affects exposure but does not directly affect conversion (i.e. ), and is independent of the hidden variable . By , the influence path of on is blocked.

[0087] proxy variable method, use a set of observed variables as proxies for the hidden variable . For example, use "user's dwell time on the detail page" and "slide speed" as proxy variables for "user's purchase intention (hidden variable)".

[0088] All identified directly observed confounders (such as user historical preferences) and hidden variables modeled through proxy variables are collectively referred to as the set of potential confounders, denoted as .

[0089] S102, orthogonalize the historical interaction log using a double machine learning model, eliminate the selection bias caused by the set of confounders by fitting a propensity score model and a result prediction model respectively, and estimate the conditional average treatment effect of the advertisement delivery on the target user.

[0090] In traditional machine learning, directly train Often, there will be selection bias due to the presence of confounding variables. For example, high-spending users are more likely to be recommended expensive items (high exposure probability) and are also more likely to make a purchase (high conversion probability), leading the model to overestimate the effect of the ad. This step uses the Double Machine Learning (DML) framework to solve this problem.

[0091] S1021, build a first machine learning regressor , build a classification or regression model (such as XGBoost, LightGBM, or deep neural network) to predict the probability of ad exposure. This model is called the propensity score model.

[0092] Input: user feature vector (containing the set of confounding variables identified from S101). Output: predicted exposure probability . Goal: fit the relationship between user features and exposure behavior, capturing the system's placement strategy preferences.

[0093] S1022, build a second machine learning regressor , build another regression model to predict the expected outcome of conversion without ad intervention (or average intervention). This model is called the outcome prediction model.

[0094] Input: user feature vector . Output: predicted conversion expectation . Goal: fit the natural relationship between user features and conversion behavior, capturing the user's natural conversion tendency.

[0095] S1023, calculate the residual and orthogonalize, calculate the residual of the ad exposure state . Here, represents the "exposure variation after removing the influence of user features", i.e. the part of exposure caused purely by random factors or experimental intervention. Calculate the residual of the conversion outcome . Here, represents the "conversion variation after removing the influence of user features", i.e. the conversion fluctuation that cannot be explained by user inherent attributes. Through this step, the direct influence of the user feature vector on and is removed, eliminating the collinearity interference caused by confounding variables and achieving orthogonalization.

[0096] S1024, estimate the Conditional Average Treatment Effect (CATE) based on the residual and , build a partial linear regression model. Assume the causal effect model is: . Estimate the Conditional Average Treatment Effect function by minimizing the following loss function :

[0097]

[0098] where, denotes the number of samples; denotes the index of the th sample, which iterates from 1 to ; is the conversion residual of the th sample; is the exposure residual of the th sample; is the vector of all features of the th user; denotes the causal effect value for the user feature to be estimated, i.e., CATE. To fit , one can use flexible non-parametric models such as causal forests or deep neural networks. This embodiment preferably uses a deep neural network to fit , with the input being the user features and the output being the causal effect value. This value quantifies the incremental conversion probability of "exposing the ad to this user" compared to "not exposing the ad".

[0099] S103, combine the Conditional Average Treatment Effect with the Shapley value-based multi-touch attribution weight to generate an initial ad placement strategy containing bid coefficients and channel allocation ratios, and issue the initial ad placement strategy to the ad placement engine for execution.

[0100] In a multi-channel marketing environment, the conversion of a user is often the result of multiple ad exposures. The traditional "last-click attribution" ignores the contribution of the assist channels. This step combines the causal effect and game theory ideas for scientific attribution.

[0101] S1031, build a cooperative game attribution model, regarding each ad touch (such as display ads, social ads, and search ads) in the user conversion path as a participant in the cooperative game. The set of all touches is denoted as .

[0102] S1032, calculate the Shapley value:

[0103] Calculate the Shapley value of the first ad touch point , the formula is:

[0104]

[0105] Where, is the set of all ad touch points; is the subset (i.e., possible touch point combination) that does not contain the ad touch point ; and respectively represent the number of elements in the set; is the conversion contribution value corresponding to the subset .

[0106] S1033, causal correction, in traditional Shapley value calculation, the conversion contribution value usually directly uses the historical conversion rate. However, the historical conversion rate contains natural conversion and cannot represent the real contribution of the advertisement. The embodiment uses the conditional average treatment effect estimated by the double machine learning model to perform causal correction on the conversion contribution value . Specifically, for the touch point combination , represents the set of all users who have experienced and only experienced the ad touch point combination ; represents the th user in ; its contribution value is defined as the sum or average of CATE of all users under the combination, rather than simply the number of conversions. That is: . In this way, only reflects the incremental conversion caused by the causal nature of the ad touch point, eliminates the part that the user will naturally convert even if he does not watch the advertisement, and avoids budget waste. Where, represents the set of users in the historical log who have only contacted the ad touch point combination ; represents the th user in the set; represents the conditional average treatment effect predicted by the model for the user.

[0107] S1034, generate an initial delivery strategy, allocate budget weights, determine the budget allocation weight of the ad touch point (or channel) based on the corrected Shapley value The more a channel contributes, the more budget it gets.

[0108] Bid coefficient, for a specific user Its base bid Adjusted by its predicted CATE value. Where is the adjustment factor. For high (i.e. the ad has a large incremental impact on it), increase the bid; for close to 0 or even negative (the ad can cause aversion), decrease the bid or not serve.

[0109] The initial ad serving strategy, containing the budget allocation for each channel and the bid coefficient matrix for different audience groups, is packaged and sent to the DSP ad serving engine for execution.

[0110] S104, in the ad serving process, real-time collection of incremental interaction data, and using incremental causal forest algorithm to dynamically update the feature split rule of decision tree node, to correct the initial ad serving strategy to get real-time serving strategy.

[0111] The advertising environment is highly dynamic, user interest and market competition change at any time. Offline trained models (such as DML in S102) may be invalid after a few hours. Therefore, online incremental learning mechanism needs to be introduced.

[0112] S1041, incremental data injection and sliding window, set a preset time sliding window, for example, set the window length hours, step minutes.

[0113] Periodically (every 5 minutes) read the incremental interaction data (IncrementalInteraction Data) in the sliding window, including the latest user features, exposure, click and conversion data. Inject the incremental interaction data into each leaf node of the current causal forest.

[0114] S1042, calculate the information gain rate of change, the causal forest is composed of multiple causal trees. For each leaf node of each tree, the node originally stores a part of historical samples for estimating the CATE of the region.

[0115] When new data enters the leaf node, for each leaf node, calculate the information gain rate of change before and after the introduction of the incremental interaction data.

[0116] The calculation of information gain is based on the principle of variance maximization of treatment effect. Assuming that the variance of treatment effect before node splitting is , the weighted variance sum of left and right child nodes after splitting is , then the gain . Calculate the rate of change: .

[0117] S1043, dynamic splitting and restructuring, determine whether the information gain rate of change is greater than the preset splitting threshold .

[0118] Trigger restructuring, if , it means that the new data brings a significant amount of information, and the current tree structure cannot well divide the characteristics of the crowd. Then trigger the reconstruction operation of the current leaf node. Use the incremental interaction data and the inventory data in the node (you can use the reservoir sampling to retain part of the historical data) to recalculate the optimal split feature (such as changing from "age" to "recently viewed category") and split threshold, generate new child nodes, and refine the crowd division.

[0119] Parameter update, if , keep the structure of the current leaf node unchanged, and only update the causal effect estimation value inside the node with new data . This is equivalent to fine-tuning the parameters without changing the model structure, with minimal computational cost. The preset splitting threshold is set to 0.05 (i.e. 5% gain change).

[0120] S1044, real-time monitoring and full-volume retraining trigger, to prevent error accumulation caused by incremental update, the method further comprises: during the operation of the incremental causal forest algorithm, real-time monitoring of the average prediction error in the last 7 time windows (for example, using the mean square error MSE of the last 5 minutes of data).

[0121] Calculate the rate of change of prediction error :

[0122] ; wherein, represents the average prediction error of the current time window, represents the average prediction error of the last time window;

[0123] Abnormality detection, if the rate of change exceeds the preset safety threshold (for example, 15%), or the data distribution is detected to have a significant concept drift through statistical testing (such as KS test), the dynamic update of the incremental causal forest algorithm is suspended. Full-volume retraining: trigger the full-volume retraining process, retrain the dual machine learning model and the causal forest model using all the historical data currently accumulated, and deploy the retrained model parameters as the new baseline model.

[0124] S1045, Federated Learning Distributed Deployment (Preferred Implementation): Considering data privacy and edge computing capabilities, the dual machine learning model and the incremental causal forest algorithm can be deployed in a federated learning-based distributed architecture.

[0125] Local Deployment: Deploy a lightweight causal model locally on multiple user terminal devices (such as mobile phone APPs). To adapt to the resources on the terminal side, the size of the lightweight causal model is compressed to below the preset storage threshold (preferably less than 100KB). The model structure can be a pruned shallow neural network or a decision tree. Local Training: On the user terminal device, only the lightweight causal model is trained using locally stored user behavior trajectory data to calculate the gradient information of the model parameters . This process does not transfer raw data across devices, protecting user privacy. Encryption Upload: Encrypt the gradient information using homomorphic encryption technology or differential privacy technology, and upload the encrypted gradient information to the cloud aggregation server. Global Aggregation: The cloud aggregation server aggregates the gradient information from each user terminal device using a federated averaging algorithm, updates the global causal model parameters, and distributes the updated parameters to the user terminal devices.

[0126] S105, Extract the feature vector of the ad material with high conditional average treatment effect, input the feature vector as a conditional constraint to the generative adversarial network, generate a new ad material file and feed it back to the ad delivery engine.

[0127] Traditional ad creative generation is often disconnected from delivery effectiveness. Although the generated materials are beautiful, they may not necessarily lead to conversion. This step guides creative generation through causal effects, achieving "effect-driven creativity".

[0128] S1051, Screening High-quality Material Set, From the historical ad material library, according to the CATE value calculated in S102, screen out the high-quality material set with the highest conditional average treatment effect ranking (such as the top 10%).

[0129] S1052, Feature Extraction, Use a pre-trained convolutional neural network (CNN, such as ResNet or VGG) to extract visual feature vectors (Visual Features, such as color, texture, composition) and semantic feature vectors (Semantic Features, text keyword vectors extracted by OCR and NLP) of the high-quality material set.

[0130] S1053, Build Conditional Generative Adversarial Network, Build a conditional generative adversarial network.

[0131] Condition vector construction: after splicing the visual feature vector and the semantic feature vector, the user feature vector of the target crowd can also be combined to form a combined condition vector This represents the "most effective visual and semantic pattern for this type of crowd". Generator : input is random noise and condition vector . The generator is committed to generating new advertising material images that meet the conditions characteristics . Discriminator : input is real effective material and generated material , and conditions . The discriminator not only judges whether the image is real, but also judges whether the image meets the conditions (that is, whether it has high CATE characteristics).

[0132] S1054, generate and feedback, use the generator and discriminator to conduct adversarial training until Nash equilibrium is reached.

[0133] After training is completed, use the generator to batch generate new advertising material files.

[0134] After automatic review of these new materials, feedback to the advertising delivery engine as candidate materials for the next round of A / B testing, forming a "delivery-evaluation-generation-re-delivery" closed loop.

[0135] The embodiment of the application also provides an advertisement feedback optimization device based on machine learning. Referring to Figure 3 , a functional module diagram of an advertisement feedback optimization device 300 based on machine learning is shown, which can include the following modules:

[0136] A causal diagram construction module 301 is used to construct a directed acyclic graph containing user feature vectors, advertising exposure states, user click behaviors and conversion results based on the full-quantity historical interaction log of the target advertising system, and a causal discovery algorithm is used to identify a set of potential confounding variables in the directed acyclic graph;

[0137] A dual machine learning inference module 302 is used to orthogonalize the historical interaction log by using a dual machine learning model, eliminate selection bias caused by the set of confounding variables by fitting a propensity score model and a result prediction model respectively, and estimate the conditional average treatment effect of the advertising delivery on the target user;

[0138] The strategy generation and execution module 303 is used to combine the conditional average processing effect and the multi-touchpoint attribution weight based on Shapley value to generate an initial advertising delivery strategy that includes bid coefficients and channel allocation ratios, and then send the initial advertising delivery strategy to the advertising delivery engine for execution.

[0139] The incremental feedback update module 304 is used to collect incremental interaction data in real time during the advertising process and use the incremental causal forest algorithm to dynamically update the feature splitting rules of the decision tree nodes in order to correct the initial advertising strategy and obtain the real-time advertising strategy.

[0140] The creative generation linkage module 305 is used to extract the feature vector of advertising materials with high conditional average processing effect, and input the feature vector as a conditional constraint into the generative adversarial network to generate new advertising material files and feed them back to the advertising delivery engine.

[0141] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.

[0142] Memory, used to store computer programs;

[0143] When a processor executes a program stored in memory, it implements the machine learning-based advertising feedback optimization method of the present invention.

[0144] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM), or non-volatile memory, such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0145] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0146] Furthermore, to achieve the above objectives, embodiments of the present invention also propose a computer-readable storage medium storing a computer program that, when executed by a processor, implements the machine learning-based advertising feedback optimization method of the embodiments of the present invention.

[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable hardware devices (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0150] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0152] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A machine learning-based advertising feedback optimization method, characterized in that, The method includes: Based on the full historical interaction logs of the target advertising system, a directed acyclic graph (DAG) containing user feature vectors, ad exposure status, user click behavior, and conversion results is constructed, and a causal discovery algorithm is used to identify the set of potential confounding variables in the DAG. The historical interaction logs are orthogonalized using a dual machine learning model. By fitting a propensity score model and an outcome prediction model respectively, the selection bias caused by the set of confounding variables is eliminated, thereby estimating the conditional average treatment effect of advertising on target users. Combining the conditional averaging effect with the multi-touchpoint attribution weight based on Shapley values, an initial ad delivery strategy including bid coefficients and channel allocation ratios is generated, and the initial ad delivery strategy is sent to the ad delivery engine for execution. During the ad delivery process, incremental interaction data is collected in real time, and the feature splitting rules of the decision tree nodes are dynamically updated using an incremental causal forest algorithm to correct the initial ad delivery strategy and obtain a real-time delivery strategy. Feature vectors of advertising creatives with high conditional average processing effects are extracted, and these feature vectors are used as conditional constraints to input into a generative adversarial network to generate new advertising creative files, which are then fed back to the advertising delivery engine.

2. The advertising feedback optimization method based on machine learning according to claim 1, characterized in that, The step of orthogonalizing the historical interaction logs using a dual machine learning model, and eliminating selection bias caused by the set of confounding variables by fitting a propensity score model and an outcome prediction model respectively, thereby estimating the conditional average treatment effect of advertising on target users, includes: Building the first machine learning regressor Based on user feature vectors Predicting ad exposure status The probability of, and the construction of a second machine learning regressor. Based on user feature vectors Predicting conversion results Expected value; Calculate the residual of ad exposure status and the residuals of the transformation results To remove user feature vectors right and The direct impact; Based on the residual and Construct a partial linear regression model and minimize the following loss function. To estimate the conditional average treatment effect function : in, Indicates the number of samples. Indicates the first Index of each sample, This indicates the user-specific characteristics to be estimated. The causal effect value, Indicates the first The residual of the transformation result of each sample, Indicates the first The exposure state residual of each sample.

3. The advertising feedback optimization method based on machine learning according to claim 1, characterized in that, The incremental causal forest algorithm is used to dynamically update the feature splitting rules of decision tree nodes, including: Set a preset time sliding window, periodically read the incremental interaction data in the sliding window, and inject the incremental interaction data into each leaf node of the current causal forest; For each leaf node, calculate the rate of change of information gain before and after introducing the incremental interactive data. ; Determine the rate of change of information gain Is it greater than the preset splitting threshold? ; like If this happens, a reconstruction operation is triggered on the current leaf node. The optimal splitting features and splitting threshold are recalculated using the incremental interaction data and the existing data within the node, and a new child node is generated. like If the structure of the current leaf node remains unchanged, only the causal effect estimate within the node is updated; Wherein, the preset splitting threshold Set to 0.

05.

4. The advertising feedback optimization method based on machine learning according to claim 3, characterized in that, The method further includes: During the operation of the incremental causal forest algorithm, continuous real-time monitoring is performed. Mean prediction error within a time window ; Calculate the rate of change of prediction error ;in, This represents the mean prediction error for the current time window. This represents the mean prediction error for the previous time window; If the rate of change If the preset safety threshold is exceeded, or if a concept drift in the data distribution is detected, the dynamic update of the incremental causal forest algorithm will be paused. Trigger a full retraining process, retrain the dual machine learning model and causal forest model using all currently accumulated historical data, and deploy the retrained model parameters as a new baseline model.

5. The advertising feedback optimization method based on machine learning according to claim 1, characterized in that, The dual machine learning model and the incremental causal forest algorithm are deployed in a federated learning-based distributed architecture, and the method further includes: A lightweight causal model is deployed locally on multiple user terminal devices, and the size of the lightweight causal model is compressed to below a preset storage threshold. The lightweight causal model is trained locally on the user terminal device using local interactive data, and the gradient information of the model parameters is calculated. ; Homomorphic encryption technology is used to process the gradient information. Encryption is performed, and the encrypted gradient information is uploaded to the cloud aggregation server; The cloud aggregation server aggregates the gradient information of each user terminal device through a federated averaging algorithm, updates the global causal model parameters, and sends the updated parameters to the user terminal devices.

6. The machine learning-based advertising feedback optimization method according to claim 5, characterized in that, The lightweight causal model is less than 100KB in size, and the local training process of the user terminal device only uses locally stored user behavior trajectory data and does not perform cross-device transmission of raw data.

7. The advertising feedback optimization method based on machine learning according to claim 1, characterized in that, The process of combining the conditional averaging effect with the multi-touchpoint attribution weights based on Shapley values ​​to generate an initial ad delivery strategy that includes bid coefficients and channel allocation ratios includes: Treat each advertising touchpoint in the user conversion path as a participant in a cooperative game; Calculate the first Shapley Value for Each Ad Touchpoint The calculation formula is: in, A collection of all advertising touchpoints; To exclude advertising touchpoints A subset of (i.e., possible combinations of touch points); and These represent the number of elements in the set, respectively. For subset The corresponding conversion contribution value; Conditional average treatment effect estimated using the dual machine learning model Contribution value to the transformation Perform causal correction so that It only reflects incremental conversions caused by the causal relationship of advertising touchpoints; Based on the corrected Shapley value Determine the first Budget allocation weights for each advertising touchpoint.

8. The advertising feedback optimization method based on machine learning according to claim 1, characterized in that, The step of extracting feature vectors from advertising creatives with high conditional averaging effects and then inputting these feature vectors as conditional constraints into a generative adversarial network includes: Conditional averaging effect was selected from the historical advertising material database. Ranked at the top A collection of high-quality materials; Visual feature vectors of the high-quality material set are extracted using a convolutional neural network. and semantic feature vectors ; Construct a conditional generative adversarial network to process the visual feature vectors. and semantic feature vectors The concatenated vector serves as a condition vector. Input to generator ; Using generators Based on random noise and condition vector Generate new advertising creative images ; Using a discriminator For the generated advertising material images Perform true / false and conditional consistency checks until the generator and discriminator reach Nash equilibrium.

9. The advertising feedback optimization method based on machine learning according to claim 1, characterized in that, The step of using a causal discovery algorithm to identify the set of potential confounding variables in the directed acyclic graph includes: Define user features as nodes Ad exposure is defined as a node. The transformation result is defined as a node. ; The conditional independence test is performed on the full set of historical interaction data based on the PC algorithm or FCI algorithm to construct an initial causal framework; Identify nodes that are not directly observed but are simultaneously observed. and nodes Latent variables with causal paths exist And establish a path from the directed acyclic graph. point to and from point to The edge; The latent variables are analyzed using instrumental variables or proxy variables. Explicit modeling is performed, and the model is incorporated into the set of confounding variables for subsequent dual machine learning correction.

10. An advertising feedback optimization device based on machine learning, characterized in that, The device includes: The causal graph construction module is used to construct a directed acyclic graph containing user feature vectors, ad exposure status, user click behavior and conversion results based on the full historical interaction logs of the target advertising system, and to identify the set of potential confounding variables in the directed acyclic graph using a causal discovery algorithm. The dual machine learning inference module is used to orthogonalize the historical interaction logs using a dual machine learning model. By fitting a propensity score model and an outcome prediction model respectively, the selection bias caused by the set of confounding variables is eliminated, thereby estimating the conditional average treatment effect of advertising on the target user. The strategy generation and execution module is used to combine the conditional average processing effect and the multi-touchpoint attribution weight based on Shapley value to generate an initial advertising delivery strategy that includes bid coefficients and channel allocation ratios, and to send the initial advertising delivery strategy to the advertising delivery engine for execution. The incremental feedback update module is used to collect incremental interaction data in real time during the advertising process and use the incremental causal forest algorithm to dynamically update the feature splitting rules of the decision tree nodes in order to correct the initial advertising strategy and obtain the real-time advertising strategy. The creative generation linkage module is used to extract feature vectors of advertising materials with high conditional average processing effect, input the feature vectors as conditional constraints into the generative adversarial network, generate new advertising material files, and feed them back to the advertising delivery engine.

Images