Marketing effect evaluation method and device, electronic equipment, medium and program product

Abstract

The invention provides a marketing effect evaluation method which can be applied to the technical field of big data and the technical field of artificial intelligence. The method comprises the following steps: acquiring target user data of a to-be-evaluated business scene; matching a target causal inference model from a plurality of pre-constructed causal inference models based on a to-be-evaluated business scene, the plurality of causal inference models being pre-constructed based on causal modeling data sets of a plurality of business scenes; inputting the target user data into the target causal inference model to obtain average processing effect information; and obtaining an evaluation result of the marketing effect based on a comparison result of the average processing effect information and a preset threshold condition.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of big data and the technical field of artificial intelligence, and more particularly to a marketing effect evaluation method, device, equipment, medium and program product. BACKGROUND

[0002] At present, the causal inference method is widely used to evaluate the real effect of various marketing activities of financial institutions. Common methods include randomized controlled trial (RCT) and propensity score matching (PSM). The former relies on strict sample randomization and intervention comparison, and the latter matches the evaluation by modeling customer characteristics to calculate the intervention probability. It has played a certain role in customer conversion, asset growth and other scenarios.

[0003] However, the existing method is easily affected by sample allocation and matching error when dealing with high-dimensional large-scale data, and has poor stability. Moreover, the existing method usually assumes that the treatment effect is homogeneous among customers, ignores individual response differences, lacks explanation for different customer groups, and is difficult to meet the demand for intelligent, real-time and refined marketing evaluation in financial scenarios. SUMMARY

[0004] In view of the above problems, the present disclosure provides a marketing effect evaluation method, device, equipment, medium and program product.

[0005] According to a first aspect of the present disclosure, a marketing effect evaluation method is provided, the method comprising: obtaining target user data of a to-be-evaluated business scenario; matching a target causal inference model from a plurality of pre-constructed causal inference models based on the to-be-evaluated business scenario, wherein the plurality of causal inference models are pre-constructed based on causal modeling data sets of a plurality of business scenarios; inputting the target user data into the target causal inference model to obtain average treatment effect information; and obtaining an evaluation result of the marketing effect based on a comparison result of the average treatment effect information and a preset threshold condition.

[0006] According to an embodiment of the present disclosure, the method further comprises: obtaining individual treatment effect information based on inputting the target user data into the target causal inference model; and in response to the evaluation result of the marketing effect being a positive result, generating a marketing strategy suggestion for the to-be-evaluated business scenario based on the individual treatment effect information.

[0007] According to an embodiment of the present disclosure, the method further comprises constructing a causal modeling data set, specifically comprising: cross-combining and dividing historical user data based on a marketing channel dimension and a marketing target dimension to obtain a plurality of mutually exclusive business scenarios; and for each business scenario, constructing a corresponding causal modeling data set, wherein the causal modeling data set comprises a result variable, an intervention variable and a covariate of the corresponding business scenario.

[0008] According to an embodiment of the present disclosure, at least one of the plurality of causal inference models is a set of causal decision trees constructed based on minimizing intervention effect error.

[0009] According to an embodiment of the present disclosure, in the process of constructing the causal inference model based on minimizing intervention effect error, the method further comprises: calculating an importance score of each feature in the causal inference model; aggregating the importance scores to obtain a feature importance ranking; and selecting features with top K rankings of the feature importance ranking to form a candidate key feature set, wherein K is determined according to at least a total amount of model input features or a business scenario target, and the candidate key features are used for pre-screening features in the process of constructing the set of causal decision trees.

[0010] According to an embodiment of the present disclosure, after constructing the causal inference model based on minimizing intervention effect error, the method further comprises: resampling the causal modeling dataset multiple times to generate multiple training subsets; retraining the causal inference model for each training subset and recording a frequency of occurrence of a feature being used to divide nodes in the causal inference model; and

[0011] constructing an important feature subset based on the frequency of occurrence, the important feature subset being used for model result interpretation.

[0012] According to an embodiment of the present disclosure, the method further comprises: selecting a business pilot object based on the important feature subset and individual treatment effect information; obtaining an actual conversion result returned by marketing intervention on the business pilot object; and performing deviation analysis on the actual conversion result and a predicted value of the causal inference model, and obtaining a stability test result based on a result of the deviation analysis.

[0013] According to an embodiment of the present disclosure, the response to the evaluation result of the marketing effect is a positive result, and the marketing strategy suggestion for the business scenario to be evaluated is generated based on the individual treatment effect information, specifically comprising: performing response sensitivity stratification on target users based on the heterogeneity distribution of the individual treatment effect to obtain a customer group list; and generating the marketing strategy suggestion for the business scenario to be evaluated based on the customer group list.

[0014] The second aspect of the present disclosure provides an evaluation device for marketing effect, the device comprising: a data acquisition module configured to acquire target user data of a business scenario to be evaluated; a model matching module configured to match a target causal inference model from a plurality of pre-constructed causal inference models based on the business scenario to be evaluated, wherein the plurality of causal inference models are pre-constructed based on causal modeling data sets of a plurality of business scenarios; a model processing module configured to input the target user data into the target causal inference model to obtain average treatment effect information; and an evaluation result acquisition module configured to obtain an evaluation result for the marketing effect based on a comparison result of the average treatment effect information and a preset threshold condition.

[0015] According to an embodiment of the present disclosure, the model matching module can be further configured to cross-combine and divide the historical user data based on a marketing channel dimension and a marketing target dimension to obtain a plurality of mutually exclusive business scenarios; and for each business scenario, construct a corresponding causal modeling data set, wherein the causal modeling data set comprises a result variable, an intervention variable and a covariate of the corresponding business scenario.

[0016] According to an embodiment of the present disclosure, the model matching module can be further configured to calculate an importance score of each feature in the causal inference model; aggregate the importance scores to obtain a feature importance ranking; and select features with top K rankings in the feature importance ranking to form a candidate key feature set, wherein K is determined at least according to a total amount of model input features and a target of the business scenario, and the candidate key features are used for pre-screening features in the process of constructing the causal decision tree set.

[0017] According to an embodiment of the present disclosure, the model matching module can be further configured to resample the causal modeling data set multiple times to generate a plurality of training subsets; retrain the causal inference model for each training subset and record the frequency of occurrence of features in the causal inference model for dividing nodes; and construct an important feature subset based on the frequency of occurrence, wherein the important feature subset is used for model result interpretation.

[0018] According to an embodiment of the present disclosure, the model matching module can be further configured to select a business pilot object based on the important feature subset and individual treatment effect information; acquire an actual conversion result returned by marketing intervention on the business pilot object; and perform deviation analysis on the actual conversion result and a predicted value of the causal inference model to obtain a stability test result based on a result of the deviation analysis.

[0019] According to an embodiment of the present disclosure, the marketing effect evaluation device can further be configured to: input the target user data into the target causal inference model to obtain individual treatment effect information; and in response to the evaluation result being a positive result, generate a marketing strategy suggestion for the business scenario to be evaluated based on the individual treatment effect information.

[0020] According to an embodiment of the present disclosure, the marketing effect evaluation device can further be configured to: based on the heterogeneity distribution of the individual treatment effect, perform response sensitivity stratification on the target users to obtain a customer list; and generate a marketing strategy suggestion for the business scenario to be evaluated based on the customer list.

[0021] A third aspect of the present disclosure provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0022] A fourth aspect of the present disclosure further provides a computer-readable storage medium having stored thereon a computer program or instructions, which, when executed by a processor, implement the steps of the above method.

[0023] A fifth aspect of the present disclosure further provides a computer program product comprising a computer program or instructions, which, when executed by a processor, implement the steps of the above method.

[0024] According to an embodiment of the present disclosure, through the matching mechanism of business scenarios and causal models, the causal forest model pre-trained on a multi-scene high-dimensional data set can be used to directly estimate the treatment effect of target user data and output the marketing effect after threshold determination, thereby reducing the high error risk of large sample random allocation and propensity matching, improving the calculation efficiency; at the same time, it can significantly reduce the experimental and preprocessing costs and improve the real-time performance, without the need to perform time-consuming and expensive RCT or complex PSM feature engineering, to give a scenario-based result in real time, meet the demand for fast feedback of financial marketing, and improve the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0025] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure, taken in conjunction with the accompanying drawings, in which:

[0026] Figure 1 An application scenario diagram of a marketing effect evaluation method, device, equipment, medium and program product according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 2 A flowchart of a marketing effect evaluation method according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 3 A flowchart illustrating a method for primary screening of training features according to an embodiment of the present disclosure is shown schematically.

[0029] Figure 4 A flowchart illustrating a method for advanced verification of training features according to an embodiment of the present disclosure is shown schematically.

[0030] Figure 5 A schematic diagram illustrating a structural block diagram of a marketing effectiveness evaluation device according to an embodiment of the present disclosure; and

[0031] Figure 6 A block diagram of an electronic device suitable for implementing a marketing effectiveness evaluation method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0032] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0033] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0034] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0035] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0036] First, the technical terms used in this article are explained and clarified as follows.

[0037] Individual Treatment Effect (ITE): This refers to the estimated difference in outcome variables between individuals (users) who receive and do not receive intervention. ITE is the foundation for differentiated marketing strategies and can be used to identify highly sensitive users and personalized reach groups.

[0038] Average Treatment Effect (ATE): This refers to the difference in expected value of outcome variables between those who receive intervention (treatment) and those who do not, within a specific business scenario or group. ATE reflects the average effect of a marketing campaign at the overall level and is a core indicator for evaluating the effectiveness of an activity.

[0039] Heterogeneous Treatment Effect (HTE): This refers to how treatment effects vary across different subgroups or individuals, reflecting "who is more affected and who experiences limited effects." HTE is typically obtained from ITE (Information Technology) summaries and used to develop feature-driven customer strategies and rule recommendations.

[0040] Causal Forest is a nonparametric modeling method that combines causal inference with random forests. It constructs multiple causal trees, using the maximization of the difference in treatment effects among different subgroups as the partitioning criterion, to simultaneously estimate the overall average treatment effect and individual treatment effects. This model inherently possesses feature selection and nonlinear modeling capabilities, making it suitable for high-dimensional and heterogeneous scenarios.

[0041] The Gini Importance Score measures the contribution of a feature to improving the difference in effects during the splitting of a causal tree; a higher score indicates that the feature is more critical. A global ranking of feature importance can be generated by statistically analyzing the frequency of use and gain contribution of each feature across multiple causal trees.

[0042] Bootstrap resampling is a nonparametric statistical method that assesses the stability of model outputs or variable importance by constructing multiple subsets from a sample set through repeated sampling with replacement.

[0043] Against the backdrop of rapid development in fintech, financial institutions such as banks, insurance companies, and securities firms are increasingly relying on data intelligence to improve operational efficiency and customer service levels, with intelligent evaluation of marketing strategies becoming a crucial component. To measure the true impact of marketing activities on key indicators such as customer conversion, transaction activity, and asset growth, causal inference methods are widely used in evaluating the effectiveness of financial marketing, assisting institutions in making scientific decisions, accurately reaching customers, and optimizing resource allocation. Currently, commonly used causal inference methods mainly include randomized controlled trials (RCTs) and propensity score matching (PSM).

[0044] Randomized controlled trials, also known as A / B tests, are currently the mainstream method for evaluating the effectiveness of marketing interventions. This method effectively controls for potential confounding variables by randomly assigning samples to experimental and control groups, ensuring that differences in results are primarily caused by the marketing treatment. The experimental process typically includes sample selection, random assignment, intervention implementation, and effect evaluation. Finally, t-tests, regression analysis, and other methods are used to determine the causal effect of the marketing activity.

[0045] Propensity score matching focuses on estimating the treatment effect by modeling the probability of individuals accepting a treatment (i.e., propensity score) under non-randomized experimental conditions. Based on historical observation variables, it matches individuals in the experimental and control groups one-to-one, comparing the differences in results before and after the treatment. This method is also highly applicable in practical business, especially in scenarios where it is difficult to implement randomized controlled trials (RCTs).

[0046] While the above methods have addressed the causal inference problem in marketing effectiveness evaluation to some extent, they still have many shortcomings in handling large-scale complex data, improving evaluation efficiency, and enhancing personalized interpretation capabilities.

[0047] First, existing methods suffer from significantly increased sample allocation and matching errors when dealing with high-dimensional heterogeneous data. Traditional causal inference methods lack robustness and scalability in handling high-dimensional features, limiting their effectiveness in large-scale data analysis. Second, RCT methods, in particular, require strict adherence to random assignment principles in practice, involving complex experimental design and intervention control. This results in long data collection cycles, high implementation costs, and difficulty in flexibly responding to rapidly changing marketing strategy adjustments. Furthermore, to achieve modeling accuracy, traditional causal inference models often require cumbersome feature engineering and data preprocessing steps, leading to high computational complexity and low deployment efficiency. More importantly, existing causal inference methods generally assume homogeneity of treatment effects among individuals, ignoring the differentiated responses of users to marketing interventions in real-world scenarios. This makes it difficult to provide targeted marketing insights, resulting in limited model interpretability and guidance value.

[0048] Based on this, embodiments of this disclosure provide a method for evaluating marketing effectiveness, including: acquiring target user data of a business scenario to be evaluated; matching a target causal inference model from multiple pre-built causal inference models based on the business scenario to be evaluated, wherein the multiple causal inference models are pre-built based on causal modeling datasets of multiple business scenarios; inputting the target user data into the target causal inference model to obtain average treatment effect information; and obtaining an evaluation result of marketing effectiveness based on the comparison result of the average treatment effect information and preset threshold conditions. Through the matching mechanism between business scenarios and causal models, models such as causal forests, pre-trained on high-dimensional datasets of multiple scenarios, can be used to directly estimate the treatment effect of target user data and output the marketing effect after threshold determination, thereby reducing the high error risk of large-sample random assignment and propensity matching, and improving computational efficiency; at the same time, it can significantly reduce experimental and preprocessing costs and improve real-time performance, eliminating the need for time-consuming and expensive RCTs or complex PSM feature engineering, and providing scenario-based results instantly, meeting the needs of financial marketing for rapid feedback and improving user experience.

[0049] It should be noted that the marketing effectiveness evaluation methods, apparatus, devices, media, and program products defined in this disclosure can be used in the fields of big data technology and artificial intelligence technology, and can also be used in a variety of fields other than big data technology and artificial intelligence technology. The application fields of the marketing effectiveness evaluation methods, apparatus, devices, media, and program products provided in the embodiments of this disclosure are not limited.

[0050] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0051] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this disclosure all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0052] Figure 1 The illustration schematically depicts application scenarios of marketing effectiveness evaluation methods, apparatus, devices, media, and program products according to embodiments of this disclosure.

[0053] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0054] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0055] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0056] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0057] It should be noted that the marketing effectiveness evaluation method provided in this embodiment can generally be executed by server 105. Correspondingly, the marketing effectiveness evaluation device provided in this embodiment can generally be located in server 105. The marketing effectiveness evaluation method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the marketing effectiveness evaluation device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0058] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0059] The following will be based on Figure 1 The described scene, through Figures 2 to 4 The method for evaluating the marketing effectiveness of the disclosed embodiments is described in detail.

[0060] Figure 2 A flowchart illustrating a method for evaluating marketing effectiveness according to an embodiment of this disclosure is shown schematically.

[0061] like Figure 2 As shown, the marketing effectiveness evaluation method of this embodiment includes operations S210 to S240, and the marketing effectiveness evaluation method can be executed by server 105.

[0062] In operation S210, target user data for the business scenario to be evaluated is obtained.

[0063] In the embodiments of this disclosure, a business scenario can refer to the intervention or interaction that a financial institution conducts with a target user through a specific channel in order to achieve a specific business objective. Therefore, a business scenario can be distinguished by clearly defining the "channel" and the "objective".

[0064] According to embodiments of this disclosure, a channel refers to the medium through which financial institutions communicate or transmit information to users. For example, digital channels include SMS, in-app push notifications, email, social media messages, instant messaging tools, website pop-up / banner ads, search engine marketing, etc. Traditional / human channels include outbound phone calls, offline store promotions, direct mail advertising, etc.

[0065] According to embodiments of this disclosure, an objective refers to a change in behavior or state that a user is expected to achieve in a particular business scenario. For example, it could encourage potential users or registered but not yet purchased users to complete their first purchase or key actions; reactivate users who have been inactive for a period of time, or win back users with a tendency to churn; incentivize existing users to purchase again, or to purchase higher-value goods / services; guide users to purchase related products or upgrade to higher-level products / services; increase the frequency, duration, or depth of user engagement with products / services; or encourage users to try newly launched features or products, etc.

[0066] By defining channels and objectives, business scenarios can be exemplified as follows: using the APP push channel to reactivate users whose activity has recently declined; using the SMS channel to push information about new products in category B to users with a history of high value who have purchased category A in order to promote cross-selling; and using outbound telephone calls to guide new customer conversion for users who have registered but have not completed their first order within 7 days.

[0067] In the embodiments of this disclosure, for the business scenario to be evaluated, target user data can be filtered out into intervention-treated (treatment group) and non-intervention-treated (or control group receiving different interventions). For example, in evaluating the scenario of "APP push to wake up silent users", target users can include users who have been pushed wake-up messages and silent users with similar characteristics who have not been pushed (or have been pushed placebo messages).

[0068] Target user data may include user identification information, such as user ID; basic user profile data, such as age, gender, geographical location, registration time, membership level, income level (if any); and user historical behavior data, such as last active time, historical access frequency, historical purchase amount, main functional modules used, historical purchase frequency, registration source channel, and whether and when the user was subject to specific interventions in the business scenario being evaluated.

[0069] It should be noted that the selection of user historical behavior data should be related to the goals and channels of the business scenario.

[0070] Optionally, the target user data may also include contextual information for accurate modeling. For example, this may include the time when the intervention occurred, the type of device the user was using at the time, and the network environment. It may also include expected outcome metrics, i.e., metrics observed over a period of time after the intervention to measure its effectiveness.

[0071] In the embodiments of this disclosure, the original target user data often has problems such as missing data, anomalies, and inconsistent formats. Directly using it for model training may lead to distorted results or the model failing to run. Therefore, appropriate preprocessing operations are required before the data is input into the model.

[0072] For example, depending on the missing percentage and feature type, imputation can be performed using the mean / median / mode, model-based imputation (such as KNN imputation, regression imputation), or by directly deleting samples / features with excessive missing values. For instance, if user age data is partially missing, and the missing percentage is not high, the mean of the sample age can be used for imputation.

[0073] For example, outliers can be identified and addressed using statistical methods or business experience. For instance, if a user's spending exceeds the normal range within a short period, it may be necessary to verify or implement a "blocking" mechanism.

[0074] For example, data can be converted into a type suitable for the model to process, such as converting a string representing a date into a datetime object, or converting a text-based number into a numeric type.

[0075] In operation S220, based on the business scenario to be evaluated, a target causal inference model is matched from multiple pre-built causal inference models, wherein the multiple causal inference models are pre-built based on causal modeling datasets of multiple business scenarios.

[0076] Specifically, to achieve accurate evaluation and personalized prediction of marketing effectiveness, the embodiments of this disclosure divide the data into scenarios according to different business needs, forming multiple causal modeling datasets, and constructing independent causal inference models on each dataset. These causal inference models take into account the differences in data characteristics, user behavior, intervention methods, and other dimensions of different business scenarios during construction, thus enabling more accurate modeling and capture of the real effects of marketing interventions in specific scenarios.

[0077] For example, in real-world business scenarios, different marketing channels (such as SMS marketing, app push notifications, and outbound calls) typically exhibit drastically different data distribution characteristics and user response patterns. Taking SMS marketing as an example, because users receive SMS messages less frequently and are easily restricted by operator policies, SMS marketing data often exhibits high sparsity, fewer feature dimensions, and shorter user conversion cycles. In contrast, app push notifications, due to more frequent user interactions, generate larger data volumes and richer feature dimensions, resulting in more complex dynamic temporal characteristics in user responses. Furthermore, outbound call channels rely more heavily on manual intervention by account managers, and their user response patterns are significantly influenced by variables such as communication skills and user profile characteristics. Therefore, using a single global causal model for prediction often fails to accurately capture these channel differences, reducing the model's predictive accuracy and generalization ability.

[0078] Furthermore, even within the same marketing channel, different marketing objectives require different causal inference models. For example, for the "new customer conversion" objective, user response characteristics are centered on "new user registration behavior" and "first order behavior," exhibiting significant short-term impulsive consumption characteristics, thus the feature space is relatively simple. However, for the "dormant user reactivation" objective, the feature space is more complex. Users often experience long incubation periods, and conversion characteristics manifest as complex sequence features such as "multiple interval visits" and "gradually increasing activity," typically requiring modeling behavioral characteristics over a longer time window. In addition, the "promoting activity" objective falls between these two, focusing on characterizing various interaction features such as user visit frequency, duration, and behavioral paths within a short period. Therefore, establishing independent causal inference models for each marketing objective is more effective in capturing the causal relationship between marketing interventions and customer responses under specific objectives.

[0079] In the embodiments of this disclosure, a dynamic causal inference model library can be constructed, which includes implementations of various causal inference methodologies and is optimized for different data characteristics and business problem types.

[0080] For example, propensity score matching can be used to estimate the probability of each user receiving the intervention (propensity score), and then users with similar scores can be matched between the treatment and control groups to balance observable confounding factors. This approach is suitable for cross-sectional data and can be used to simulate random assignment when randomized experiments are not feasible.

[0081] For example, the conditional average treatment effect can be estimated using existing machine learning models (such as random forests, gradient boosting machines, and neural networks) based on a meta-learner. For instance, independent machine learning models can be trained on treatment and control group data respectively to predict outcomes, and the conditional average treatment effect can be estimated by the difference between the predictions of the two models.

[0082] For example, the heterogeneity treatment effect and the average treatment effect can be estimated based on causal forests.

[0083] When constructing multiple causal inference models, different model structures and parameter configurations can be selected for each. For example, for scenarios with large amounts of data and complex feature relationships (such as waking up dormant users in APP push notifications), a larger-scale causal forest model can be constructed, with a higher number of trees and node partitioning depth to capture complex user response patterns more meticulously. On the other hand, for scenarios with sparse data and simple features in SMS marketing, a simpler causal decision tree model or a more concise causal forest model can be used to improve the stability and efficiency of the model.

[0084] According to embodiments of this disclosure, during the model building phase, each pre-trained causal inference model can be bound to a unique business scenario identifier, which consists of multiple dimensions, such as "marketing channel + marketing goal + user segmentation tag". During the evaluation phase, the system can extract the corresponding scenario identifier based on the scenario definition of the business to be evaluated, and find the model that matches the identifier as the target causal inference model.

[0085] Optionally, a matching strategy based on business feature similarity can be adopted to extract the core feature statistics of the scenario to be evaluated, calculate the feature statistics distance with the corresponding scenario of each pre-trained model, such as cosine similarity, KL divergence, cluster center distance, etc., and select the model with the highest similarity as the target causal inference model for the current business.

[0086] In operation S230, the target user data is input into the target causal inference model to obtain average treatment effect information. ATE can be used to measure the improvement level of expected business results (such as activity, conversion rate, and spending) of the entire target user group after receiving a specific marketing intervention compared to the situation without intervention.

[0087] In operation S240, based on the comparison results of the average processing effect information and the preset threshold conditions, the evaluation results of the marketing effect are obtained.

[0088] In the embodiments of this disclosure, the preset threshold conditions can be set according to specific business objectives, scenario characteristics, or historical experience.

[0089] For example, a preset ATE ≥ 0.05 (meaning that the marketing increased the conversion probability by 5%) is considered effective, while anything below this threshold is considered ineffective.

[0090] For example, a threshold can be automatically calculated based on the distribution of ATEs in historical scenarios or business deployment requirements, such as setting it as "the median of ATEs in historical scenarios + one standard deviation".

[0091] For example, different thresholds can be applied to different channels, targets, or user groups. For instance, SMS channels require ATE ≥ 0.03, APP channels require ATE ≥ 0.10, and telephone channels require 0.00.

[0092] In addition to the ATE value itself, other outputs of the model can be combined to assist decision-making. For example, ATE confidence intervals (such as 95% CI); comparisons of ATE between subgroups (such as male and female, different risk levels), etc.

[0093] For example, if ATE > 0 and the confidence level is high, the campaign is considered effective overall and can continue to be implemented in layers; if ATE is close to 0 or the confidence interval contains negative values, the intervention method or target group needs to be reassessed; if ATE < 0, the marketing strategy has a negative impact on the target group and the campaign needs to be suspended or the intervention path switched.

[0094] In the embodiments of this disclosure, after completing the ATE assessment and determining that the current marketing campaign has a positive impact on the business scenario, fine-grained individual processing effect information can be generated for individual users, and actionable marketing strategy recommendations can be generated accordingly.

[0095] Individual treatment effect information (ITE) is the individual-level prediction result obtained synchronously during the inference process of the target causal inference model. ITE is used to characterize the difference between the predicted business outcomes for each user under different treatment conditions:

[0096] I (1)

[0097] in, This represents the predicted outcome (such as conversion probability) for user i when receiving marketing treatment. This represents the predicted result before any intervention. A higher ITE indicates that the user is more likely to be positively affected by the marketing intervention. After calculating the ITE for each target user group, the system can generate a complete list of individual treatment effect scores, which can be used for further stratified processing and strategy generation.

[0098] In embodiments of this disclosure, based on the heterogeneous distribution of the individual treatment effects, the system can stratify target users according to their response sensitivity to generate a customer group list. The stratification can be achieved by setting an adjustable threshold or by automatic quantile sorting. For example, the stratification may include the following:

[0099] 1. Highly sensitive group: ITE scores in the top 20% to 30% percentile indicate a high responsiveness to current marketing campaigns, making them a recommended target group for targeted advertising;

[0100] 2. Moderately sensitive groups: ITE values ​​in the middle range (e.g., 30%–70%) may be effective for the activity, but the response is unstable;

[0101] 3. Low-sensitivity groups: Those with lower ITE values ​​(e.g., the bottom 30%) have weak marginal effects on their response to activities, so it is recommended to reduce the frequency of outreach.

[0102] 4. Negative response group: ITE < 0, indicating that the marketing activity may have a negative effect and should be set to exclude or postpone the reach.

[0103] After obtaining the customer list, the system can output various types of marketing strategy suggestions based on ITE (Information Context) stratification results and individual characteristics. For example, highly sensitive users can be prioritized for marketing campaigns, while users with negative responses can be excluded. For example, the best channels such as SMS, app push notifications, and outbound calls can be matched to different groups based on ITE and user preferences. For example, the most relevant product recommendations or incentives can be pushed based on the behavioral characteristics of high ITE users (such as browsing products and historical purchase categories). For example, different marketing budgets can be allocated to groups with different response levels.

[0104] The preferred embodiments of this disclosure will be introduced below using a causal inference model based on a causal forest as an example.

[0105] In the embodiments of this disclosure, at least one of the multiple causal inference models is a set of causal decision trees constructed based on minimizing the error of the intervention effect, namely a causal forest model. It is essentially a nonparametric modeling method that combines traditional random forest with the principle of causal inference. It can be used to estimate ITE and ATE simultaneously and has strong nonlinear modeling ability and individual response difference capture ability.

[0106] The core objective of causal forests is to maximize the diversity of random trees after partitioning based on feature variables. Furthermore, when constructing the random causal tree model, the causal forest algorithm employs a different standard metric, abandoning the random forest's focus on mean squared error and instead using mean squared error based on the intervention effect as the standard:

[0107] (2)

[0108] In this process, P and C represent the positions of the parent node and the child node, respectively, while n is the number of samples, and θ represents the processing effect to be found, that is, to find a way to determine the most ideal cut point so that the processing effect between the two child nodes has the greatest difference.

[0109] For each internal node, all data is homogeneous. Within each node, the active ATE can be estimated using the following formula:

[0110] (3)

[0111] By continuously dividing the nodes, the causal tree is constructed once certain pre-set termination criteria are met (such as minimum sample size, degree of imbalance in treatment effects, etc.).

[0112] In the embodiments of this disclosure, the business scenario can be divided using a cross-combination strategy of marketing channel dimension and marketing objective dimension. For example, the marketing channel dimension includes, but is not limited to: SMS push, APP push, outbound telephone calls, email marketing, mini-program reminders, etc.; the marketing objective dimension includes, but is not limited to: new customer conversion, dormant customer reactivation, user activity improvement, purchase frequency increase, repeat purchase incentives, benefit redemption, and upgrade guidance, etc.

[0113] By combining the two dimensions mentioned above using a Cartesian product, such as SMS + new customer conversion, or APP push + silent wake-up, we can obtain a set of business sub-scenarios with clear semantic boundaries. Within each sub-scenarios, the user's intervention logic, response behavior, time window, and feature composition are all different.

[0114] For each defined business sub-scenario, a dedicated causal modeling dataset can be constructed, containing three core variables required for causal inference within that scenario:

[0115] 1. Outcome Variable (Y): This is a target business indicator used to measure whether a marketing intervention has produced the expected impact. It has a clear business meaning, must be strongly correlated with the marketing objective, and must be observable in the data. Examples include: conversion rate (0 / 1); activity score (e.g., number of days visited, total page dwell time); order amount / frequency; whether benefits were claimed, whether upgrades were made, and whether the user was retained.

[0116] 2. Intervention variable (T) indicates whether the user accepted the marketing treatment defined in this scenario. It is usually a binary variable (0 indicates no treatment, 1 indicates treatment). For example: whether the user received an SMS; whether the user answered a phone call; whether the user was reached within the app. Intervention variables can be generated from operation logs, sending records, and feedback from third-party push systems.

[0117] 3. Covariates (X) are a set of features used to control for confounding effects. For example, they can include two categories: static profile features: gender, age, region, customer level, account opening time, risk rating, etc.; dynamic behavioral features: number of visits in the past 30 days, click path, number of active days, number of historical conversions, page dwell time, device type, etc.

[0118] It should be noted that covariates should be observable, business-relevant, and potentially influence both intervention and outcome variables, thus meeting the prerequisites for confounding control.

[0119] In the embodiments of this disclosure, each piece of historical marketing data can belong to a business sub-scenario, that is, the business scenarios are mutually exclusive, in order to avoid interference or label conflicts during model training and ensure the accuracy of processing effect estimation.

[0120] Furthermore, to improve the efficiency of causal inference model construction, enhance the stability of treatment effect estimation, and improve the interpretability of the model, embodiments of this disclosure introduce a mechanism for primary screening of training features during the model training phase.

[0121] Figure 3 A flowchart illustrating a method for primary screening of training features according to an embodiment of the present disclosure is shown.

[0122] like Figure 3 As shown, the method for primary screening of training features in this embodiment may include operations S310 to S330.

[0123] In operation S310, the importance score of each feature in the causal inference model is calculated.

[0124] Specifically, in the process of constructing multiple causal decision trees, the causal forest model selects a feature as the splitting variable each time a node is split. The system can record the frequency with which each feature is used for splitting across all causal trees; the improvement in treatment effect difference resulting from node splitting (such as improving ITE variance discrimination); or calculate the feature gain score in splitting based on the Gini coefficient. These metrics can serve as importance scores for each feature in the causal modeling task, reflecting its actual contribution to revealing treatment effect differences.

[0125] In operation S320, the importance scores are aggregated to obtain a feature importance ranking. After obtaining the importance scores of each feature in a single tree or multiple sub-models, these indicators can be aggregated to generate a global feature importance ranking list. Aggregation methods may include: taking the average score; weighted summation (e.g., reducing the weight of deeper nodes); ranking statistics (e.g., using a voting method to count the frequency of entering the top N positions), etc.

[0126] In operation S330, the top K features ranked by feature importance are selected to form a candidate key feature set. Here, K is determined based on the total number of features input to the model and the business scenario objectives. These candidate key features are used to pre-screen features during the construction of the causal decision tree set.

[0127] For example, the value of K can be flexibly configured based on the following factors: the total number of original features (e.g., taking the top 20%, top 50, etc.); model training resource constraints (e.g., memory / computing time).

[0128] Business scenario requirements (e.g., if more emphasis is placed on interpretability, the value can be smaller; if more emphasis is placed on model capabilities, the value can be more flexible).

[0129] According to embodiments of this disclosure, the candidate key feature set will be used as a priority variable pool for partitioning during the causal forest modeling process. Specifically, this can reduce the partitioning search space and improve the efficiency of maximizing the effect differences of each causal tree during the construction process; at the same time, the final model result depends more on key variables, which is beneficial for explaining to the business "which variables drive the effectiveness of the strategy". Furthermore, in resource-constrained devices or large-scale real-time application scenarios, the candidate feature mechanism can significantly reduce inference latency.

[0130] For example, in a scenario involving in-app push marketing and activating dormant users, the original set of features consists of 130 items. After sorting, the top 40 are selected as candidate key features. These features may include "number of logins in the last 30 days," "whether promotional pages were viewed," and "average time spent in the app." This approach reduces model training time, significantly improves HTE diversity, and ensures stable interpretation results.

[0131] Embodiments of this disclosure also provide a verification method based on stability testing.

[0132] Figure 4 A flowchart illustrating a method for advanced verification of training features according to an embodiment of the present disclosure is shown.

[0133] like Figure 4 As shown, the advanced verification method for training features in this embodiment may include operations S410 to S430.

[0134] When operating the S410, the causal modeling dataset is resampled multiple times to generate multiple training subsets.

[0135] Specifically, multiple rounds of Bootstrap resampling can be performed on the original causal modeling dataset to generate several statistically diverse training subsets. For example, N training sets (e.g., N=100) can be constructed by sampling with replacement from the original dataset; each subset maintains the same sample size as the original to simulate different data perturbation environments. Simultaneously, the distribution structures of intervention variables, covariates, and outcome variables should be statistically comparable but differ in specific samples. This step provides sample-level perturbation sources for subsequent model training, simulating the differences in feature performance under different sample contexts, and forms the basis for feature stability testing.

[0136] In operation S420, for each training subset, the causal inference model is retrained, and the frequency of occurrence of features used to partition nodes in the causal inference model is recorded.

[0137] For example, the total number of times a feature is used as a splitting variable can be counted to obtain the stable performance of each feature across N model training iterations. Additionally, the stable performance of a feature can be determined by considering the depth distribution of the node it belongs to and whether it is the first splitting node.

[0138] In operation S430, an important feature subset is constructed based on the occurrence frequency, and the important feature subset is used for the interpretation of model results.

[0139] For example, a stability screening threshold (such as 80%, 90%, etc.) can be set to filter out variables that are selected in most models, forming the final important feature subset.

[0140] For example, in the scenario of SMS marketing + dormant customer reactivation, there are 50 original features; 100 resampling cycles are performed to train 100 causal forest models; it is found that "number of visits in the past 30 days", "whether it has been successfully reactivated", and "registration channel" are the most frequently occurring variables; finally, 10 important features are output to build HTE clustering strategies and visualization reports.

[0141] According to embodiments of this disclosure, the advanced verification mechanism can improve the credibility of feature selection in the causal inference process, avoid misselection of variables due to bias in training data, and make the final interpretation results more representative and suitable for business presentation.

[0142] In the embodiments of this disclosure, in order to further verify the applicability of the causal inference model in the actual business environment and the credibility of the prediction results, the system can select a group of representative target users as business pilot objects based on the determined important feature subset and individual processing effect information.

[0143] For example, pilot programs can select target groups from highly sensitive demographics or those covered by specific strategies, ensuring consistency in their characteristics and a high predictive response value on ITE (Information Technology). The system can then implement small-scale marketing interventions on these user groups, such as sending customized SMS messages, pushing activation content, or arranging outbound calls, and record their actual business conversion results over a subsequent period, including behaviors such as whether they log in, make a purchase, or complete activation.

[0144] Furthermore, a deviation analysis can be performed between the actual conversion data of the pilot subjects and their predicted values ​​in the causal inference model, comparing the consistency of the two in terms of directionality (positive / negative) and numerical deviation (such as error range, hit rate, etc.). If the deviation is within the set tolerance range, or the prediction accuracy reaches the business acceptance standard, the model is considered to have good stability under the current feature space and target strategy; conversely, if the deviation is significant, it indicates that the model has a risk of generalization or insufficient feature interpretation, which can trigger subsequent mechanisms such as model adjustment, feature re-screening, or further sampling training. This process constitutes a closed-loop verification mechanism for model evaluation to feed back to actual business, ensuring that the model output not only performs well on offline data, but also maintains reasonable prediction effects in real user behavior, improving the reliability and feasibility of the model in the business deployment stage.

[0145] According to embodiments of this disclosure, by introducing a "causal inference model based on business scenario matching," a high degree of automation and scenario-based divide-and-conquer approach to the marketing effectiveness evaluation process is achieved. Compared to causal inference methods based on RCT or PSM, this approach offers significant technical advantages and beneficial effects. First, the solution according to embodiments of this disclosure leverages the structural characteristics of nonlinear models such as causal forests, enabling automatic feature selection and modeling of complex variable relationships. This effectively simplifies the extensive feature engineering operations required by traditional linear models, reducing data processing costs. Second, causal forests do not rely on strict experimental / control group sample construction conditions, resulting in stronger adaptability. This avoids the stringent requirements of RCT experiments regarding sample quantity and distribution consistency, enhancing the model's practicality and flexibility in real marketing operations. Furthermore, through model pre-training and matching mechanisms across multiple business scenarios, scenario-based evaluation and modeling divide-and-conquer based on both channel and marketing objectives are achieved. This breaks through the limitations of traditional single-model applications, improving the relevance and accuracy of evaluation results and meeting the urgent needs of financial institutions for intelligent and refined marketing evaluation.

[0146] Corresponding to the above-described method for evaluating marketing effectiveness, embodiments of this disclosure also provide a device for evaluating marketing effectiveness.

[0147] Figure 5 A schematic block diagram of a marketing effectiveness evaluation apparatus according to an embodiment of the present disclosure is shown.

[0148] like Figure 5 As shown, the marketing effectiveness evaluation device 500 of this embodiment includes a data acquisition module 510, a model matching module 520, a model processing module 530, and an evaluation result acquisition module 540.

[0149] The data acquisition module 510 can be used to acquire target user data for the business scenario to be evaluated. In one embodiment, the data acquisition module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0150] The model matching module 520 can be used to match a target causal inference model from a plurality of pre-built causal inference models based on the business scenario to be evaluated, wherein the plurality of causal inference models are pre-built based on causal modeling datasets of multiple business scenarios. In one embodiment, the model matching module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0151] The model processing module 530 can be used to input the target user data into the target causal inference model to obtain average treatment effect information. In one embodiment, the model processing module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0152] The evaluation result acquisition module 540 can be used to obtain an evaluation result of the marketing effect based on the comparison result of the average processing effect information and the preset threshold conditions. In one embodiment, the evaluation result acquisition module 540 can be used to perform the operation S240 described above, which will not be repeated here.

[0153] According to embodiments of this disclosure, the model matching module 520 can also be used to cross-combine and divide historical user data based on marketing channel dimension and marketing goal dimension to obtain multiple mutually exclusive business scenarios; and to construct a corresponding causal modeling dataset for each business scenario, wherein the causal modeling dataset includes the outcome variable, intervention variable and covariate of the corresponding business scenario.

[0154] According to embodiments of this disclosure, the model matching module 520 can also be used to calculate the importance score of each feature in the causal inference model; aggregate the importance scores to obtain the feature importance ranking; and select the features with the top K importance rankings to form a candidate key feature set, wherein K is determined according to the total number of model input features and the business scenario objective, and the candidate key features are used to pre-screen features during the construction of the causal decision tree set.

[0155] According to embodiments of this disclosure, the model matching module 520 can also be used to resample the causal modeling dataset multiple times to generate multiple training subsets; for each training subset, retrain the causal inference model and record the frequency of occurrence of features used to divide nodes in the causal inference model; and construct an important feature subset based on the occurrence frequency, the important feature subset being used for model result interpretation.

[0156] According to embodiments of this disclosure, the model matching module 520 can also be used to select business pilot targets based on the important feature subset and individual treatment effect information; obtain the actual conversion results returned by marketing intervention on the business pilot targets; and perform deviation analysis between the actual conversion results and the predicted values ​​of the causal inference model, and obtain stability test results based on the results of the deviation analysis.

[0157] According to embodiments of this disclosure, the marketing effectiveness evaluation device 500 can also be used to obtain individual treatment effect information by inputting the target user data into the target causal inference model; and in response to a positive marketing effectiveness evaluation result, generate marketing strategy suggestions for the business scenario to be evaluated based on the individual treatment effect information.

[0158] According to embodiments of this disclosure, the marketing effectiveness evaluation device 500 can also be used to perform response sensitivity stratification of target users based on the heterogeneous distribution of the individual processing effects, obtain a customer group list, and generate marketing strategy suggestions for the business scenario to be evaluated based on the customer group list.

[0159] According to embodiments of this disclosure, any multiple modules among the data acquisition module 510, model matching module 520, model processing module 530, and evaluation result acquisition module 540 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the data acquisition module 510, model matching module 520, model processing module 530, and evaluation result acquisition module 540 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the data acquisition module 510, model matching module 520, model processing module 530, and evaluation result acquisition module 540 can be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0160] Figure 6 A block diagram of an electronic device suitable for implementing a marketing effectiveness evaluation method according to an embodiment of the present disclosure is shown schematically.

[0161] like Figure 6As shown, an electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage portion 606 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0162] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0163] According to embodiments of this disclosure, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0164] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0165] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0166] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to enable the computer system to implement the marketing effectiveness evaluation methods provided by embodiments of this disclosure.

[0167] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0168] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0169] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0170] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0172] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0173] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for evaluating marketing effectiveness, characterized in that, The method includes: Obtain target user data for the business scenario to be evaluated; Based on the business scenario to be evaluated, a target causal inference model is matched from multiple pre-built causal inference models, wherein the multiple causal inference models are pre-built based on causal modeling datasets of multiple business scenarios; The target user data is input into the target causal inference model to obtain average treatment effect information; and Based on the comparison results of the average treatment effect information and the preset threshold conditions, the evaluation results of the marketing effect are obtained.

2. The method according to claim 1, characterized in that, The method further includes: Based on inputting the target user data into the target causal inference model, individual treatment effect information is obtained; and In response to a positive evaluation result for marketing effectiveness, marketing strategy recommendations for the business scenario to be evaluated are generated based on the individual processing effect information.

3. The method according to claim 2, characterized in that, The method also includes constructing a causal modeling dataset, specifically including: Historical user data is cross-combined and segmented based on marketing channel and marketing objective dimensions to obtain multiple mutually exclusive business scenarios; and For each business scenario, a corresponding causal modeling dataset is constructed, which includes the outcome variables, intervention variables, and covariates of the corresponding business scenario.

4. The method according to claim 2 or 3, characterized in that, At least one of the multiple causal inference models is a set of causal decision trees constructed based on minimizing the error of the intervention effect.

5. The method according to claim 4, characterized in that, In the process of constructing a causal inference model based on minimizing the error of the intervention effect, the method further includes: Calculate the importance score of each feature in the causal inference model; The importance scores are aggregated to obtain a ranking of feature importance; and Select the top K features by importance to form a candidate key feature set, where K is determined at least based on the total number of input features of the model or the business scenario objective. The candidate key features are used to pre-screen features during the construction of the causal decision tree set.

6. The method according to claim 4 or 5, characterized in that, After constructing a causal inference model based on minimizing the error of the intervention effect, the method further includes: The causal modeling dataset is resampled multiple times to generate multiple training subsets; For each training subset, the causal inference model is retrained, and the frequency of features used to partition nodes in the causal inference model is recorded; and An important feature subset is constructed based on the frequency of occurrence, and this important feature subset is used for interpreting the model results.

7. The method according to claim 6, characterized in that, The method further includes: Based on the aforementioned important feature subset and individual processing effect information, pilot projects were selected. Obtain the actual conversion results returned by the marketing intervention on the pilot business targets; and The actual transformation results are compared with the predicted values ​​of the causal inference model using a deviation analysis, and the stability test results are obtained based on the deviation analysis results.

8. The method according to any one of claims 2 to 3, 5, and 7, characterized in that, The evaluation result of the marketing effectiveness is positive. Based on the individual processing effect information, marketing strategy suggestions are generated for the business scenario to be evaluated, specifically including: Based on the heterogeneous distribution of the individual treatment effects, target users are stratified by response sensitivity to obtain a customer list; and Based on the customer list, marketing strategy recommendations are generated for the business scenarios to be evaluated.

9. A device for evaluating marketing effectiveness, characterized in that, The device includes: The data acquisition module is used to: acquire target user data for the business scenario to be evaluated; The model matching module is used to: match a target causal inference model from a plurality of pre-built causal inference models based on the business scenario to be evaluated, wherein the plurality of causal inference models are pre-built based on causal modeling datasets of the plurality of business scenarios; The model processing module is used to: input the target user data into the target causal inference model to obtain average treatment effect information; and The evaluation result acquisition module is used to obtain the evaluation result of the marketing effect based on the comparison result of the average processing effect information and the preset threshold conditions.

10. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images