Zero sample marketing gain modeling method and system based on large language model

Abstract

The invention discloses a zero-sample marketing gain modeling method and system based on a large language model, and effectively solves the technical bottleneck that an individual processing effect cannot be effectively estimated due to lack of historical samples when a traditional gain model faces brand-new marketing intervention. The core of the method comprises the following steps: converting user characteristics and marketing scene description into natural language prompts, activating pre-trained consumer behavior common sense by using a large language model to perform semantic reasoning, and respectively estimating potential results of a user under intervention and non-intervention by means of a double-branch anti-fact prediction architecture, and the individual processing effect is calculated. According to the method, accurate gain estimation without historical data of a target scene is realized, the method can be widely applied to cold start scenes such as personalized marketing and new product popularization, the A / B test cost of an enterprise is remarkably reduced, and the marketing decision intelligent level is improved.

Description

Technical Field

[0001] This invention relates to artificial intelligence and data mining technology, specifically the application of artificial intelligence in marketing technology, and particularly to a zero-sample marketing gain modeling method and system based on a large language model. Background Technology

[0002] In the digital economy, personalized marketing is key for businesses to improve conversion efficiency. Traditional marketing models (such as click-through rate prediction) only focus on "whether users respond" and ignore whether marketing interventions actually "trigger" behavioral changes. Therefore, marketing modeling is shifting from "response prediction" to "incremental response estimation", that is, assessing the individual treatment effect (ITE) or uplift modeling.

[0003] The core objective of gain modeling is to estimate the causal effect of an intervention on individual outcomes from observational data. Its formal definition is:

[0004]

[0005] Where: Y(1) represents the potential outcome of an individual under intervention; Y(0) represents the potential outcome of an individual without intervention; X is a user covariate (such as historical behavior, profile, etc.).

[0006] Gain value This intuitively reflects the expected incremental value that can be obtained by applying intervention to a specific user x. According to Based on the positive and negative values ​​and magnitudes, users can be divided into four typical groups (such as...). Figure 1 As shown): (1) Persuadables: >0, the target population that will only respond when intervention is received. (2) Sure Things: For users who will respond regardless of intervention, intervention would be a waste of resources. (3) Dormant users (LostCauses): Users who do not respond regardless of intervention. (4) Do-Not-Disturbs: <0, users who are resentful of intervention.

[0007] Existing gain modeling methods can be mainly divided into three categories: (1) Meta-learner methods: These methods reuse traditional supervised learning models and combine different approaches to fit potential results. Representative methods include S-learner, T-learner, X-learner, etc. Their advantages are simple implementation and strong compatibility, but their performance is highly dependent on the expressive power of the base model and the sample coverage. (2) Tree-based methods: such as Uplift Tree, Causal Tree and Causal Forest, which identify user subgroups sensitive to intervention by modifying the splitting criteria, have good interpretability, but have limited modeling ability in high-dimensional nonlinear scenarios. (3) Neural network-based methods: such as TarNet and CFRNet, which use deep networks to learn causal representations and introduce distribution alignment mechanisms to alleviate selection bias. Subsequent works such as EFIN and DESCN further optimize feature interaction and full-space modeling, improving the stability of estimation.

[0008] However, the methods mentioned above all rely on supervised training using historical intervention-response data, which inherently cannot handle unseen intervention types and lack zero-shot cross-scenario generalization ability. When faced with entirely new marketing strategies, companies often need to conduct expensive and time-consuming A / B testing to collect data, resulting in high cold start costs and long cycles.

[0009] In recent years, large language models (LLMs) have shown significant potential in causal inference tasks. Research directions mainly include: (1) Combining causal inference with large models: LLMs have been used for tasks such as variable extraction, causal discovery, and counterfactual generation. For example, the NATURAL framework enables end-to-end estimation of causal effects from unstructured text; Causal Prompting alleviates cue bias through front-door adjustment; and some studies have used LLMs as causal effect generators or to construct autonomous causal analysis agents (such as Causal-Copilot). However, existing work mainly focuses on average treatment effect estimation or qualitative causal discovery, and no research has systematically applied LLMs to individual treatment effect estimation, especially in zero-sample marketing gain modeling tasks. (2) Application of large models in search recommendation systems: LLMs have significantly improved the cold-start performance and interpretability of recommendation systems through semantic encoding, interest summary generation, retrieval enhancement, and other means. Mainstream paradigms include LLM-to-Rec (using LLM to enhance feature representation), Rec-to-LLM (converting user behavior into prompts to directly generate recommendations), and native recommendation models (Generative Recommenders). These advancements provide important insights for integrating semantic understanding into gain modeling. (3) General structured data models: such as TURL and LimitX, which learn high-order interactions and distribution patterns between features through pre-training on large-scale tabular data, exhibiting strong generalization capabilities under zero-shot or small-shot settings. This provides a new model foundation for zero-shot causal inference based directly on structured user features.

[0010] While LLM has made progress in tasks such as click-through rate prediction, its application in the core marketing task of gain modeling remains unexplored. Gain modeling requires models to possess counterfactual reasoning capabilities and robustness to selection bias, but existing LLM research has not systematically designed to address these needs. Therefore, effectively integrating the semantic priors and causal inference frameworks of LLM to achieve zero-sample individual treatment effect estimation has become a key challenge.

[0011] Based on the analysis of the current research status in the field of gain modeling, the existing technology has the following main shortcomings and technical bottlenecks: 1. Lack of zero-shot cross-intervention generalization ability: Whether it's meta-learner methods, decision tree-based methods, or causal representation learning methods based on neural networks (such as CFRNet), their training and inference heavily rely on the sample distribution in historical intervention-response data. When faced with entirely new or undescribed marketing interventions (such as new promotional themes or newly launched interactive formats), the lack of corresponding historical samples prevents the model from making effective estimates. Data must be collected and retrained, resulting in high cold-start costs and lengthy cycles, failing to meet the needs of rapid iterative marketing decision-making.

[0012] 2. Insufficient Semantic Modeling and Causal Reasoning Abilities: Traditional methods typically compress user characteristics (such as spending amount and channel) and intervention descriptions into low-dimensional numerical vectors or one-hot encodings. This representation severs the rich semantic relationships between features, making it difficult to characterize causal logic such as "why users are sensitive to specific interventions." The model essentially learns statistical correlations rather than causal reasoning based on semantic common sense. Therefore, its generalization performance significantly declines when facing cross-domain transfer or semantically novel interventions.

[0013] 3. Failure to systematically integrate the semantic priors and causal inference framework of large language models: Although large language models have demonstrated excellent semantic understanding and zero-shot reasoning capabilities in various cognitive tasks, and existing research has explored their applications in causal variable extraction and effect generation, no work has yet systematically and deeply integrated their semantic priors with Individual Treatment Effects (ITE) estimation tasks. In particular, under zero-shot conditions, how to guide LLMs to understand counterfactual task instructions, activate relevant consumer behavioral common sense, and output robust ITE estimates remains a largely unexplored technological gap.

[0014] In summary, existing methods for estimating individual treatment effects cannot directly estimate them when faced with new marketing scenarios due to the lack of corresponding historical samples. Traditional methods rely on historical intervention-response data for supervised training, resulting in a significant performance drop in cold-start scenarios. Specifically, they cannot achieve zero-sample individual treatment effect estimation across marketing scenarios; they lack causal reasoning capabilities based on semantic logic when facing unseen marketing scenarios; and the models rely solely on structured numerical features, severing the semantic connection between user characteristics and marketing scenarios, thus limiting generalization ability.

[0015] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0016] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide a zero-sample marketing gain modeling method and system based on a large language model.

[0017] To achieve the above objectives, the present invention adopts the following technical solution: A zero-shot marketing gain modeling method based on a large language model includes the following steps: S1. Prompt Construction: Based on the structured characteristics of the target user and the text description of the target marketing intervention, construct natural language prompts containing counterfactual task instructions; S2. Semantic Encoding and Representation Learning: Encode the natural language prompts using a pre-trained large language model and extract a joint representation vector that integrates user semantics and intervention semantics; S3. Counterfactual prediction: Input the joint representation vector into a two-branch prediction network. The first branch of the network outputs the first potential outcome prediction value when the user receives the target marketing intervention, and the second branch outputs the second potential outcome prediction value when the user does not receive any intervention. S4. Gain Calculation and Output: Calculate the difference between the first potential outcome prediction value and the second potential outcome prediction value, and output it as the individual treatment effect estimate of the user for the target marketing intervention.

[0018] Further, in step S1, constructing the natural language prompt containing counterfactual task instructions specifically includes: Transform the numerical and categorical structured features of the target user into descriptive natural language fragments; Use the textual description of the targeted marketing intervention as the context of the intervention; The descriptive natural language fragments, the intervention context, and the counterfactual task instructions that explicitly require the model to estimate the potential outcomes for the user under intervention and without intervention are combined to form a complete natural language prompt text.

[0019] Furthermore, in step S2, when encoding using a pre-trained large language model, a parameter-efficient fine-tuning technique is employed to adapt the large language model.

[0020] Further, in step S2, the extraction of the joint representation vector that integrates user semantics and intervention semantics specifically includes: The sequence representation output by the large language model is pooled to generate the joint representation vector of fixed dimensions.

[0021] Furthermore, between step S2 and step S3, step S2a, representing distribution alignment, is also included: During model training, based on the joint representation vector extracted from the large language model, the distribution distance between the intervention group samples and the control group samples in the representation space is calculated, and this distribution distance is added as a regularization term to the loss function for optimization, so as to reduce the distribution difference between the two groups.

[0022] Furthermore, the representation distribution alignment is achieved by optimizing a joint loss function, which includes a fact prediction loss term and a representation balance loss term; The fact prediction loss term is calculated solely based on the actual observed results of each sample and its corresponding branch prediction value; The representation balance loss term is calculated based on a selected distribution distance metric and is used to make the representation distributions of the intervention group and the control group converge.

[0023] Furthermore, the distribution distance metric used in the balance loss term is the maximum mean difference.

[0024] Furthermore, in step S3, the two branches of the dual-branch prediction network are neural networks with non-shared parameters, sharing the joint representation vector extracted in step S2 as input; The first branch is used to receive the joint representation vector and output a first potential outcome prediction value, and the second branch is used to receive the joint representation vector and output a second potential outcome prediction value.

[0025] A zero-shot marketing gain modeling system based on a large language model includes: Prompt building module: Based on the structured characteristics of the target user and the text description of the target marketing intervention, construct natural language prompts containing counterfactual task instructions; Semantic encoding and representation learning module: Encodes the natural language prompts using a pre-trained large language model and extracts a joint representation vector that integrates user semantics and intervention semantics; Counterfactual prediction module: The joint representation vector is input into a two-branch prediction network. The first branch of the network outputs the first potential outcome prediction value when the user receives the target marketing intervention, and the second branch outputs the second potential outcome prediction value when the user does not receive any intervention. Gain Calculation and Output Module: Calculates the difference between the first potential outcome prediction value and the second potential outcome prediction value, and outputs it as an estimate of the individual treatment effect of the user on the target marketing intervention.

[0026] Furthermore, the system also includes: Distribution alignment module: During model training, based on the joint representation vector extracted from the large language model, the distribution distance between the intervention group samples and the control group samples in the representation space is calculated, and this distribution distance is added as a regularization term to the loss function for optimization, so as to reduce the distribution difference between the two groups.

[0027] The present invention has the following beneficial effects: This invention provides a zero-shot marketing gain modeling method and system based on a large language model, effectively solving the technical bottleneck of traditional gain models in estimating individual treatment effects due to the lack of historical samples when facing novel marketing intervention scenarios. Its core lies in transforming the causal inference task into a natural language understanding task, activating the semantic priors of the large language model through natural language prompts, thereby achieving zero-shot causal inference. Specifically, this method transforms the user's structured features and the textual description of the target marketing scenario into unified natural language prompts, guiding the large language model to understand user profiles, intervention content, and counterfactual task instructions in the semantic space. This activates the consumer behavior common sense learned during its pre-training phase, enabling reasonable causal inference for novel intervention scenarios.

[0028] Compared to traditional methods that heavily rely on historical intervention-response data for supervised training (such as those based on meta-learners, decision trees, or neural networks), this invention overcomes this dependence on historical samples, achieving zero-shot generalization across interventions. The solution primarily relies on three key technical aspects: first, it transforms structured features and intervention descriptions into natural language prompts through prompt construction, laying the foundation for semantic understanding and zero-shot generalization; second, it utilizes a large language model as a semantic encoder, activating its inherent world knowledge to achieve zero-shot reasoning; and third, it employs a bi-branch prediction structure to estimate the potential outcomes for users under intervention and non-intervention conditions, providing the necessary architecture for explicit computation of Individual Treatment Effects (ITE). These designs collectively enable the method to estimate gains for entirely new marketing scenarios without requiring historical samples from the target marketing scenario. This allows for wide application in cold-start scenarios such as personalized marketing and new product promotion, significantly reducing A / B testing costs and improving the intelligence level of marketing decisions.

[0029] Furthermore, the representation distribution alignment mechanism introduced in the method (such as using the maximum mean difference as a regularization term) is an optional performance enhancement module that can effectively alleviate selection bias and improve the unbiasedness and cross-scenario stability of the estimation. If this module is removed, the model can still work, but the estimation accuracy and robustness will decrease.

[0030] Experimental results demonstrate that this method not only performs exceptionally well in standard intra-domain tests but also achieves significant performance breakthroughs in the critical scenario of cross-intervention cold start. In this scenario, the method achieves an AUUC of 1.0009, outperforming the best baseline model CFRNet (0.9174); and a Lift@30% metric of 0.7321, also higher than CFRNet's 0.6581. This fully validates its effectiveness in activating the inherent consumer behavior common sense within the large language model through natural language prompting engineering and achieving reasonable generalization to unseen interventions, providing a practical and effective solution to real-world business pain points such as marketing cold start and new product promotion.

[0031] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0032] Figure 1 This is a marketing audience four-quadrant diagram.

[0033] Figure 2 This is an overall diagram of the zero-sample marketing gain modeling method based on a large language model, as described in this invention.

[0034] Figure 3 This is a system architecture diagram of zero-sample marketing gain modeling based on a large language model, according to an embodiment of the present invention.

[0035] Figure 4 This is a model architecture diagram of the zero-sample marketing gain modeling method based on a large language model according to an embodiment of the present invention. Detailed Implementation

[0036] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0037] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0038] This invention aims to address the technical bottleneck of traditional marketing gain models in effectively estimating individual treatment effects in novel intervention scenarios due to a lack of historical samples. It proposes a method that transforms causal inference tasks into natural language prompting tasks and utilizes the semantic priors of large language models for zero-shot inference. By activating the consumer behavior common sense inherent in large language models through natural language prompts and combining a two-branch counterfactual prediction and representation alignment mechanism, it achieves accurate gain estimation without the need for historical data of the target scenario. This significantly reduces the cost of A / B testing in cold start scenarios and improves the intelligence level of marketing decisions.

[0039] See Figures 2 to 4 This invention provides a zero-shot marketing gain modeling method based on a large language model, comprising the following steps: Step S1, Prompt Construction: Based on the structured characteristics of the target user and the text description of the target marketing intervention, construct a natural language prompt containing counterfactual task instructions; In some embodiments, step S1, specifically constructing the natural language prompt containing counterfactual task instructions, includes: S1.1 Transform the numerical and categorical structured features of the target user into descriptive natural language fragments; S1.2. Use the textual description of the target marketing intervention as the context of the intervention; S1.3. Combine the descriptive natural language fragments, the intervention context, and the counterfactual task instructions that explicitly require the model to estimate the potential outcomes for the user under intervention and without intervention, respectively, to form a complete natural language prompt text.

[0040] The prompt template only needs to fulfill the core functions of "feature description + intervention description + counterfactual instruction", and its specific wording and structure are all adjustable details.

[0041] Step S2, Semantic Encoding and Representation Learning: Encode the natural language prompts using a pre-trained large language model and extract a joint representation vector that integrates user semantics and intervention semantics; In some embodiments, during step S2, when encoding using a pre-trained large language model, a parameter-efficient fine-tuning technique is employed to adapt the large language model, such as Low-Rank Adaptive (LoRA) technology. The large language model can be a pre-trained autoregressive or masked language model.

[0042] In some embodiments, step S2, extracting the joint representation vector that integrates user semantics and intervention semantics, specifically includes: performing a pooling operation on the sequence representation output by the large language model to generate the joint representation vector of fixed dimensions.

[0043] In some embodiments, between step S2 and step S3, the following further step is included: Step S2a, Distribution Alignment: During model training, based on the joint representation vector extracted from the large language model, the distribution distance between the intervention group samples and the control group samples in the representation space is calculated. This distribution distance is then used as a regularization term in the loss function for optimization, reducing the distribution difference between the two groups. The distribution distance metric can be the maximum mean difference or Wasserstein distance, etc. Introducing the distribution distance metric as a regularization term can effectively alleviate selection bias and improve the unbiasedness and cross-scene stability of the estimation.

[0044] In some embodiments, the representation distribution alignment is achieved by optimizing a joint loss function, which includes a fact prediction loss term and a representation balance loss term. The fact prediction loss term is calculated based solely on the actual observed results of each sample and their corresponding branch prediction values. The representation balance loss term is calculated based on a selected distribution distance metric to encourage the representation distributions of the intervention group and the control group to converge. In one embodiment, the distribution distance metric used in the representation balance loss term is the maximum mean difference.

[0045] Step S3, Counterfactual Prediction: Input the joint representation vector into a two-branch prediction network. The first branch of the network outputs the first potential outcome prediction value when the user receives the target marketing intervention, and the second branch outputs the second potential outcome prediction value when the user does not receive any intervention. In some embodiments, in step S3, the two branches of the dual-branch prediction network are non-parameter-sharing neural networks, such as non-parameter-sharing multilayer perceptrons, sharing the joint representation vector extracted in step S2 as input; the first branch is used to receive the joint representation vector and output a first potential outcome prediction value, and the second branch is used to receive the joint representation vector and output a second potential outcome prediction value.

[0046] Step S4, Gain Calculation and Output: Calculate the difference between the first potential outcome prediction value and the second potential outcome prediction value, and output it as the individual treatment effect estimate of the user for the target marketing intervention.

[0047] This invention also provides a zero-shot marketing gain modeling system based on a large language model, comprising: Prompt building module: Based on the structured characteristics of the target user and the text description of the target marketing intervention, construct natural language prompts containing counterfactual task instructions; Semantic encoding and representation learning module: Encodes the natural language prompts using a pre-trained large language model and extracts a joint representation vector that integrates user semantics and intervention semantics; Counterfactual prediction module: The joint representation vector is input into a two-branch prediction network. The first branch of the network outputs the first potential outcome prediction value when the user receives the target marketing intervention, and the second branch outputs the second potential outcome prediction value when the user does not receive any intervention. Gain Calculation and Output Module: Calculates the difference between the first potential outcome prediction value and the second potential outcome prediction value, and outputs it as an estimate of the individual treatment effect of the user on the target marketing intervention.

[0048] In some embodiments, the system further includes: Distribution alignment module: During model training, based on the joint representation vector extracted from the large language model, the distribution distance between the intervention group samples and the control group samples in the representation space is calculated, and this distribution distance is added as a regularization term to the loss function for optimization, so as to reduce the distribution difference between the two groups.

[0049] This invention proposes a zero-shot marketing gain modeling method based on a large language model. By transforming user structured features and marketing intervention text into unified natural language prompts, and utilizing a pre-trained large language model for semantic encoding and reasoning, it overcomes the technical limitations of traditional gain modeling methods that heavily rely on historical intervention data and cannot handle unseen marketing scenarios. This method achieves accurate estimation of individual treatment effects under zero-shot conditions. By introducing a bi-branch counterfactual prediction architecture and an optional representation distribution alignment mechanism, it can not only output the potential results of users under intervention and non-intervention conditions and directly calculate the gain value, but also effectively alleviate selection bias. This significantly improves the model's estimation unbiasedness, stability, and generalization ability in cross-scenario cold-start tasks, thus providing efficient and low-cost intelligent decision support for personalized marketing and rapid promotion of new products without the need for historical training data of the target scenario.

[0050] The following further describes specific embodiments and experimental verifications of the present invention.

[0051] The zero-sample marketing gain modeling scheme of this invention is specifically implemented as a counterfactual cue gain modeling method (named PromptUplift) and system based on a large language model. By converting structured user features and marketing scenario descriptions into natural language cue, the prior knowledge of consumer behavior in the large language model is activated to achieve zero-sample individual treatment effect estimation.

[0052] The overall architecture of the system is as follows Figure 3 As shown, the marketing gain modeling system provided mainly includes an input layer, a core processing layer, and an output layer.

[0053] (1) The input layer includes a user feature access module, a scenario description access module and a marketing activity (intervention description) access module, which are used to receive or obtain the structured feature data of the user to be analyzed, the background text description of the current marketing scenario and the text description of specific marketing intervention measures (such as promotional activities), respectively.

[0054] (2) The core processing layer includes, in sequence: prompting construction module, semantic encoding module, representation alignment module (optional), and counterfactual prediction module. These modules will be introduced in detail later.

[0055] (3) The output layer includes an individual gain calculation module and a result analysis module. The gain calculation module calculates the difference between the two potential outcome values ​​output by the counterfactual prediction module as an estimate of the individual treatment effect and outputs it. The result analysis module can be used to output statistical indicators such as the model AUUC during the model testing phase.

[0056] The natural language prompt construction mechanism forms the starting point of the processing: the structured user features and text descriptions of target marketing interventions received by the input layer are transformed into unified natural language prompts containing explicit counterfactual task instructions in the prompt construction module of the core processing layer, thus completing the transformation from numerical features to semantic descriptions and laying a structured input foundation for the subsequent activation of the prior knowledge of the large language model.

[0057] Subsequently, the large language model acts as a semantic encoder: the semantic encoding module uses the pre-trained large language model to deeply encode the aforementioned natural language prompts, extracting a joint representation vector that integrates user semantics, intervention semantics, and task instructions. This step activates the model's inherent common sense about consumer behavior and is the core of achieving zero-shot semantic reasoning. To further improve the robustness of the estimation, a representation distribution alignment mechanism can be introduced as an optional enhancement during the training phase: the representation alignment module explicitly aligns the distributions of the intervention group and the control group in the joint representation space using a distribution distance metric (such as maximum mean difference), thereby mitigating selection bias in the data and improving the unbiasedness and cross-scenario stability of the model estimation.

[0058] Based on the learned joint representation, the two-branch counterfactual prediction architecture performs key computations: the counterfactual prediction module designs a parameter-discretionary two-branch prediction network that shares the aforementioned joint representation vector as input and predicts the potential outcomes for users under intervention and without intervention, thus directly supporting the explicit computation of individual treatment effects. Finally, in the output layer, the gain calculation module calculates the difference between the predicted values ​​of the two branches as the output individual gain estimate, while the results analysis module provides a quantitative evaluation of the model's performance.

[0059] By organically integrating the key mechanisms mentioned above, such as cue construction, semantic encoding, representation alignment, and bi-branch prediction, a complete end-to-end zero-shot gain modeling system has been formed. This system supports the entire process from raw data input to individual processing effect estimation output, achieving zero-shot generalization capability for novel marketing intervention scenarios.

[0060] The implementation method corresponds to the system architecture described above, and specifically includes the following parts (see...). Figure 4 ): (1) Prompt-based semantic modeling module This module is used to transform structured user characteristics, marketing scenario descriptions, and counterfactual task instructions into a unified sequence of natural language prompts. The constructed triple inputs include:

[0061] d: Marketing scenario context, describing the marketing background; u: User profiles and behavioral history transform structured features (such as purchase frequency, channel preference, etc.) into natural language descriptions; s: Counterfactual task instruction, explicitly requiring the model to estimate the potential outcomes for users in different marketing scenarios. Example prompt template: This user is a customer of an online retailer specializing in clothing and accessories. This user made a purchase within the last month. This customer has purchased x items in the past and resides in district xx. Task: Estimate this user's behavior in the following two marketing scenarios. Expected spending amount (USD): (A) Do not send email marketing to this user. (B) Send this user promotions targeting women. mail."

[0062] (2) Large Language Model Semantic Encoding Module A pre-trained large language model (such as Qwen3-0.6B) is used as a semantic encoder to encode the above prompt text: 1. Lexicalization and Embedding: The prompt text is converted into a sequence of lexical characters by a lexicalizer and then input into the embedding layer of the large language model.

[0063] 2. Efficient parameter fine-tuning: The model is fine-tuned using low-rank adaptive techniques, which only update the low-rank matrix of some attention projection layers, thus preserving the original semantic knowledge of the model.

[0064] 3. Semantic representation extraction: Obtain the semantic representation vector output by the last Transformer layer through average pooling.

[0065] (3) Two-branch counterfactual prediction and representation correction alignment module 1. Two-branch counterfactual prediction head: Two independent multilayer perceptrons were set up, corresponding to the intervention group and the control group in the marketing scenario, respectively. The input is the semantic representation h, and the outputs are respectively (Expected outcomes under intervention) and (Expected outcome without intervention); The estimated value of the individual treatment effect is: .

[0066] 2. Represents the correction module: The maximum mean difference was used as a distribution alignment metric to calculate the distribution distance between the intervention group and the control group in the semantic representation space; By optimizing through backpropagation, the two sets of representations are made to converge, satisfying the strong ignorability assumption.

[0067] (4) Training methods and loss functions The total loss function is:

[0068] To predict losses based on facts, supervision is conducted using only observed actual results; To represent the balancing loss, MMD is used; This is a balance coefficient used to control the intensity of the alignment.

[0069] Experimental verification The experimental example provides a specific and complete implementation flow of the method described in this invention, covering the entire process from data preparation, model building, training to inference. This example uses the open-source large language model Qwen3-0.6B as the semantic encoding backbone and the publicly available Hillstrom email marketing dataset as the experimental benchmark, detailing how to implement a gain modeling system for historical data without target intervention.

[0070] First, prepare the environment and preprocess the data. If you need to fine-tune a large model, the hardware needs to be configured with a GPU with sufficient video memory, and the software environment includes Python 3.9 or above, PyTorch 2.0 or above framework, Transformers library, and load the Qwen3-0.6B model weights.

[0071] The experimental data used the Hillstrom email marketing dataset, which originated from a real A / B test and included 64,000 users randomly assigned to three groups: "No Email," "Men's Product Email," and "Women's Product Email." To construct the task for this embodiment, a binary intervention comparison was focused on, and the specific sample composition is shown in Table 1.

[0072] Table 1: Statistical Information of Experimental Groups

[0073] The experimental data used the Hillstrom email marketing dataset, from which samples of approximately 42,613 user records were extracted, representing the "men's email" intervention group and the "no email" control group. Each record contained 10 structured feature fields, including "number of months since last purchase," "historical spending amount," and "shopping channel," as well as the target variable "spending amount in the next 30 days." The core of the data processing was to transform these structured features into natural language description fragments and embed them into a pre-defined prompt template.

[0074] This template integrates user profiles, intervention scenario descriptions, and explicit counterfactual task instructions. A specific example of the prompt template is as follows: "This user is a customer of an online retailer specializing in clothing and accessories. This user's most recent purchase was {recency} months ago. This user's total historical spending is {history} dollars, belonging to the {history_segment} value segment. This user {has purchased men's products?}. This user's residential area is {zip_code}. This user {new user status description}. This user's primary shopping channel is {channel}. Task: Please predict this user's expected spending amount (in dollars) in the following two marketing scenarios: Scenario (A): No email marketing campaigns will be sent to this user. Scenario (B): Send this user promotional emails targeting men's products." The content within the curly braces is dynamically filled based on the original feature values. For example, if the feature value for "has purchased men's products?" is yes, then it will be filled with "has purchased men's products in the past."

[0075] In terms of model architecture, the semantic encoding module uses a pre-trained Qwen3-0.6B model. To efficiently adapt to downstream tasks, a low-rank adaptive technique is used to fine-tune the model. Specifically, LoRA adapters are added only to the query projection matrix and value projection matrix in the last two Transformer layers of the model, with a rank of 2 and a scaling factor of 4. After inputting the prompt text into the model, a 1024-dimensional semantic representation vector is obtained by average pooling the output of the last layer. To mitigate selection bias, an optional representation distribution alignment module is introduced during the training phase. This module uses the maximum mean difference to measure the distribution distance between the intervention group and the control group in the representation space. The counterfactual prediction module consists of two parameter-independent multilayer perceptrons. The semantic representation vector is simultaneously input into the two towers, and the prediction results for the user under intervention and without intervention are output respectively. The training process adopts end-to-end joint optimization. The total loss function is the sum of the fact prediction loss and the representation balance loss, with the balance loss weight set to 1. The AdamW optimizer was used for training, with an initial learning rate of 1e-3, a batch size of 256, and a total of 20 training epochs, employing an early stopping strategy.

[0076] To comprehensively evaluate this method, comparative experiments were conducted with mainstream baseline models, and key ablation experiments were also performed. The baseline models compared included classic meta-learners (S-learner, T-learner), tree-based methods (CausalForest), and deep causal models based on neural networks (TarNet, CFRNet). All experiments were repeated under four different random seeds, and the results are reported as mean ± standard deviation. Table 2 shows the performance comparison in the same domain test scenario of "Men's Emails" (i.e., the model was trained and tested on the "Men's Emails" intervention).

[0077] Table 2: Comparison of generalization performance in the same domain test scenario (mean ± standard deviation of training on male emails and testing on male emails)

[0078] As shown in Table 2, in the same domain test scenario, compared with the traditional model, the present invention achieves the best performance in the three indicators of AUUC, AUQC and Lift@30% and the standard deviation of the core evaluation indicator AUUC is significantly lower than that of most baseline models, indicating that its estimation results are not only more accurate but also more stable, which shows that the present method has a stronger ability to identify high-gain users.

[0079] Table 3 illustrates the core value of this method—its performance in a cross-intervention zero-sample cold-start scenario. This test simulates one of the most challenging real-world business scenarios: where only the results of a marketing campaign are available in the historical samples, but model gain prediction is required for a new marketing campaign. In other words, the model is trained only on "men's emails" data, but must directly estimate the individual treatment effects of a previously unseen "women's emails" intervention.

[0080] Table 3: Comparison of cold start generalization performance (mean ± standard deviation of training on male emails and testing on female emails)

[0081] In cross-intervention cold-start scenarios, traditional methods (such as single / dual learners and causal forests) show significant performance degradation, even exhibiting negative lift (e.g., Lift@30% = -0.2897 for single learner), indicating their inability to effectively generalize to new intervention types. In contrast, the PromptUplift model proposed in this invention not only surpasses all baselines in AUUC (1.0009) and AUQC (1.0153), but also achieves a Lift@30% of 0.7321, an improvement of approximately 11.2% over the best baseline CFRNet (0.6581). Furthermore, it further improves performance and stability on the ablation version (without MMD alignment). This fully validates that by activating the consumer behavior common sense inherent in the large language model through natural language prompting engineering, and combining a dual-branch counterfactual prediction and representation distribution alignment mechanism, it is possible to effectively estimate the treatment effect of zero-sample individuals without marketing interventions, fundamentally breaking through the cold-start bottleneck of traditional gain models that rely on historical intervention data.

[0082] In summary, this invention proposes a zero-shot marketing gain modeling method and system based on a large language model. The important innovative contributions and outstanding features of this invention include:

[0083] 1. A proposed implementation path for transforming causal inference tasks into natural language prompting tasks is presented. By constructing a unified natural language prompt containing counterfactual task instructions, structured user features and marketing intervention descriptions are semantically fused, providing a structured input foundation for activating the prior knowledge of large language models.

[0084] 2. Innovatively, a pre-trained large language model is used as a semantic encoder to extract a joint representation vector that integrates user, intervention, and task semantics, thereby activating the consumer behavior common sense inherent in the model and achieving zero-shot semantic reasoning.

[0085] 3. A two-branch counterfactual prediction architecture with non-shared parameters was designed, which predicts the potential outcomes under intervention and no intervention based on the joint representation vector, directly supporting the explicit calculation of individual treatment effects.

[0086] 4. An optional representation distribution alignment mechanism (such as based on the maximum mean difference) is introduced to explicitly alleviate selection bias by aligning the distributions of the intervention group and the control group in the representation space, thereby improving the unbiasedness and cross-scenario stability of the estimation.

[0087] 5. An end-to-end zero-sample gain modeling system was constructed, integrating the above modules to form a complete solution for estimating the individual treatment effect of new marketing scenarios without the need for target intervention on historical samples.

[0088] Compared with the prior art, the significant technical advantages of the present invention are reflected in the following aspects: Existing gain modeling methods (such as meta-learners, decision trees, and neural networks) heavily rely on supervised training using historical intervention-response data, inherently failing to handle unseen intervention types and lacking zero-shot generalization ability across interventions. This invention overcomes the dependence on historical samples by "transforming causal inference tasks into natural language understanding tasks," leveraging the powerful semantic priors and contextual reasoning capabilities of large language models. Specifically, by converting structured features and intervention descriptions into natural language cues, the large language model is guided to understand user profiles, intervention content, and counterfactual task instructions in the semantic space, thereby activating its pre-trained consumer behavior common sense and enabling reasonable causal inference for novel intervention scenarios.

[0089] Experiments show that in cross-intervention cold start scenarios, our proposed method significantly outperforms the existing best baseline model CFRNet (AUUC: 1.0009 vs. 0.9174; Lift@30%: 0.7321 vs. 0.6581) on key metrics such as AUUC and Lift@30%, validating its effectiveness and superiority in estimating the effects of zero-sample individual treatments. Detailed experimental data demonstrate that the proposed method not only performs excellently in conventional intra-domain tests but also achieves a significant performance breakthrough in the critical scenario of cross-intervention cold start. Its core lies in transforming the causal inference task into a semantic reasoning task that can be processed by a large language model through natural language prompting engineering. This effectively activates the consumer behavior common sense inherent in the large language model, thereby achieving reasonable generalization to unseen marketing interventions and providing a practical and effective solution to real-world business pain points such as marketing cold start and new product promotion.

[0090] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0091] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0092] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0093] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0094] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0095] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0096] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0097] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0098] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0099] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0100] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0101] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0102] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. A zero-shot marketing gain modeling method based on a large language model, characterized in that, Includes the following steps: S1. Prompt Construction: Based on the structured characteristics of the target user and the text description of the target marketing intervention, construct natural language prompts containing counterfactual task instructions; S2. Semantic Encoding and Representation Learning: Encode the natural language prompts using a pre-trained large language model and extract a joint representation vector that integrates user semantics and intervention semantics; S3. Counterfactual prediction: Input the joint representation vector into a two-branch prediction network. The first branch of the network outputs the first potential outcome prediction value when the user receives the target marketing intervention, and the second branch outputs the second potential outcome prediction value when the user does not receive any intervention. S4. Gain Calculation and Output: Calculate the difference between the first potential outcome prediction value and the second potential outcome prediction value, and output it as the individual treatment effect estimate of the user for the target marketing intervention.

2. The zero-shot marketing gain modeling method based on a large language model as described in claim 1, characterized in that, In step S1, constructing the natural language prompt containing counterfactual task instructions specifically includes: Transform the numerical and categorical structured features of the target user into descriptive natural language fragments; Use the textual description of the targeted marketing intervention as the context of the intervention; The descriptive natural language fragments, the intervention context, and the counterfactual task instructions that explicitly require the model to estimate the potential outcomes for the user under intervention and without intervention are combined to form a complete natural language prompt text.

3. The zero-shot marketing gain modeling method based on a large language model as described in claim 1, characterized in that, In step S2, when encoding using a pre-trained large language model, a parameter-efficient fine-tuning technique is employed to adapt the large language model.

4. The zero-shot marketing gain modeling method based on a large language model as described in claim 1, characterized in that, In step S2, extracting the joint representation vector that integrates user semantics and intervention semantics specifically includes: The sequence representation output by the large language model is pooled to generate the joint representation vector of fixed dimensions.

5. The zero-shot marketing gain modeling method based on a large language model as described in claim 1, characterized in that, Between step S2 and step S3, there is also step S2a, which indicates distribution alignment: During model training, based on the joint representation vector extracted from the large language model, the distribution distance between the intervention group samples and the control group samples in the representation space is calculated, and this distribution distance is added as a regularization term to the loss function for optimization, so as to reduce the distribution difference between the two groups.

6. The zero-shot marketing gain modeling method based on a large language model as described in claim 5, characterized in that: The representation distribution alignment is achieved by optimizing a joint loss function, which includes a fact prediction loss term and a representation balance loss term. The fact prediction loss term is calculated solely based on the actual observed results of each sample and its corresponding branch prediction value; The representation balance loss term is calculated based on a selected distribution distance metric and is used to make the representation distributions of the intervention group and the control group converge.

7. The zero-shot marketing gain modeling method based on a large language model as described in claim 6, characterized in that, The distribution distance metric used in the balance loss term is the maximum mean difference.

8. The zero-shot marketing gain modeling method based on a large language model as described in claim 1, characterized in that, In step S3, the two branches of the dual-branch prediction network are neural networks with non-shared parameters, and share the joint representation vector extracted in step S2 as input; The first branch is used to receive the joint representation vector and output a first potential outcome prediction value, and the second branch is used to receive the joint representation vector and output a second potential outcome prediction value.

9. A zero-shot marketing gain modeling system based on a large language model, characterized in that, include: Prompt building module: Based on the structured characteristics of the target user and the text description of the target marketing intervention, construct natural language prompts containing counterfactual task instructions; Semantic encoding and representation learning module: Encodes the natural language prompts using a pre-trained large language model and extracts a joint representation vector that integrates user semantics and intervention semantics; Counterfactual prediction module: The joint representation vector is input into a two-branch prediction network. The first branch of the network outputs the first potential outcome prediction value when the user receives the target marketing intervention, and the second branch outputs the second potential outcome prediction value when the user does not receive any intervention. Gain Calculation and Output Module: Calculates the difference between the first potential outcome prediction value and the second potential outcome prediction value, and outputs it as an estimate of the individual treatment effect of the user on the target marketing intervention.

10. The zero-shot marketing gain modeling method based on a large language model as described in claim 9, characterized in that, Also includes: Distribution alignment module: During model training, based on the joint representation vector extracted from the large language model, the distribution distance between the intervention group samples and the control group samples in the representation space is calculated, and this distribution distance is added as a regularization term to the loss function for optimization, so as to reduce the distribution difference between the two groups.

Images