A real-time data analysis method and system applied to a marketing scenario

Abstract

The application discloses a kind of real-time data analysis method and system applied to marketing scene, it is related to data processing technical field.The method includes: collecting real-time marketing event stream to generate real-time alignment feature vector;It is input baseline conversion model and marketing response model to calculate marketing effective increment and predicted confidence;Real-time monitoring budget and inventory to calculate comprehensive resource constraint coefficient;The marketing effective increment, the predicted confidence and the comprehensive resource constraint coefficient are input to decision function to calculate expected net income, and marketing action is issued or intercepted accordingly;Collect feedback result to calculate weighted gradient, and update marketing response model weight online.The application solves the problems of the prior art, such as inaccurate marketing decision, lack of dynamic adjustment of resource constraints, and inability of model to update online.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a real-time data analysis method and system for marketing scenarios. Background Technology

[0002] With the development of e-commerce, marketing activities (such as coupons, discounts, and push notifications) have become important means for businesses to acquire customers and improve conversion rates. However, in actual marketing scenarios, how to accurately identify users sensitive to marketing interventions and dynamically make optimal marketing decisions under the constraints of limited budgets and inventory resources is a technical problem that urgently needs to be solved.

[0003] Traditional marketing decision-making systems typically rely on offline-trained conversion rate prediction models, which have the following drawbacks: First, offline models cannot reflect the dynamic changes in user interests and the market environment in a timely manner; second, feature processing often fails to effectively align and discretize real-time continuous features, resulting in weak model generalization ability; third, marketing decisions often focus only on conversion rates, ignoring effective incremental marketing (i.e., additional conversion improvement brought about by intervention), easily wasting marketing resources on users who would naturally convert; furthermore, existing systems lack comprehensive consideration of real-time business status such as budget consumption rate and remaining inventory when making decisions, easily leading to budget overruns or overselling of inventory; finally, after the model runs online, it cannot perform real-time online learning and adaptive updates based on real user feedback, causing model performance to degrade over time.

[0004] Therefore, there is an urgent need for a real-time data analysis method and system that can integrate real-time feature alignment, incremental prediction, resource-constrained dynamic decision-making, and online fine-tuning. Summary of the Invention

[0005] This invention provides a real-time data analysis method and system for marketing scenarios. This invention solves the problems of insufficient accuracy in marketing decisions, lack of dynamic adjustment due to resource constraints, and the inability of models to update adaptively online in existing technologies.

[0006] In a first aspect, embodiments of the present invention provide a real-time data analysis method applied to marketing scenarios, the method comprising: Collect real-time marketing event streams, calculate real-time continuous features of users based on sliding windows, query the pre-stored offline quantile boundary table to perform bucket discretization mapping on the real-time continuous features, and generate real-time aligned feature vectors after splicing. The real-time aligned feature vectors are input into the baseline conversion model and the marketing response model respectively to generate natural conversion probability and intervention conversion probability, and the effective marketing increment and prediction confidence are calculated based on the difference between the two. Monitor changes in the business database in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient; The effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient are input into the decision function to calculate the expected net revenue. If the expected net revenue is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is blocked. The system collects feedback results corresponding to marketing decision actions, calculates weighted gradients, updates the weight vector of the marketing response model using an online learning algorithm, and uses the updated weight vector for the next real-time data analysis.

[0007] The technical solution provided in this application has at least the following beneficial effects: This application abandons the simple conversion rate-based prediction method and adopts a T-Learner dual-model framework to construct a baseline conversion model and a marketing response model. The difference between the two is used to obtain the true effective marketing increment. A maximum value function is used to eliminate negative incremental users (whose conversion rate was actually reduced by coupon issuance). Prediction confidence is introduced to calibrate the incremental prediction results, accurately identifying truly sensitive "convertible" users who are sensitive to marketing intervention. This avoids the waste of marketing resources allocated to users who would naturally convert, significantly improving the accuracy of marketing decisions and resource utilization. By monitoring changes in the business database in real time and dynamically acquiring budget consumption rate, remaining budget ratio, and product inventory, a comprehensive resource constraint coefficient including budget constraint factors and inventory constraint factors is innovatively constructed. When the budget is tight or consumption surges, the budget constraint factor smoothly decays, reducing the probability of marketing actions being issued. When inventory is insufficient, the inventory constraint factor forcibly blocks marketing actions. By integrating the comprehensive resource constraint coefficient into the expected net revenue decision function, the actual... It features automatic degradation and circuit breaker mechanisms during resource shortages, effectively avoiding budget overruns and overselling risks, ensuring the smooth operation of the business system and maximizing profits. Breaking the limitations of poor timeliness in traditional offline models, it constructs an online learning closed loop of "decision-feedback-update." After obtaining real conversion tags, it calculates incremental sensitive weights, imposing greater penalties on prediction errors of high-marketing incremental groups during gradient descent. An online stochastic gradient descent algorithm is used to incrementally update and hot-load the marketing response model. This mechanism enables the model to adaptively evolve in real time as user interests shift and the market environment changes, prioritizing the correction of prediction biases for high-value groups, thereby maintaining and continuously improving the model's online prediction performance over the long term. By calculating real-time continuous features through a sliding window and querying a pre-stored offline quantile boundary table for bucket discretization mapping, it ensures the consistency between the online real-time feature distribution and the offline training feature distribution, effectively eliminating feature crossing and distribution shift problems, further improving the stability and reliability of real-time prediction results.

[0008] In one optional implementation, real-time marketing event streams are collected, real-time continuous features of users are calculated based on a sliding window, a pre-stored offline quantile boundary table is queried to perform bucket discretization mapping on the real-time continuous features, and the resulting concatenation generates a real-time aligned feature vector, including: Collect real-time marketing event streams and define each marketing event in the real-time marketing event stream as a tuple, wherein the tuple includes user identifier, behavior type, context information and event timestamp; Based on user identifiers, the real-time marketing event stream is grouped and a sliding time window is assigned to extract the set of events that meet the time conditions at the current moment; For the event set, an aggregation function is used to calculate the real-time continuous features of each dimension, forming a real-time continuous feature vector; For each dimension of the real-time continuous feature vector, query the pre-stored offline quantile boundary table to obtain the corresponding offline stage quantile boundary array calculated based on the historical full data. Map the real-time continuous features of each dimension to the bucket intervals corresponding to the quantile boundary array obtained from the query, calculate the discrete bucket index, and form a discrete bucket index vector. One-hot encoding is performed on each discrete bucket index in the discrete bucket index vector to generate a sparse representation vector of the real-time continuous features in each dimension. The vectors are then concatenated along the dimensions to generate the final real-time aligned feature vector.

[0009] In one optional implementation, the baseline conversion model and marketing response model are built based on the T-Learner dual-model framework, and the specific construction methods include: The user state without marketing intervention was used as the control group, and the user state with specific marketing intervention was used as the experimental group. Two response function models were trained independently for each group. One of the response function models is trained offline using control group sample data from historical A / B experiments to obtain a baseline conversion model. The base learner of the baseline conversion model uses logistic regression or a shallow fully connected neural network to output a stable natural conversion probability. Another response function model is trained offline using experimental group sample data from historical A / B experiments to obtain a marketing response model. The base learner of the marketing response model adopts logistic regression or a neural network containing differentiable output layer weights to support incremental updates of its output layer weights using an online gradient descent algorithm.

[0010] In one optional implementation, the real-time aligned feature vector is input into the baseline conversion model and the marketing response model, respectively, to generate the natural conversion probability and the intervention conversion probability. Based on the difference between the two, the effective marketing increment and prediction confidence are calculated, including: The real-time aligned feature vectors are input into the baseline conversion model to generate natural conversion probabilities; The real-time aligned feature vectors are input into the marketing response model to generate intervention conversion probabilities. Calculate the difference between the natural conversion probability and the intervention conversion probability as the estimated value of marketing increment; The effectiveness of the marketing increment estimate is verified, and the effective marketing increment is calculated to remove negative incremental users that actually reduce the conversion rate when coupons are issued; Based on the natural conversion probability and the intervention conversion probability, a prediction variance index is defined, and the prediction confidence level is calculated.

[0011] In one optional implementation, real-time monitoring of changes in the business database is performed to obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, as well as the inventory levels of related products. The overall resource constraint coefficient is then calculated, including: By using the stream computing engine to monitor the change logs of the business database in real time, the current business status parameters can be extracted. Calculate the budget constraint factor and inventory constraint factor based on business status parameters; The budget constraint factor and the inventory constraint factor are multiplied and fused to generate the comprehensive resource constraint coefficient for the current moment.

[0012] In one optional implementation, the business status parameters include budget data and inventory data; The budget data includes the total marketing campaign budget, the budget already spent, and the budget spent within a preset time window (such as the most recent 10 minutes). The inventory data includes the current inventory of related products and the system's preset safety stock threshold.

[0013] In one optional implementation, the budget constraint factor and inventory constraint factor are calculated based on business status parameters, including: Based on the budget data in the business status parameters, calculate the current remaining budget percentage and the current budget consumption rate; Based on the current remaining budget percentage and the current budget consumption rate, a standardized indicator of budget consumption rate is introduced to calculate budget pressure. The Sigmoid function is then used to map the budget pressure to the (0,1] interval to calculate the budget constraint factor. Calculate the inventory constraint factor based on the inventory data in the business status parameters.

[0014] In one optional implementation, the effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient are input into the decision function to calculate the expected net revenue. If the expected net revenue is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is intercepted, including: The business revenue parameters associated with the current marketing campaign can be queried in real time from the marketing campaign configuration center. These business revenue parameters include the profit per transaction of the target product and the execution cost of the current single marketing decision action. The effective marketing increment, the prediction confidence level, the comprehensive resource constraint coefficient, and the business revenue parameters are input into the decision function to calculate the expected net revenue. If the expected net profit is greater than 0, a marketing decision action is generated and issued, and the context information of this decision is encapsulated as a context record and saved asynchronously; otherwise, the marketing decision action is intercepted.

[0015] In one optional implementation, feedback results corresponding to marketing decision actions are collected, a weighted gradient is calculated, an online learning algorithm is used to update the weight vector of the marketing response model, and the updated weight vector is used for the next real-time data analysis, including: At the end of the preset observation window after the marketing decision action is issued, the user's transaction conversion behavior is monitored through the stream computing engine to obtain the real conversion tag corresponding to the marketing decision action request at the current moment as a feedback result. Extract the real-time aligned feature vector, intervention conversion probability, and effective marketing increment corresponding to the marketing decision action from the asynchronously saved context record; Based on the effective incremental marketing data, the incremental sensitive weight is calculated, and the weighted gradient is calculated based on the cross-entropy loss function of logistic regression, the real conversion label, the real-time aligned feature vector, the intervention conversion probability, and the incremental sensitive weight. The marketing response model's weight vector is incrementally updated using an online stochastic gradient descent algorithm combined with weighted gradients. The updated weight vector is then hot-loaded through the model parameter center for real-time data analysis in the next cycle.

[0016] Secondly, embodiments of the present invention provide a real-time data analysis system applied to marketing scenarios, used to implement a real-time data analysis method, the system comprising: The feature alignment unit is used to collect real-time marketing event streams, calculate the real-time continuous features of users based on a sliding window, query the pre-stored offline quantile boundary table to perform bucket discretization mapping on the real-time continuous features, and generate a real-time aligned feature vector after splicing. The incremental prediction unit is used to input the real-time aligned feature vector into the baseline conversion model and the marketing response model respectively, generate the natural conversion probability and the intervention conversion probability, and calculate the effective marketing increment and prediction confidence based on the difference between the two. The resource constraint unit is used to monitor changes in the business database in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient. The dynamic decision-making unit is used to input the effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient into the decision function to calculate the expected net revenue. If it is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is intercepted. The online fine-tuning unit is used to collect feedback results corresponding to marketing decision actions, calculate weighted gradients, update the weight vector of the marketing response model using an online learning algorithm, and use the updated weight vector for the next real-time data analysis.

[0017] A third aspect of this invention provides an electronic device, which includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, such that the at least one processor can perform the method proposed in the first aspect of the present invention.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first aspect of the present invention. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the steps of a real-time data analysis method applied to a marketing scenario, as provided in an embodiment of the present invention. Figure 3 This is a functional unit diagram of a real-time data analysis system applied to marketing scenarios, provided by an embodiment of the present invention. Detailed Implementation

[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] The present invention will be further described below with reference to the accompanying drawings.

[0022] Reference Figure 1 , Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0023] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0024] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0025] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and an electronic program for a real-time data analysis system applied to marketing scenarios.

[0026] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device. The electronic device calls the electronic program of the real-time data analysis system for marketing scenarios stored in the memory 1005 through the processor 1001, and executes the real-time data analysis method for marketing scenarios provided in the embodiments of the present invention.

[0027] Reference Figure 2 The present invention provides a real-time data analysis method for marketing scenarios, the method comprising: S201: Collect real-time marketing event streams, calculate real-time continuous features of users based on sliding windows, query the pre-stored offline quantile boundary table to perform bucket discretization mapping on the real-time continuous features, and generate real-time aligned feature vectors after splicing. S202: Input the real-time aligned feature vector into the baseline conversion model and the marketing response model respectively to generate the natural conversion probability and the intervention conversion probability, and calculate the effective marketing increment and prediction confidence based on the difference between the two. S203: Monitor changes in the business database in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient. S204: Input the effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient into the decision function to calculate the expected net revenue. If it is greater than 0, generate and issue a marketing decision action; otherwise, intercept the marketing decision action. S205: Collect feedback results corresponding to marketing decision actions, calculate weighted gradients, update the weight vector of the marketing response model using an online learning algorithm, and use the updated weight vector for the next real-time data analysis.

[0028] The technical solution provided in this application has at least the following beneficial effects: This application abandons the simple conversion rate-based prediction method and adopts a T-Learner dual-model framework to construct a baseline conversion model and a marketing response model. The difference between the two is used to obtain the true effective marketing increment. A maximum value function is used to eliminate negative incremental users (whose conversion rate was actually reduced by coupon issuance). Prediction confidence is introduced to calibrate the incremental prediction results, accurately identifying truly sensitive "convertible" users who are sensitive to marketing intervention. This avoids the waste of marketing resources allocated to users who would naturally convert, significantly improving the accuracy of marketing decisions and resource utilization. By monitoring changes in the business database in real time and dynamically acquiring budget consumption rate, remaining budget ratio, and product inventory, a comprehensive resource constraint coefficient including budget constraint factors and inventory constraint factors is innovatively constructed. When the budget is tight or consumption surges, the budget constraint factor smoothly decays, reducing the probability of marketing actions being issued. When inventory is insufficient, the inventory constraint factor forcibly blocks marketing actions. By integrating the comprehensive resource constraint coefficient into the expected net revenue decision function, the actual... It features automatic degradation and circuit breaker mechanisms during resource shortages, effectively avoiding budget overruns and overselling risks, ensuring the smooth operation of the business system and maximizing profits. Breaking the limitations of poor timeliness in traditional offline models, it constructs an online learning closed loop of "decision-feedback-update." After obtaining real conversion tags, it calculates incremental sensitive weights, imposing greater penalties on prediction errors of high-marketing incremental groups during gradient descent. An online stochastic gradient descent algorithm is used to incrementally update and hot-load the marketing response model. This mechanism enables the model to adaptively evolve in real time as user interests shift and the market environment changes, prioritizing the correction of prediction biases for high-value groups, thereby maintaining and continuously improving the model's online prediction performance over the long term. By calculating real-time continuous features through a sliding window and querying a pre-stored offline quantile boundary table for bucket discretization mapping, it ensures the consistency between the online real-time feature distribution and the offline training feature distribution, effectively eliminating feature crossing and distribution shift problems, further improving the stability and reliability of real-time prediction results.

[0029] In one optional implementation, real-time marketing event streams are collected, real-time continuous features of users are calculated based on a sliding window, a pre-stored offline quantile boundary table is queried to perform bucket discretization mapping on the real-time continuous features, and the resulting concatenation generates a real-time aligned feature vector, including: S2011: Collect real-time marketing event streams and define each marketing event in the real-time marketing event stream as a tuple, wherein the tuple includes user identifier, behavior type, context information and event timestamp; S2012: Group the real-time marketing event stream according to user identifiers, allocate sliding time windows, and extract the set of events that meet the time conditions at the current moment. The formula for the time conditions is: In the formula, For the first i User's event timestamps; The current moment; This is the size of the sliding time window; i Index for users; S2013: For the aforementioned event set, use an aggregation function to calculate the real-time continuous features for each dimension, forming a real-time continuous feature vector, as shown in the formula: In the formula, For the first t A collection of events at a given moment In the j Real-time continuous features of dimensions; For the first t A collection of events at any given moment; For aggregation operators; j For dimension indexing; t For time indexing; S2014: For each dimension of the real-time continuous feature vector, query the pre-stored offline quantile boundary table to obtain the corresponding offline stage quantile boundary array calculated based on the historical full data. S2015: Map the real-time continuous features of each dimension to the bucket intervals corresponding to the quantile boundary array obtained from the query, calculate the discrete bucket index, and form a discrete bucket index vector. S2016: Perform one-hot encoding on each discrete bucket index in the discrete bucket index vector to generate a sparse representation vector of the real-time continuous features in each dimension, and concatenate them according to the dimensions to generate the final real-time aligned feature vector.

[0030] In one optional implementation, the baseline conversion model and marketing response model are built based on the T-Learner dual-model framework, and the specific construction methods include: The user state without marketing intervention was used as the control group, and the user state with specific marketing intervention was used as the experimental group. Two response function models were trained independently for each group. One of the response function models is trained offline using control group sample data from historical A / B experiments to obtain a baseline conversion model. The base learner of the baseline conversion model uses logistic regression or a shallow fully connected neural network to output a stable natural conversion probability. Another response function model is trained offline using experimental group sample data from historical A / B experiments to obtain a marketing response model. The base learner of the marketing response model adopts logistic regression or a neural network containing differentiable output layer weights to support incremental updates of its output layer weights using an online gradient descent algorithm.

[0031] In this embodiment, taking the coupon distribution marketing scenario of an e-commerce platform as an example, the model construction and offline training mechanism based on the T-Learner dual-model framework is explained in detail. Historical A / B experiment data preparation and randomization: To build an accurate causal inference model, the system needs to obtain sample data generated through rigorous randomized A / B experiments in historical marketing campaigns. In these historical campaigns, the system randomly distributed all users. Control group (no intervention): This group of users browsed and made purchases in a natural state and did not receive any marketing intervention (such as no coupons). Experimental group (with specific intervention): This group of users received specific marketing interventions during the browsing process (such as receiving discount coupons). Extract the feature data of two sets of users (such as user profile features, historical behavior features, real-time conversation features, etc., denoted as the feature matrix) and the actual conversion tag of whether or not a purchase was made (denoted as the feature matrix). Y (∈{0,1}); Since the split is random, the latent feature distributions of the two groups of users tend to be consistent, eliminating selection bias and laying the foundation for subsequent calculation of causal effects; Construction and offline training of the baseline conversion model for the control group: The baseline conversion model aims to learn the natural conversion patterns of users when they are not subject to marketing intervention.

[0032] Data input: Use control group sample data; Model selection and construction: The base learner of the baseline transformation model adopts logistic regression or shallow fully connected neural network (such as a multilayer perceptron with only 1-2 hidden layers). The reason for choosing such simpler models is that the natural transformation law is relatively stable, and simple models have stronger generalization ability and anti-overfitting ability, and can output a smoother and more robust natural transformation probability baseline. Offline training: Using cross-entropy as the loss function, multiple rounds of iterative training are performed on the control group data through the backpropagation algorithm until the model converges. After training, all network weights and biases of the baseline conversion model are fixed and deployed to the online prediction node. During online operation, the parameters of the baseline conversion model remain frozen to ensure the stability of the baseline probability, serving as a reliable reference for evaluating marketing increments. Architecture design and offline training of the experimental group's marketing response model: Marketing response models aim to learn the conversion patterns of users when subjected to specific marketing interventions, and must be able to support online gradient descent incremental updates; Data input: Use experimental group sample data; Model Architecture Design: The basic learner of the marketing response model adopts logistic regression or a neural network containing differentiable output layer weights. In this embodiment, a neural network architecture of "deep feature extraction layer + differentiable linear output layer" is preferred. Specifically, the front end of the network consists of multiple fully connected layers and activation functions to extract high-order nonlinear feature representations. The end of the network is connected to a differentiable linear output layer (i.e., the output layer weight matrix and bias are independent and differentiable). The advantage of this architecture design is that during online incremental updates, the parameters of the front deep feature extraction layer can be frozen, and only the weights of the linear output layer at the end can be updated by gradient descent. This not only greatly reduces the computational overhead and latency of online learning and meets the low latency requirements of real-time data analysis, but also avoids the collapse of deep network parameters due to individual noise feedback, thus ensuring the stability of the main structure of the model. Offline training: Using the same cross-entropy loss function, the entire neural network was trained end-to-end offline on the experimental data. After the model converged, the parameters of all layers were recorded, including the parameters of the preceding layers and the weight vector of the output layer. T-Learner Dual-Model Collaborative Prediction and Incremental Calculation: The offline-trained baseline conversion model and marketing response model are deployed to the online system to form the T-Learner dual-model framework. When a real-time data analysis request arrives, the system simultaneously inputs the real-time aligned feature vector of the same user into both models: the baseline conversion model outputs the natural conversion probability; the marketing response model outputs the intervention conversion probability. Subsequently, the system calculates the difference between the two. Since the baseline conversion model provides a solid reference for natural conversion, and the marketing response model continuously approaches the real intervention conversion rate through online fine-tuning, the synergistic work of the two enables the system to accurately isolate the pure causal effect (i.e., effective marketing increment) brought about by marketing intervention.

[0033] In one optional implementation, the real-time aligned feature vector is input into the baseline conversion model and the marketing response model, respectively, to generate the natural conversion probability and the intervention conversion probability. Based on the difference between the two, the effective marketing increment and prediction confidence are calculated, including: S2021: Input the real-time aligned feature vectors into the baseline transformation model to generate the natural transformation probability, using the following formula: In the formula, For the first t The probability of natural conversion (baseline probability) of a user at any given moment without any marketing intervention (such as no coupons issued), and its value ranges from (0,1); For the baseline transformation model in the first tThe linear combination score (also known as logits) calculated at time points is essentially the unnormalized log score of a user's natural conversion. Euler's natural constant has a value of approximately 2.71828. The sigmoid activation function is a nonlinear mapping function that maps the real number field (...). Linear combination score on (∞, +∞) Smoothly compress the mapping to the probability interval (0,1); It is the bias term (scalar) of the baseline transformation model, used to shift the decision boundary of the linear model and compensate for the baseline transformation rate that is not covered by the features; This represents the weight vector of the baseline transformation model; It is the transpose symbol; For the first t Real-time aligned feature vector at time step; t For time indexing; S2022: Input the real-time aligned feature vectors into the marketing response model to generate the intervention conversion probability, using the following formula: In the formula, For the first t The conversion probability of a user at any given time under specific marketing intervention (such as issuing coupons), with a value range of (0,1); For the marketing response model in the first t The linear combination score (also known as logits) calculated at each moment is essentially the unnormalized logarithmic score of a user's conversion under marketing intervention. Euler's natural constant has a value of approximately 2.71828. The sigmoid activation function is a nonlinear mapping function that maps the real number field (...). Linear combination score on (∞, +∞) Smoothly compress the mapping to the probability interval (0,1); It is the bias term (scalar) of the marketing response model, used in the decision boundary of the marketing response model to compensate for the baseline conversion rate not covered by the features; This represents the weight vector for the marketing response model; It is the transpose symbol; For the first t Real-time aligned feature vector at time step; t For time indexing; S2023: Calculate the difference between the natural conversion probability and the intervention conversion probability as the estimated marketing increment value. The formula is: In the formula, For the first t Estimated marketing growth at any given moment; For the first t The probability of user conversion under specific marketing intervention at any given moment; For the first t The probability of a user's natural conversion at any given moment without any marketing intervention; S2024: Verify the effectiveness of the estimated marketing increment, calculate the effective marketing increment, and use this to eliminate negative incremental users who actually reduce the conversion rate due to coupon issuance. The formula is: In the formula, For the first t Effective incremental marketing at any given moment; For the first t Estimated marketing growth at any given moment; For the function that maximizes the value; if ≤0, then =0 indicates that the user should not consume marketing resources; if >0, then = This indicates that the user is a marketing-sensitive individual; S2025: Based on the natural conversion probability and the intervention conversion probability, define the prediction variance index and calculate the prediction confidence level. The formula is as follows: In the formula, For the first t The prediction confidence level at time t; its value ranges from [0.5, 1]; The predictive variance index for natural conversion probability and intervention conversion probability; For the first t The probability of user conversion under specific marketing intervention at any given moment; For the first t The probability of a user's natural conversion at any given moment without any marketing intervention; t For time indexing.

[0034] In one optional implementation, real-time monitoring of changes in the business database is performed to obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, as well as the inventory levels of related products. The overall resource constraint coefficient is then calculated, including: S2031: Real-time monitoring of change logs in the business database via the stream computing engine to extract current business status parameters; S2032: Calculate the budget constraint factor and inventory constraint factor based on business status parameters; S2033: Multiply and fuse the budget constraint factor and inventory constraint factor to generate the comprehensive resource constraint coefficient for the current moment, using the following formula: In the formula, For the first t The comprehensive resource constraint coefficient at time t is [0,1]; For inventory constraint factors; This is a budget constraint factor.

[0035] In one optional implementation, the business status parameters include budget data and inventory data; The budget data includes the total marketing campaign budget, the budget already spent, and the budget spent within a preset time window (such as the most recent 10 minutes). The inventory data includes the current inventory of related products and the system's preset safety stock threshold.

[0036] In one optional implementation, the budget constraint factor and inventory constraint factor are calculated based on business status parameters, including: S20321: Based on the budget data in the business status parameters, calculate the current remaining budget percentage and the current budget consumption rate, using the following formula: In the formula, This represents the current remaining budget percentage, with a value range of [0,1]. It is 1 when the activity has just started and has not yet been consumed, and 0 when the budget is exhausted. The total marketing campaign budget (scalar, such as 100,000 yuan) in the budget data. The consumed budget in the budget data is accumulated by monitoring changes in the business database in real time. This represents the current budget consumption rate. This refers to the budgeted expenditure amount in the budget data; The preset time window size is used to calculate the short-term consumption rate and smooth out instantaneous fluctuations; S20322: Based on the current remaining budget percentage and the current budget consumption rate, a standardized indicator of budget consumption rate is introduced to calculate budget pressure. The Sigmoid function is then used to map the budget pressure to the (0,1] interval to calculate the budget constraint factor. The formula is as follows: In the formula, The value of budget pressure is [0, +∞). The larger the value, the tighter the budget or the faster it is consumed. This represents the current remaining budget percentage; The rate penalty weight hyperparameter (scalar, e.g., 0.5) is used to adjust the contribution of consumption rate anomalies to overall stress. The larger the value, the more sensitive the system is to short-term surges in power consumption; The standardized metric for budget consumption rate is a dimensionless ratio, when... A value greater than 1 indicates that current consumption is faster than expected. A value less than 1 indicates that consumption is slower than expected; This refers to the total marketing campaign budget in the budget data; This represents the current budget consumption rate, reflecting the amount of funds consumed per unit of time. The total duration of the event (e.g., 7 days); This is a budget constraint factor, with a value range of (0,1]. It is used when the budget is sufficient and consumption is stable. Smaller Approaching 1, when the budget is about to run out or consumption surges. Increase The smooth decay approaches 0; This is a sensitivity adjustment coefficient (a scalar, such as 1.0), used to control the smoothness of the Sigmoid function curve. The larger the value, the steeper the function drops as pressure increases; It is an exponential function; S20323: Based on the inventory data in the business status parameters, calculate the inventory constraint factor to prevent overselling of goods. The formula is: In the formula, This is the inventory constraint factor, with a value range of [0,1]. ≥ At that time, the inventory was sufficient. =1, no loss to decision-making; when 0 < < When inventory enters the warning range, As inventory decreases, it decays linearly to 0, when When the value is ≤0, the inventory is sold out. =0 will forcibly block subsequent marketing activities; This refers to the current inventory level in the budget data; The safety stock threshold (number of units) in the budget data is used to trigger inventory alerts; This is a function that takes the minimum value.

[0037] In one optional implementation, the effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient are input into the decision function to calculate the expected net revenue. If the expected net revenue is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is intercepted, including: S2041: Query the business revenue parameters associated with the current marketing campaign in real time from the marketing campaign configuration center. The business revenue parameters include the profit per transaction of the target product and the execution cost of the current single marketing decision action. S2042: Input the effective marketing increment, the prediction confidence level, the comprehensive resource constraint coefficient, and the business revenue parameters into the decision function to calculate the expected net revenue, using the following formula: In the formula, For the first t The system will generate and issue a marketing intervention action only if the expected net revenue at any given time is strictly greater than 0. For the first t Effective incremental marketing at any given moment; For the first t The confidence level of the prediction at any given time; For the first t The comprehensive resource constraint coefficient at any given moment; The profit per transaction in the business revenue parameters reflects the direct economic value brought about by positive conversion; The execution costs (such as coupon redemption costs, SMS channel fees, etc.) are included in the business revenue parameters. S2043: If the expected net profit is greater than 0, generate and issue a marketing decision action, and encapsulate the context information of this decision into a context record for asynchronous saving; otherwise, intercept the marketing decision action. The formula is: In the formula, For the first t Moment-based marketing decision-making action markers; =1 indicates that a marketing intervention action (such as issuing coupons) is generated and issued. =0 indicates that an interception action is generated (i.e., no intervention is given, and the natural transformation state is preserved).

[0038] In one optional implementation, feedback results corresponding to marketing decision actions are collected, a weighted gradient is calculated, an online learning algorithm is used to update the weight vector of the marketing response model, and the updated weight vector is used for the next real-time data analysis, including: S2051: At the end of the preset observation window (e.g., 24 hours) after the marketing decision action is issued, the user's transaction conversion behavior is monitored through the stream computing engine to obtain the real conversion tag corresponding to the marketing decision action request at the current moment as the feedback result; S2052: Extract the real-time aligned feature vector, intervention conversion probability, and effective marketing increment corresponding to the marketing decision action from the asynchronously saved context record; S2053: Based on the effective incremental marketing data, calculate the incremental sensitive weights, and based on the cross-entropy loss function of logistic regression, calculate the weighted gradient according to the real conversion labels, real-time aligned feature vectors, intervention conversion probabilities, and incremental sensitive weights. The formula is as follows: In the formula, For the first t The increment-sensitive weight at time step (TMT), with a value ≥ 1, is used to adjust the weight for high-increment groups when calculating the loss and gradient. Larger prediction errors result in a greater penalty, forcing the model to prioritize correcting prediction biases for this high-value group during iterations. For the first t The probability of user conversion under specific marketing intervention at any given moment; For the first t Estimated marketing growth at any given moment; This is the hyperparameter for the incremental penalty coefficient (a scalar, such as 0.5 or 1.0), used to control the multiplier effect of the incremental effect on the loss weight; For the first t The weighted gradient after the incremental sensitive weights are amplified at each time step has the same dimension as the real-time aligned feature vector and is used to guide the direction and step size of weight updates. For the first t Real-time conversion tags; For the first t Real-time aligned feature vector at time step; The cross-entropy loss function is used for logistic regression. This represents the weight vector for the marketing response model; Let be the partial derivative of the cross-entropy loss function of logistic regression with respect to the weight vector of the marketing response model, i.e., the standard gradient; S2054: Using the online stochastic gradient descent algorithm combined with weighted gradients, the weight vector of the marketing response model is incrementally updated to obtain the updated weight vector. This updated weight vector is then hot-loaded through the model parameter center for real-time data analysis in the next cycle. The formula is as follows: In the formula, For the first t+ 1, t The weight vector of the marketing response model at any given moment; The online learning rate (a scalar, such as 0.01) is used to control the step size of a single gradient descent update. For the first t The weighted gradient after the increment-sensitive weights are amplified at each time step; t For time indexing.

[0039] This invention also provides a real-time data analysis system 300 for marketing scenarios, as described in the embodiments of the present invention. Figure 3 The system may include the following units: The feature alignment unit 301 is used to collect real-time marketing event streams, calculate the real-time continuous features of users based on a sliding window, query the pre-stored offline quantile boundary table to perform bucket discretization mapping on the real-time continuous features, and generate a real-time aligned feature vector after splicing. The incremental prediction unit 302 is used to input the real-time aligned feature vector into the baseline conversion model and the marketing response model respectively, generate the natural conversion probability and the intervention conversion probability, and calculate the effective marketing increment and prediction confidence based on the difference between the two. Resource constraint unit 303 is used to monitor changes in the business database in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient. The dynamic decision-making unit 304 is used to input the effective marketing increment, the prediction confidence level and the comprehensive resource constraint coefficient into the decision function to calculate the expected net income. If it is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is intercepted. The online fine-tuning unit 305 is used to collect feedback results corresponding to marketing decision actions, calculate weighted gradients, update the weight vector of the marketing response model using an online learning algorithm, and use the updated weight vector for the next real-time data analysis.

[0040] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; When a processor executes a program stored in memory, it implements the real-time data analysis method of the present invention applicable to marketing scenarios.

[0041] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM) or Non-Volatile Memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0042] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0043] Furthermore, to achieve the above objectives, embodiments of the present invention also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the real-time data analysis method of the present invention applied to a marketing scenario.

[0044] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable hardware devices (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0049] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A real-time data analysis method applied to marketing scenarios, characterized in that, The method includes: Collect real-time marketing event streams, calculate real-time continuous features of users based on sliding windows, query the pre-stored offline quantile boundary table to perform bucket discretization mapping on the real-time continuous features, and generate real-time aligned feature vectors after splicing. The real-time aligned feature vectors are input into the baseline conversion model and the marketing response model respectively to generate natural conversion probability and intervention conversion probability, and the effective marketing increment and prediction confidence are calculated based on the difference between the two. Monitor changes in the business database in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient; The effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient are input into the decision function to calculate the expected net revenue. If the expected net revenue is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is blocked. The system collects feedback results corresponding to marketing decision actions, calculates weighted gradients, updates the weight vector of the marketing response model using an online learning algorithm, and uses the updated weight vector for the next real-time data analysis.

2. The real-time data analysis method applied to marketing scenarios according to claim 1, characterized in that, Real-time marketing event streams are collected, and real-time continuous features of users are calculated based on a sliding window. A pre-stored offline quantile boundary table is queried to perform bucketed discretization mapping on the real-time continuous features. The resulting concatenated features generate a real-time aligned feature vector, including: Collect real-time marketing event streams and define each marketing event in the real-time marketing event stream as a tuple, wherein the tuple includes user identifier, behavior type, context information and event timestamp; Based on user identifiers, the real-time marketing event stream is grouped and a sliding time window is assigned to extract the set of events that meet the time conditions at the current moment; For the event set, an aggregation function is used to calculate the real-time continuous features of each dimension, forming a real-time continuous feature vector; For each dimension of the real-time continuous feature vector, query the pre-stored offline quantile boundary table to obtain the corresponding offline stage quantile boundary array calculated based on the historical full data. Map the real-time continuous features of each dimension to the bucket intervals corresponding to the quantile boundary array obtained from the query, calculate the discrete bucket index, and form a discrete bucket index vector. One-hot encoding is performed on each discrete bucket index in the discrete bucket index vector to generate a sparse representation vector of the real-time continuous features in each dimension. The vectors are then concatenated according to the dimensions to generate the final real-time aligned feature vector.

3. The real-time data analysis method applied to marketing scenarios according to claim 2, characterized in that, The baseline conversion model and marketing response model are built on the T-Learner dual-model framework, and the specific construction methods include: The user state without marketing intervention was used as the control group, and the user state with specific marketing intervention was used as the experimental group. Two response function models were trained independently for each group. One of the response function models is trained offline using control group sample data from historical A / B experiments to obtain a baseline conversion model. The base learner of the baseline conversion model uses logistic regression or a shallow fully connected neural network to output a stable natural conversion probability. Another response function model is trained offline using experimental group sample data from historical A / B experiments to obtain a marketing response model. The base learner of the marketing response model adopts logistic regression or a neural network containing differentiable output layer weights to support incremental updates of its output layer weights using an online gradient descent algorithm.

4. The real-time data analysis method for marketing scenarios according to claim 3, characterized in that, The real-time aligned feature vectors are input into the baseline conversion model and the marketing response model, respectively, to generate natural conversion probabilities and intervention conversion probabilities. Based on the difference between the two, the effective marketing increment and prediction confidence are calculated, including: The real-time aligned feature vectors are input into the baseline conversion model to generate natural conversion probabilities; The real-time aligned feature vectors are input into the marketing response model to generate intervention conversion probabilities. Calculate the difference between the natural conversion probability and the intervention conversion probability as the estimated value of marketing increment; The effectiveness of the marketing increment estimate is verified, and the effective marketing increment is calculated to remove negative incremental users that actually reduce the conversion rate when coupons are issued; Based on the natural conversion probability and the intervention conversion probability, a prediction variance index is defined, and the prediction confidence level is calculated.

5. The real-time data analysis method for marketing scenarios according to claim 4, characterized in that, Monitor business database changes in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient, including: By using the stream computing engine to monitor the change logs of the business database in real time, the current business status parameters can be extracted. Calculate the budget constraint factor and inventory constraint factor based on business status parameters; The budget constraint factor and the inventory constraint factor are multiplied and fused to generate the comprehensive resource constraint coefficient for the current moment.

6. The real-time data analysis method applied to marketing scenarios according to claim 5, characterized in that, The business status parameters include budget data and inventory data; The budget data includes the total marketing campaign budget, the budget already spent, and the budget spent within a preset time window (such as the most recent 10 minutes). The inventory data includes the current inventory of related products and the system's preset safety stock threshold.

7. The real-time data analysis method for marketing scenarios according to claim 6, characterized in that, Based on business status parameters, calculate budget constraint factors and inventory constraint factors, including: Based on the budget data in the business status parameters, calculate the current remaining budget percentage and the current budget consumption rate; Based on the current remaining budget percentage and the current budget consumption rate, a standardized indicator of budget consumption rate is introduced to calculate budget pressure. The Sigmoid function is then used to map the budget pressure to the (0,1] interval to calculate the budget constraint factor. Calculate the inventory constraint factor based on the inventory data in the business status parameters.

8. The real-time data analysis method for marketing scenarios according to claim 7, characterized in that, The effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient are input into the decision function to calculate the expected net revenue. If the expected net revenue is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is intercepted, including: The business revenue parameters associated with the current marketing campaign can be queried in real time from the marketing campaign configuration center. These business revenue parameters include the profit per transaction of the target product and the execution cost of the current single marketing decision action. The effective marketing increment, the prediction confidence level, the comprehensive resource constraint coefficient, and the business revenue parameters are input into the decision function to calculate the expected net revenue. If the expected net profit is greater than 0, a marketing decision action is generated and issued, and the context information of this decision is encapsulated as a context record and saved asynchronously; otherwise, the marketing decision action is intercepted.

9. The real-time data analysis method for marketing scenarios according to claim 8, characterized in that, The process involves collecting feedback results corresponding to marketing decision-making actions, calculating weighted gradients, updating the weight vector of the marketing response model using an online learning algorithm, and then using the updated weight vector for the next real-time data analysis, including: At the end of the preset observation window after the marketing decision action is issued, the user's transaction conversion behavior is monitored through the stream computing engine to obtain the real conversion tag corresponding to the marketing decision action request at the current moment as a feedback result. Extract the real-time aligned feature vector, intervention conversion probability, and effective marketing increment corresponding to the marketing decision action from the asynchronously saved context record; Based on the effective incremental marketing data, the incremental sensitive weight is calculated, and the weighted gradient is calculated based on the cross-entropy loss function of logistic regression, the real conversion label, the real-time aligned feature vector, the intervention conversion probability, and the incremental sensitive weight. The marketing response model's weight vector is incrementally updated using an online stochastic gradient descent algorithm combined with weighted gradients. The updated weight vector is then hot-loaded through the model parameter center for real-time data analysis in the next cycle.

10. A real-time data analysis system applied to marketing scenarios, used to implement the real-time data analysis method as described in any one of claims 1-9, characterized in that, The system includes: The feature alignment unit is used to collect real-time marketing event streams, calculate the real-time continuous features of users based on a sliding window, query the pre-stored offline quantile boundary table to perform bucket discretization mapping on the real-time continuous features, and generate a real-time aligned feature vector after splicing. The incremental prediction unit is used to input the real-time aligned feature vector into the baseline conversion model and the marketing response model respectively, generate the natural conversion probability and the intervention conversion probability, and calculate the effective marketing increment and prediction confidence based on the difference between the two. The resource constraint unit is used to monitor changes in the business database in real time, obtain the remaining budget percentage and budget consumption rate of the current marketing campaign, obtain the inventory of related products, and calculate the comprehensive resource constraint coefficient. The dynamic decision-making unit is used to input the effective marketing increment, the prediction confidence level, and the comprehensive resource constraint coefficient into the decision function to calculate the expected net revenue. If it is greater than 0, a marketing decision action is generated and issued; otherwise, the marketing decision action is intercepted. The online fine-tuning unit is used to collect feedback results corresponding to marketing decision actions, calculate weighted gradients, update the weight vector of the marketing response model using an online learning algorithm, and use the updated weight vector for the next real-time data analysis.

Images