Bank marketing model construction method and system based on machine learning

Abstract

The invention relates to a bank marketing model construction method and system based on machine learning, and the method comprises the following steps: constructing a customer multi-dimensional feature engineering system which is used for integrating customer multi-dimensional features, and extracting customer behavior period features through employing a time sequence feature coding technology; meanwhile, a graph neural network is adopted to construct customer social influence features; constructing a dynamic customer value analysis model integrated with XGBoost and establishing a dynamic attenuation function of customer life cycle value; establishing a marketing response prediction rate model of a hybrid model architecture combining LightGBM and Transform, and embedding an adversarial training sample generation mechanism; a marketing strategy generation model is constructed, the marketing strategy generation model is based on a multi-objective optimization function considering the marketing response rate, the marketing cost and the customer satisfaction, and a marketing strategy is generated through a marketing path planning algorithm of Monte Carlo tree search and the multi-objective optimization function.

Description

Technical Field

[0001] The present invention relates to a method and system for constructing a bank marketing model based on machine learning, belonging to the technical field of bank marketing systems. Background Art

[0002] With the rapid development of fintech, the marketing mode of the banking industry is undergoing profound changes, and traditional marketing methods are difficult to meet the accurate and personalized customer needs. Currently, the bank marketing field mainly faces problems such as insufficient customer feature mining, low efficiency of marketing models, and insufficient intelligence of strategy generation mechanisms.

[0003] In terms of customer feature mining, existing technologies mainly rely on static attributes and basic transaction data, lacking in-depth analysis of the temporal patterns of customer behavior. Although some studies have used the dynamic time warping algorithm to match and analyze the time series of consumer behavior and perform pattern recognition, these methods often ignore dimensions such as the influence of the customer's social network, resulting in an insufficient degree of completeness of the customer portrait. CN117557299A discloses a computer-aided marketing planning method that constructs a dynamic model through state space analysis, but this method fails to effectively integrate multi-dimensional features of customers, especially the integration of external credit data and internal data is still insufficient.

[0004] In terms of marketing prediction models, traditional models such as logistic regression are difficult to capture complex non-linear relationships, resulting in generally low prediction accuracy. CN112541817A discloses a method for processing marketing responses of potential customers for personal consumption loans, which uses machine learning training to establish a customer response probability prediction model, but does not consider the dynamic decay characteristics of customer value, resulting in a relatively high error rate in the identification of high-value customers. In addition, the existing models have a long update cycle, cannot adapt to the rapidly changing market environment, and lack an effective guarantee mechanism for model stability.

[0005] In terms of marketing strategy generation, CN119151584A discloses a method and system for processing bank business marketing data based on multiple channels, which uses machine learning algorithms to generate customer portraits, analyze their behavior patterns and potential needs, and generate personalized marketing strategies. However, this method still overly relies on artificial experience rules and lacks the ability of multi-objective collaborative optimization. CN118863937A proposes a targeted marketing method for private domain traffic platforms based on user purchase frequency analysis, which uses a GNN-Transformer hybrid model to train a user behavior prediction model, but this method fails to establish a real-time feedback closed-loop, the application of A / B test results has obvious delays, and the cross-channel collaborative strategy is missing, resulting in a relatively high waste rate of marketing costs.

[0006] Improvement attempts in the prior art also have obvious defects. The customer segmentation method based on random forest fails to solve the problem of effective encoding of high-dimensional sparse features, lacks the ability to dynamically model the customer life cycle value, and has insufficient model interpretability. CN119151584A describes using natural language processing technology to ensure the timeliness and pertinence of marketing content, but fails to build an end-to-end strategy optimization system, making it difficult to achieve the automated generation and optimization of marketing strategies.

[0007] In addition, when dealing with time series data, the prior art mostly adopts methods such as LSTM. However, these methods are difficult to be compatible with structured and unstructured data and do not consider the impact of data drift on model stability. Although some research has proposed using A / B testing to optimize marketing activities and form a closed-loop feedback, there is a lack of a multi-objective collaborative optimization mechanism for marketing response rate, marketing cost, and customer satisfaction, making it difficult to achieve the optimal marketing effect in complex scenarios.

[0008] In summary, the existing bank marketing technologies have problems such as a single dimension of customer feature mining, low efficiency of marketing prediction models, and insufficient intelligence in the strategy generation mechanism. There is an urgent need for a method for constructing a bank marketing model that can integrate multi-dimensional customer features, improve prediction accuracy, and achieve intelligent strategy generation. Summary of the Invention

[0009] To solve the problems existing in the above prior art, the present invention proposes a method and system for constructing a bank marketing model based on machine learning.

[0010] The technical solution of the present invention is as follows:

[0011] On the one hand, the present invention proposes a method for constructing a bank marketing model based on machine learning, including the following steps:

[0012] Construct a customer multi-dimensional feature engineering system for integrating multi-dimensional customer features, including customer transaction flow data, asset allocation data, channel touchpoint data, and external credit investigation data; and adopt time series feature encoding technology to extract customer behavior cycle features; at the same time, use a graph neural network to construct customer social influence features;

[0013] Construct a dynamic customer value analysis model integrating XGBoost, introduce SHAP values to analyze the feature contribution degrees of customer behavior cycle features and customer social influence features; and establish a dynamic decay function for customer life cycle value;

[0014] Establish a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer, and embed an adversarial training sample generation mechanism to generate training samples for training the marketing response rate prediction model;

[0015] Build a marketing strategy generation model. The marketing strategy generation model is based on a multi-objective optimization function that considers marketing response rate, marketing cost, and customer satisfaction. Design a dynamic weight allocation module based on reinforcement learning to allocate weights to the marketing response rate, marketing cost, and customer satisfaction, and generate marketing strategies through a marketing path planning algorithm based on Monte Carlo tree search and the multi-objective optimization function. Deploy the built marketing strategy generation model in the bank marketing system.

[0016] As a preferred implementation manner, the step of extracting the customer behavior cycle pattern by using the time series feature encoding technology includes:

[0017] The time series feature encoding technology adopts a dual attention mechanism, and the specific implementation formula is:

[0018]

[0019] Where Attn(·) is the attention calculation formula, Q, K, and V are the query vector, key vector, and value vector respectively, Softmax(·) is the Softmax normalization function, M is the domain knowledge constraint matrix, α and β are learnable time attention parameters, and d k is the feature dimension of the customer multi-dimensional features;

[0020] Among them, the query vector and the key vector are obtained by extracting the customer multi-dimensional features, and the value vector is the value corresponding to the customer multi-dimensional features.

[0021] As a preferred implementation manner, the dynamic decay function is modeled by the Hawkes process, and its intensity function is:

[0022]

[0023] Where λ(t) represents the event occurrence intensity at time node t, μ is the base intensity, representing the probability of the event occurring spontaneously, t i represents the time point when the i-th historical event occurred, γ represents the influence intensity of the historical event on the future event, and v represents the decay rate of the influence.

[0024] As a preferred implementation manner, the adversarial training sample generation mechanism includes:

[0025] Use WassersteinGAN to generate boundary samples;

[0026] Design a feature-level perturbation injection module;

[0027] Establish an adversarial training strategy with dynamic difficulty adjustment.

[0028] As a preferred implementation manner, the multi-objective optimization function adopts the Pareto front search algorithm, and the specific formula is:

[0029] max{ω1ResponseRate - ω2Cost + ω3NPS};

[0030] Where max{·} represents the maximum value function, ResponseRate is the marketing response rate, ω1 is the weight of the marketing response rate, Cost is the marketing cost, ω2 is the weight of the marketing cost, NPS is the customer satisfaction, and ω3 is the weight of the customer satisfaction;

[0031] At the same time, regulatory constraints, risk control constraints, and resource limit constraints are set.

[0032] On the other hand, the present invention proposes a bank marketing model construction system based on machine learning, including:

[0033] A feature engineering system construction module for constructing a multi-dimensional customer feature engineering system to integrate multi-dimensional customer features, including customer transaction flow data, asset allocation data, channel contact data, and external credit investigation data; and using time series feature encoding technology to extract customer behavior cycle features; at the same time, using a graph neural network to construct customer social influence features;

[0034] A customer value analysis module for constructing a dynamic customer value analysis model integrating XGBoost, introducing SHAP values for feature contribution degree analysis of customer behavior cycle features and customer social influence features; and establishing a dynamic decay function for customer life cycle value;

[0035] A marketing response rate prediction model construction module for establishing a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer, and embedding an adversarial training sample generation mechanism to generate training samples for training the marketing response rate prediction model;

[0036] A marketing model construction module for constructing a marketing strategy generation model. The marketing strategy generation model is based on a multi-objective optimization function considering the marketing response rate, marketing cost, and customer satisfaction, designs a dynamic weight allocation module based on reinforcement learning to allocate weights to the marketing response rate, marketing cost, and customer satisfaction, and generates marketing strategies through a marketing path planning algorithm of Monte Carlo tree search and the multi-objective optimization function; and deploys the constructed marketing strategy generation model in the bank marketing system.

[0037] As a preferred implementation manner, the steps of using time series feature encoding technology to extract customer behavior cycle patterns include:

[0038] The time series feature encoding technology adopts a dual attention mechanism, and the specific implementation formula is:

[0039]

[0040] Among them, Attn(·) is the attention calculation formula, Q, K, and V are the query vector, key vector, and value vector respectively, Softmax(·) is the Softmax normalization function, M is the domain knowledge constraint matrix, α and β are learnable time attention parameters, and d k is the feature dimension of the customer's multi-dimensional features;

[0041] Among them, the query vector and the key vector are obtained by extracting the customer's multi-dimensional features, and the value vector is the value corresponding to the customer's multi-dimensional features.

[0042] As a preferred implementation, the dynamic decay function is modeled by the Hawkes process, and its intensity function is:

[0043]

[0044] Among them, λ(t) represents the event occurrence intensity at time node t, μ is the base intensity, representing the probability of the event occurring spontaneously, t i represents the time point when the i-th historical event occurred, γ represents the influence intensity of historical events on future events, and ν represents the decay rate of the influence.

[0045] On the other hand, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements a method for constructing a bank marketing model based on machine learning as described in any embodiment of the present invention.

[0046] On the other hand, the present invention also proposes a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements a method for constructing a bank marketing model based on machine learning as described in any embodiment of the present invention.

[0047] The beneficial effects of the present invention are as follows:

[0048] 1. The present invention integrates structured and unstructured data through a multi-dimensional feature engineering system, solves the problem that the integrity of the customer portrait in the traditional system is less than 60%, and improves the comprehensiveness of customer feature mining. Experimental data shows that after adopting the multi-dimensional feature engineering system of the present invention, the integrity of the customer portrait is increased to more than 85%, laying a data foundation for subsequent precision marketing.

[0049] 2. The dynamic customer value stratification model integrating XGBoost of the present invention introduces SHAP value analysis and a dynamic decay function, effectively solving the problem that the error rate of high-value customer identification in traditional models exceeds 30%, and improving the accuracy of customer value evaluation. Through the dynamic decay function modeled by the Hawkes process, the present invention can accurately capture the time-varying characteristics of customer value and reduce the error rate of high-value customer identification to less than 12%.

[0050] 3. By combining the hybrid model architecture of LightGBM and Transformer, the present invention overcomes the defect that the prediction accuracy of traditional logistic regression models is lower than 65% in complex scenarios, and improves the accuracy of marketing prediction. In practical applications, the prediction accuracy of this hybrid model architecture reaches 82%, which is 17 percentage points higher than that of traditional models.

[0051] 4. The marketing strategy generation model constructed by the present invention is based on multi-objective optimization functions and dynamic weight allocation of reinforcement learning, solving the problems that traditional marketing strategies rely on manual experience rules and lack the ability of multi-objective collaborative optimization, and reducing the marketing cost waste rate. After applying the marketing strategy generation model of the present invention, the marketing cost waste rate is reduced from 40% to 15%, and the customer satisfaction is increased by 25% at the same time.

[0052] 5. Through the dynamic model update mechanism, the present invention overcomes the disadvantage of the long model update cycle of traditional models, enabling the system to adapt to the rapidly changing market environment. The present invention realizes automatic model update every week, and shortens the response time to market changes from several weeks to several hours.

[0053] Additional aspects and advantages of the present invention will be set forth in the following description, and some of them will be obvious from the description, or can be learned by practicing the present invention. In addition, the various aspects and advantages of the present invention can be realized and obtained by the method steps and combinations specifically pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic flowchart of the method according to Embodiment 1 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0056] It should be understood that the step numbers used in the text are only for convenience of description and do not limit the order of execution of the steps.

[0057] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include the plural forms.

[0058] The terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their combinations.

[0059] The term "and / or" refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0060] Embodiment 1:

[0061] See Figure 1 , this embodiment proposes a method for constructing a bank marketing model based on machine learning, which specifically includes the following steps:

[0062] S100. Construct a customer multi-dimensional feature engineering system:

[0063] In this step, a customer multi-dimensional feature engineering system is constructed to integrate customer multi-dimensional features, including customer transaction flow data, asset allocation data, channel contact data, and external credit investigation data; and time series feature encoding technology is used to extract customer behavior cycle features; at the same time, a graph neural network is used to construct customer social influence features.

[0064] In this embodiment, the construction process of the customer multi-dimensional feature engineering system is as follows:

[0065] First, process the customer transaction flow data to extract features such as transaction frequency, transaction amount, transaction type, and transaction time. The transaction flow data includes all transaction records of the customer in the bank system, such as operations like transfers, withdrawals, deposits, and wealth management product purchases. The system extracts the customer transaction data for the past 24 months from the bank's core business system and groups it according to the unique identifier of each customer. For each customer, statistical features such as monthly transaction frequency, average transaction amount, maximum transaction amount, and transaction type distribution are calculated.

[0066] Secondly, process the customer's asset allocation data and extract features such as the total asset amount, asset structure, and risk preference of the customer. The asset allocation data includes various asset situations of the customer in the bank, such as current deposits, fixed deposits, wealth management products, funds, insurance, etc. The system calculates indicators such as the total asset amount of the customer, the proportion of various assets, and asset volatility, and derives the customer's risk preference index based on the customer's investment product selection history.

[0067] Then, process the channel touchpoint data and extract features such as the channel types used by the customer, channel usage frequency, and channel preference. The channel touchpoint data includes the interaction records of the customer with each service channel of the bank, such as online banking login, mobile banking usage, ATM operations, counter services, etc. The system statistics the activity, usage duration, and function usage of the customer on each channel to form the customer's channel usage portrait.

[0068] Finally, integrate the external credit investigation data and extract features such as the customer's credit score, debt situation, and repayment ability. The external credit investigation data comes from the People's Bank of China credit investigation system and third-party credit investigation agencies, including the customer's credit score, loan records, overdue situations, etc. The system matches these data with the bank's internal data to form a complete customer credit portrait.

[0069] When extracting the customer behavior cycle features, the time series feature encoding technology is adopted, and the specific implementation is as follows:

[0070] The system first arranges the customer's transaction data in chronological order to form a time series. For each customer, sequence data containing features such as transaction amount, transaction type, and transaction time is constructed. Then, a dual attention mechanism is used to encode the time series data to extract the periodic patterns of customer behavior.

[0071] The specific implementation formula of the dual attention mechanism is:

[0072]

[0073] Among them, Attn(·) is the attention calculation formula, Q, K, and V are the query vector, key vector, and value vector respectively, Softmax(·) is the Softmax normalization function, M is the domain knowledge constraint matrix, α and β are learnable time attention parameters, and d k is the feature dimension of the customer's multi-dimensional features.

[0074] In this embodiment, the query vector and the key vector are obtained by extracting the customer's multi-dimensional features, and the value vector is the value corresponding to the customer's multi-dimensional features. Specifically, for the customer's transaction sequence data, the system first uses a fully connected layer to map the original features to the hidden space to obtain the query vector Q, the key vector K, and the value vector V. Then, the dot product of the query vector and the key vector is calculated and scaled by dividing by the square root of the feature dimension to avoid the vanishing gradient problem. Next, a domain knowledge constraint matrix M is introduced, which encodes the prior knowledge of banking experts about customer behavior patterns, such as increased consumption after receiving salary at the beginning of the month, and increased purchase of wealth management products at the end of the year. Finally, the result is normalized through the Softmax function and multiplied by the value vector to obtain the final attention output.

[0075] Through the dual attention mechanism, the system can capture the long-term dependencies and periodic patterns of customer behavior, such as consumption patterns after monthly salary receipt, reinvestment behavior after the maturity of quarterly wealth management products, etc. These periodic features are of great significance for predicting customers' future behavior and needs.

[0076] When constructing the customer social influence features, the graph neural network technology is adopted, and the specific implementation is as follows:

[0077] The system first constructs a customer social network graph, where nodes represent customers and edges represent the social relationships between customers. The social relationships can come from multiple channels, such as customers' transfer records, jointly participated bank activities, family member relationships, etc. For each customer node, the system extracts node features, including the customer's basic information, transaction behavior features, etc. For each edge, the system extracts edge features, such as transfer frequency, transfer amount, relationship type, etc.

[0078] Then, the system uses a graph convolutional network (GCN) to model the customer social network. The core idea of GCN is to update the representation of the central node by aggregating the information of neighbor nodes. Specifically, for customer i, its representation at the l-th layer is updated through the following formula:

[0079]

[0080] where N(i) represents the set of neighbors of customer i, AGGREGATE represents an aggregation function, such as average, maximum value, or attention weighted sum, W l is the weight matrix at the l-th layer, and σ is an activation function, such as ReLU.

[0081] Through multi-layer graph convolution operations, the system can capture the structural features and influence of customers in the social network. Specifically, the system calculates centrality metrics of customers, such as degree centrality, betweenness centrality, and eigenvector centrality, to quantify the importance and influence of customers in the social network. At the same time, the system also identifies community structures, groups customers into different social circles, and analyzes the interaction patterns and influence propagation paths within the communities.

[0082] Finally, the system integrates the social network features of customers with features from other dimensions to form a complete customer feature representation. These features will be used for subsequent customer value stratification and marketing response prediction.

[0083] S200. Build a dynamic customer value stratification model integrating XGBoost

[0084] In this step, build a dynamic customer value stratification model integrating XGBoost, introduce SHAP values to analyze the feature contribution degrees of customer behavior cycle features and customer social influence features; and establish a dynamic decay function for customer lifetime value.

[0085] In this embodiment, the construction process of the dynamic customer value stratification model is as follows:

[0086] The system first trains an XGBoost model to predict customer value based on the multi-dimensional customer features constructed in step S100. XGBoost is an ensemble learning algorithm based on gradient boosting decision trees, which has the advantages of high efficiency, scalability, and accuracy. In this embodiment, the target variable of the XGBoost model is the customer value index, which comprehensively considers factors such as the customer's contribution profit, asset scale, and product holding quantity.

[0087] The training process of the XGBoost model is as follows:

[0088] The system first divides the customer dataset into a training set and a validation set with a ratio of 8:2. Then, set the hyperparameters of XGBoost, including the maximum tree depth, learning rate, regularization parameters, etc. In this embodiment, the maximum tree depth is set to 6, the learning rate is set to 0.1, the L1 regularization parameter is set to 0.01, and the L2 regularization parameter is set to 0.1. Next, the system uses the training set to train the XGBoost model and evaluates the model performance on the validation set. To avoid overfitting, the system adopts an early stopping strategy and stops training when the performance on the validation set has not improved for 5 consecutive rounds.

[0089] After training is completed, the system uses SHAP (SHapley Additive exPlanations) values to analyze feature contribution degrees. SHAP values are a game theory-based method for explaining the prediction results of machine learning models. For each feature of each customer, the SHAP value represents the contribution of that feature to the model's prediction. Specifically, the system calculates the SHAP values of each feature of each customer and analyzes the impact of different features on customer value prediction.

[0090] In particular, the system focuses on analyzing the contribution degrees of customer behavior cycle features and customer social influence features. Through SHAP value analysis, the system can identify which periodic behavior patterns and social network features have a significant impact on customer value. For example, the system may find that customers who immediately purchase financial products after their monthly salary is credited usually have higher value; or customers with high centrality in the social network can often influence the product choices of other customers and thus have higher value.

[0091] Based on the results of SHAP value analysis, the system optimizes the XGBoost model, adjusts the feature weights, and improves the prediction accuracy of the model. At the same time, the system classifies customers into different levels according to the predicted value, such as high-value customers, medium-value customers, and low-value customers, providing a basis for formulating subsequent marketing strategies.

[0092] When establishing the dynamic decay function of customer lifetime value, the system uses the Hawkes process for modeling. The Hawkes process is a self-exciting point process that can capture the time dependence and clustering of event occurrences. In this embodiment, the system uses the Hawkes process to model the time pattern of customer transaction behavior and constructs a dynamic decay function based on this.

[0093] The intensity function of the Hawkes process is:

[0094]

[0095] where λ(t) represents the intensity of event occurrence at time node t, μ is the baseline intensity, representing the probability of an event occurring spontaneously, t i represents the time point of the i-th historical event occurrence, γ represents the influence intensity of historical events on future events, and v represents the decay rate of the influence.

[0096] In this embodiment, the system regards each transaction of a customer as an event and uses the Hawkes process to model the customer's transaction behavior. By estimating the parameters μ, γ, and v, the system can capture the time dependence and periodicity of customer transaction behavior. Based on the parameter estimation results of the Hawkes process, the system constructs a dynamic decay function of customer lifetime value:

[0097]

[0098] Among them, CLV(t) represents the customer lifetime value at time t, CLV0 represents the initial customer value, γ represents the base decay rate, and α i represents the contribution of the i-th transaction to the customer value, and β represents the decay rate of the contribution.

[0099] Through the dynamic decay function, the system can update the customer value assessment in real time, reflecting the changes in customer behavior and the fluctuations in value. This dynamic assessment method can more accurately reflect the true value of customers than the traditional static stratification method, providing support for precision marketing.

[0100] S300: Establish a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer:

[0101] In this step, a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer is established, and an adversarial training sample generation mechanism is embedded to generate training samples for training the marketing response rate prediction model.

[0102] In this embodiment, the establishment process of the marketing response prediction rate model is as follows:

[0103] The system first constructs a hybrid model architecture, combining the advantages of LightGBM and Transformer. LightGBM is an efficient gradient boosting decision tree framework suitable for processing structured data; Transformer is a deep learning model based on self-attention mechanism, suitable for processing sequence data and capturing long-term dependencies.

[0104] The design of the hybrid model architecture is as follows:

[0105] For the static features of customers, such as demographic features, credit scores, etc., the system uses LightGBM for modeling. The output of the LightGBM model is the preliminary prediction probability of the customer's response to the marketing activity.

[0106] For the dynamic behavior sequences of customers, such as transaction history, channel usage records, etc., the system uses Transformer for modeling. The input of the Transformer model is the embedded representation of the customer behavior sequence, and the output is the sequence-level feature representation.

[0107] The system fuses the output of LightGBM and the output of Transformer, and maps them to the final response probability prediction through a fully connected layer. The fusion method adopts the attention mechanism, and dynamically adjusts the weights of the outputs of the two models according to different customers and marketing scenarios.

[0108] The training process of the hybrid model is as follows:

[0109] The system first trains the LightGBM model and the Transformer model separately. For the LightGBM model, the system uses the customer's static features as input and the historical marketing response results as labels, and trains using cross-validation. For the Transformer model, the system uses the customer's behavior sequence as input and the historical marketing response results as labels, and trains using a sequence-to-scalar architecture.

[0110] Then, the system fixes the parameters of LightGBM and Transformer and trains the parameters of the fusion layer. The training objective of the fusion layer is to minimize the cross-entropy loss between the predicted response probability and the actual response result.

[0111] Finally, the system performs end-to-end fine-tuning on the entire hybrid model to further optimize the model performance.

[0112] To enhance the robustness and generalization ability of the model, the system embeds an adversarial training sample generation mechanism to generate high-quality training samples. The adversarial training sample generation mechanism includes the following three components:

[0113] 1. WassersteinGAN generates boundary samples:

[0114] The system uses WassersteinGAN (WGAN) to generate samples located near the decision boundary. WGAN is a variant of the generative adversarial network that trains the generator by minimizing the Wasserstein distance between the real data distribution and the generated data distribution.

[0115] In this embodiment, the generator G of WGAN receives random noise z and conditional information c as input and generates a customer feature vector x'. The conditional information c includes the target response label y' and the customer stratification information. The discriminator D receives the feature vector as input and outputs a scalar value indicating the likelihood that the input is a real sample.

[0116] Through WGAN, the system can generate samples that are similar to the real customer feature distribution but are located near the decision boundary, and these samples have high information value for model training.

[0117] 2. Feature-level perturbation injection module:

[0118] The system designs a feature-level perturbation injection module to perturbe the customer features in a targeted manner to generate adversarial samples. Specifically, the system first uses SHAP value analysis to identify the key features that have a greater impact on the prediction results. Then, these key features are perturbed to generate adversarial samples.

[0119] The ways of perturbation include:

[0120] For continuous features, add random noise that conforms to the normal distribution N(0,σ 2 ), where σ is the perturbation intensity dynamically adjusted according to feature importance.

[0121] For discrete features, replace them with other possible values with a certain probability p, and the probability p is also dynamically adjusted according to feature importance.

[0122] For time series features, perform perturbation in the time dimension, such as adjusting the order or time interval of event occurrences.

[0123] Through feature-level perturbation, the system can generate adversarial samples that are similar to the original samples in the feature space but may have different prediction results. These samples can help the model learn more robust decision boundaries.

[0124] 3. Adversarial training strategy with dynamic difficulty adjustment:

[0125] The system establishes an adversarial training strategy with dynamic difficulty adjustment, and dynamically adjusts the generation difficulty of adversarial samples according to the training progress and performance of the model. Specifically, the system defines a difficulty index d for adversarial samples, which comprehensively considers factors such as the distance between the sample and the decision boundary and the amplitude of feature perturbation.

[0126] In the initial stage of training, the system generates adversarial samples with relatively low difficulty to help the model establish a basic decision boundary. As training progresses, the system gradually increases the difficulty of adversarial samples to challenge the model's decision boundary and prompt the model to learn more complex patterns. When the model performance reaches a plateau on the validation set, the system generates more challenging adversarial samples to help the model break through the performance bottleneck.

[0127] Through dynamic difficulty adjustment, the system can maintain an appropriate learning difficulty during training, avoiding both the problem that overly simple samples cannot provide effective information and the problem that overly difficult samples cause the model to be unable to learn.

[0128] In this embodiment, the specific implementation of the adversarial training strategy with dynamic difficulty adjustment is as follows:

[0129] The system defines a difficulty index d for adversarial samples, which comprehensively considers factors such as the distance between the sample and the decision boundary and the amplitude of feature perturbation. Specifically, the calculation formula for the difficulty index d is:

[0130]

[0131] where p is the prediction probability of the model for the sample, and |p - 0.5|·2 represents the distance between the sample and the decision boundary. Represents the relative amplitude of feature perturbation. w1 and w2 are weight coefficients satisfying w1 + w2 = 1.

[0132] During the training process, the system dynamically adjusts the generation difficulty of adversarial samples according to the training progress and performance of the model. Specifically, the system defines a difficulty adjustment function f(e, v), where e is the current training round and v is the performance metric on the validation set. The form of the difficulty adjustment function is:

[0133]

[0134] where d min is the minimum difficulty, usually set to 0.1; d max is the maximum difficulty, usually set to 0.9; e nax is the preset maximum number of training rounds; g(v) is an adjustment factor based on the validation set performance. When the performance improves, g(v) increases, prompting the system to generate more challenging samples; when the performance deteriorates, g(v) decreases, causing the system to generate simpler samples.

[0135] Through dynamic difficulty adjustment, the system can maintain an appropriate learning difficulty during the training process, avoiding both the situation where overly simple samples cannot provide effective information and the problem that overly difficult samples prevent the model from learning.

[0136] The complete process of adversarial training is as follows:

[0137] The system first trains an initial model using the original training data.

[0138] Then, it uses WGAN to generate samples near the decision boundary.

[0139] Next, it uses the feature-level perturbation module to perturb the key features to generate adversarial samples.

[0140] According to the current training progress and model performance, dynamically adjust the generation difficulty of adversarial samples.

[0141] Mix the original samples and adversarial samples to form an enhanced training set.

[0142] Use the enhanced training set to retrain the model.

[0143] Repeat the above steps until the model performance converges or reaches the preset number of training rounds.

[0144] Through adversarial training, the system can significantly improve the robustness and generalization ability of the marketing response prediction model, especially showing better prediction performance when dealing with new customers or new marketing scenarios.

[0145] Finally, retrain the model using the enhanced training set and dynamically adjust the generation strategy of adversarial samples according to the model performance.

[0146] Through adversarial training, the system can significantly improve the robustness and generalization ability of the marketing response prediction model, especially when dealing with new customers or new marketing scenarios, showing better prediction performance.

[0147] S400: Build a marketing strategy generation model;

[0148] In this step, build a marketing strategy generation model. The marketing strategy generation model is based on a multi-objective optimization function that considers the marketing response rate, marketing cost, and customer satisfaction. Design a dynamic weight allocation module based on reinforcement learning to allocate weights to the marketing response rate, marketing cost, and customer satisfaction, and generate marketing strategies through a marketing path planning algorithm based on Monte Carlo tree search and the multi-objective optimization function; deploy the built marketing strategy generation model into the bank marketing system.

[0149] In this embodiment, the construction process of the marketing strategy generation model is as follows:

[0150] The system first defines a multi-objective optimization function, comprehensively considering three objectives: the marketing response rate, marketing cost, and customer satisfaction. The multi-objective optimization function uses the Pareto front search algorithm, and the specific formula is:

[0151] max{ω1ResponseRate - ω2Cost + ω3NPS};

[0152] where max{·} represents the maximum value function, ResponseRate is the marketing response rate, ω1 is the weight of the marketing response rate, Cost is the marketing cost, ω2 is the weight of the marketing cost, NPS is the customer satisfaction, and ω3 is the weight of the customer satisfaction.

[0153] At the same time, the system sets the following constraint conditions:

[0154] Regulatory constraint conditions: Ensure that the marketing strategy complies with financial regulatory requirements, such as not promoting high-risk products to specific customer groups.

[0155] Risk control constraints: Ensure that the marketing strategy does not cause the bank to bear excessive risks, such as controlling the proportion of high-risk loans.

[0156] Resource limitation constraints: Ensure that the marketing strategy is executable within the bank's resource limitations, such as controlling the marketing budget, human resources, etc.

[0157] To dynamically adjust the weights in the multi-objective optimization function, the system designs a dynamic weight allocation module based on reinforcement learning. This module uses a Deep Q-Network (DQN) to learn the optimal weight allocation strategy. Specifically, the system defines the state space S, the action space A, and the reward function R:

[0158] The state space S includes the characteristics of the current customer, historical marketing response situations, market environment indicators, etc.

[0159] The action space A is the set of possible values of the weight vector (ω1, ω2, ω3), satisfying ω1 + ω2 + ω3 = 1 and ω i ≥ 0.

[0160] The reward function R is the actual revenue after the implementation of the marketing strategy, including the revenue brought by responses, marketing costs, and changes in customer satisfaction.

[0161] The training process of the DQN is as follows:

[0162] The system first initializes the parameters θ of the Q-network and the parameters θ' of the target network.

[0163] Then, for each training episode, the system observes the current state s, selects an action a (i.e., the weight vector), executes the marketing strategy, observes the reward r and the next state s'.

[0164] Next, the system stores the experience tuple (s, a, r, s') in the experience replay buffer.

[0165] Randomly sample a batch of experiences from the experience replay buffer and calculate the target Q-value: y = r + γ·max_a'Q(s', a'; θ'), where γ is the discount factor.

[0166] Update the Q-network parameters θ, minimizing the loss function L(θ) = E[(y - Q(s, a; θ)) 2 .

[0167] Every certain number of steps, copy the parameters of the Q-network to the target network: θ' ← θ.

[0168] Through the DQN, the system can learn a strategy to dynamically adjust the weights according to different customer characteristics and market environments, thereby generating more personalized and effective marketing strategies.

[0169] When generating a specific marketing strategy, the system uses the Monte Carlo Tree Search (MCTS) algorithm for marketing path planning. MCTS is a simulation-based search algorithm that evaluates the value of different decisions through a large number of simulations. In this embodiment, MCTS is used to plan the optimal sequence and timing of marketing activities.

[0170] The execution process of MCTS is as follows:

[0171] The system first defines a decision tree, where nodes represent marketing states and edges represent marketing actions. Marketing states include the current state of the customer, the sequence of executed marketing activities, etc. Marketing actions include the type of recommended products, the recommended channels, the recommended timing, etc.

[0172] Then, starting from the root node, the system performs the following four steps:

[0173] Selection: Starting from the root node, select child nodes according to the UCB (Upper Confidence Bound) formula until an unexpanded node is reached. The UCB formula is: UCB = Q(s,a) + c·√(ln(N(s)) / N(s,a)), where Q(s,a) is the average reward for executing action a in state s, N(s) is the number of times state s has been visited, N(s,a) is the number of times action a has been executed in state s, and c is the exploration constant.

[0174] Expansion: For an unexpanded node, randomly select an untried action and add the corresponding child node.

[0175] Simulation: Starting from the newly added node, perform simulation using a random policy or a simple heuristic policy until a terminal state is reached. In this embodiment, the simulation uses the marketing response prediction model trained in step three to estimate the response probability of the customer.

[0176] Backpropagation: Starting from the terminal state of the simulation, backpropagate the reward value to the root node and update the statistical information of each node on the path.

[0177] The system repeats the above steps until the computing resources are exhausted or a predetermined number of simulations is reached. Then, the system selects the action corresponding to the child node with the most visits under the root node as the optimal marketing action.

[0178] Through MCTS, the system can plan a marketing path considering long-term benefits, rather than just pursuing short-term response rates. For example, the system may first recommend a low-cost introductory product to build customer trust, and then recommend high-value core products to achieve higher long-term benefits.

[0179] Finally, the system deploys the generated marketing strategy to the bank marketing system. The deployment process includes the following steps:

[0180] The system first exports the trained models (including the customer value stratification model, the marketing response prediction model, and the marketing strategy generation model) to a standard format, such as the ONNX (Open Neural Network Exchange) format.

[0181] Then, the system integrates the model into the bank's marketing decision-making engine and interfaces with the existing business systems.

[0182] Next, the system designs an A / B test plan, allocates some customers to the new model group and some customers to the control group, and compares the marketing effects of the two groups.

[0183] Finally, based on the results of the A / B test, the system makes necessary adjustments and optimizations, and gradually expands the application scope of the new model.

[0184] Through the above steps, the system constructs an end-to-end bank marketing model, which can extract multi-dimensional features from customer data, dynamically evaluate customer value, accurately predict the marketing response rate, and generate personalized marketing strategies. This machine learning-based marketing model has higher accuracy, stronger personalization ability and better adaptability compared with traditional methods, and can significantly improve the bank's marketing effect and customer experience.

[0185] Example Two:

[0186] This example provides a system for constructing a bank marketing model based on machine learning, including:

[0187] A feature engineering system construction module, which is used to construct a customer multi-dimensional feature engineering system for integrating customer multi-dimensional features, including customer transaction flow data, asset allocation data, channel contact data, and external credit investigation data; and adopts time series feature encoding technology to extract customer behavior cycle features; at the same time, adopts a graph neural network to construct customer social influence features; this module is used to implement the function of step S100 in Example One, which will not be elaborated here;

[0188] A customer value analysis module, which is used to construct a dynamic customer value analysis model integrating XGBoost, introduce SHAP values to analyze the feature contribution degrees of customer behavior cycle features and customer social influence features; and establish a dynamic decay function for customer life cycle value; this module is used to implement the function of step S200 in Example One, which will not be elaborated here;

[0189] A marketing response rate prediction model construction module, which is used to establish a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer, and embed an adversarial training sample generation mechanism to generate training samples for training the marketing response rate prediction model; this module is used to implement the function of step S300 in Example One, which will not be elaborated here;

[0190] A marketing model construction module for constructing a marketing strategy generation model. The marketing strategy generation model is based on a multi-objective optimization function considering marketing response rate, marketing cost, and customer satisfaction. It designs a dynamic weight allocation module based on reinforcement learning to allocate weights to the marketing response rate, marketing cost, and customer satisfaction, and generates marketing strategies through a marketing path planning algorithm based on Monte Carlo tree search and the multi-objective optimization function. The constructed marketing strategy generation model is deployed in the bank marketing system. This module is used to implement the function of step S400 in Embodiment 1, which will not be elaborated here.

[0191] As a preferred implementation manner of this embodiment, the step of extracting the customer behavior cycle pattern by using the time series feature encoding technology includes:

[0192] The time series feature encoding technology adopts a dual attention mechanism, and the specific implementation formula is:

[0193]

[0194] where Attn(·) is the attention calculation formula, Q, K, and V are the query vector, key vector, and value vector respectively, Softmax(·) is the Softmax normalization function, M is the domain knowledge constraint matrix, α and β are learnable time attention parameters, and d k is the feature dimension of the customer multi-dimensional features;

[0195] Among them, the query vector and the key vector are extracted from the customer multi-dimensional features, and the value vector is the value corresponding to the customer multi-dimensional features.

[0196] As a preferred implementation manner of this embodiment, the dynamic decay function is modeled by the Hawkes process, and its intensity function is:

[0197]

[0198] where λ(t) represents the event occurrence intensity at time node t, μ is the base intensity, representing the probability of the event occurring spontaneously, t i represents the time point when the i-th historical event occurred, γ represents the influence intensity of historical events on future events, and ν represents the attenuation rate of the influence.

[0199] Embodiment 3:

[0200] This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements a machine learning-based bank marketing model construction method as described in any embodiment of the present invention.

[0201] Embodiment 4:

[0202] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements a method for constructing a bank marketing model based on machine learning as described in any embodiment of the present invention.

[0203] In the embodiments of the present application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the cases of A existing alone, A and B existing simultaneously, and B existing alone. Where A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c may represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c may be single or multiple.

[0204] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0205] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0206] In several embodiments provided by the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (hereinafter referred to as ROM), random access memories (hereinafter referred to as RAM), magnetic disks, or optical discs that can store program codes.

[0207] The above are only embodiments of the present invention, and thus do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for constructing a bank marketing model based on machine learning, characterized in that, It includes the following steps: Construct a multi-dimensional customer feature engineering system for integrating multi-dimensional customer features, including customer transaction flow data, asset allocation data, channel touchpoint data, and external credit investigation data; adopt time series feature encoding technology to extract customer behavior cycle features; and use a graph neural network to construct customer social influence features; Construct a dynamic customer value analysis model integrating XGBoost, introduce SHAP values to analyze the feature contribution degrees of customer behavior cycle features and customer social influence features; and establish a dynamic decay function for customer life cycle value; Establish a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer, and embed an adversarial training sample generation mechanism to generate training samples for training the marketing response rate prediction model; Construct a marketing strategy generation model. The marketing strategy generation model is based on a multi-objective optimization function considering marketing response rate, marketing cost, and customer satisfaction. Design a dynamic weight allocation module based on reinforcement learning to allocate weights to the marketing response rate, marketing cost, and customer satisfaction, and generate marketing strategies through a marketing path planning algorithm of Monte Carlo tree search and the multi-objective optimization function; Deploy the constructed marketing strategy generation model into the bank's marketing system.

2. The method for constructing a bank marketing model based on machine learning according to claim 1, wherein The step of adopting time series feature encoding technology to extract customer behavior cycle patterns includes: The time series feature encoding technology adopts a dual attention mechanism, and the specific implementation formula is: where Attn(·) is the attention calculation formula, Q, K, and V are the query vector, key vector, and value vector respectively, Softmax(·) is the Softmax normalization function, M is the domain knowledge constraint matrix, α and β are learnable temporal attention parameters, and d k is the feature dimension of the customer's multi-dimensional features; Among them, the query vector and the key vector are obtained by extracting multi-dimensional customer features, and the value vector is the value corresponding to the multi-dimensional customer features.

3. A method for constructing a bank marketing model based on machine learning according to claim 1, characterized in that, The dynamic decay function is modeled by the Hawkes process, and its intensity function is: Among them, λ(t) represents the event occurrence intensity at time node t, μ is the base intensity, representing the probability of the event occurring spontaneously, and t i represents the time point when the i-th historical event occurred, γ represents the influence intensity of historical events on future events, and ν represents the attenuation rate of the influence.

4. A method for constructing a bank marketing model based on machine learning according to claim 1, characterized in that, The adversarial training sample generation mechanism includes: Use Wasserstein GAN to generate boundary samples; Design a feature-level perturbation injection module; Establish an adversarial training strategy with dynamic difficulty adjustment.

5. A method for constructing a bank marketing model based on machine learning according to claim 1, characterized in that, The multi-objective optimization function adopts the Pareto front search algorithm, and the specific formula is: max{ω1ResponseRate - ω2Cost + ω3NPS}; Among them, max{·} represents the maximum value function, ResponseRate is the marketing response rate, ω1 is the weight of the marketing response rate, Cost is the marketing cost, ω2 is the weight of the marketing cost, NPS is the customer satisfaction, and ω3 is the weight of the customer satisfaction; At the same time, set regulatory constraint conditions, risk control constraints, and resource limit constraints.

6. A bank marketing model construction system based on machine learning, characterized in that, It includes: A feature engineering system construction module for constructing a multi-dimensional customer feature engineering system for integrating multi-dimensional customer features, including customer transaction flow data, asset allocation data, channel touchpoint data, and external credit investigation data; adopting time series feature encoding technology to extract customer behavior cycle features; and using a graph neural network to construct customer social influence features; A customer value analysis module for constructing a dynamic customer value analysis model integrating XGBoost, introducing SHAP values to analyze the feature contribution degrees of customer behavior cycle features and customer social influence features; and establishing a dynamic decay function for customer life cycle value; A marketing response rate prediction model construction module, which is used to establish a marketing response prediction rate model with a hybrid model architecture combining LightGBM and Transformer, and embed an adversarial training sample generation mechanism to generate training samples for training the marketing response rate prediction model; A marketing model construction module, which is used to construct a marketing strategy generation model. The marketing strategy generation model is based on a multi-objective optimization function considering marketing response rate, marketing cost, and customer satisfaction. It designs a dynamic weight allocation module based on reinforcement learning to allocate weights to the marketing response rate, marketing cost, and customer satisfaction, and generates marketing strategies through a marketing path planning algorithm based on Monte Carlo tree search and the multi-objective optimization function; Deploy the constructed marketing strategy generation model into the bank marketing system.

7. A system for constructing a bank marketing model based on machine learning according to claim 6, characterized in that, The steps of extracting the customer behavior cycle pattern by using the time series feature encoding technology include: The time series feature encoding technology adopts a dual attention mechanism, and the specific implementation formula is: Among them, Attn(·) is the attention calculation formula, Q, K, and V are the query vector, key vector, and value vector respectively, Softmax(·) is the Softmax normalization function, M is the domain knowledge constraint matrix, α and β are learnable temporal attention parameters, and d k is the feature dimension of the customer's multi-dimensional features; Among them, the query vector and the key vector are obtained by extracting customer multi-dimensional features, and the value vector is the value corresponding to the customer multi-dimensional features.

8. The system for constructing a bank marketing model based on machine learning according to claim 6, wherein, The dynamic decay function is modeled by the Hawkes process, and its intensity function is: Among them, λ(t) represents the event occurrence intensity at time node t, μ is the basic intensity, representing the probability of the event occurring spontaneously, t i represents the time point when the i-th historical event occurred, γ represents the influence intensity of historical events on future events, and ν represents the attenuation rate of the influence.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 5.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method described in any one of claims 1 to 5.

Images