Credit card application intention prediction method and system based on multi-model fusion
Through multi-model fusion and ensemble learning methods, combined with diversified customer data characteristics, the problem of insufficient overfitting and generalization capabilities of a single model in the prediction of credit card application intention is solved, and more efficient credit card marketing object recognition and accuracy are achieved Marketing support.
Patent Information
- Application Number
- CN202411219284.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2025-01-03
AI Technical Summary
In the prior art, a single model is used to predict overfitting risks and limited generalization capabilities for credit card application intention, which is sensitive to specific types of data, cannot effectively utilize computing resources, and is difficult to accurately identify users' credit card application intentions, which in turn affects marketing objects. Accurate choice.
Blending adopts the parallel integration method, combining the prediction of multiple models, using cross-validation and Bayesian optimization algorithm to adjust parameters, train multi-layer models to smooth the errors of individual models, and integrates learning methods to improve overall performance, combining operators or bank customers. diversified data characteristics, reduce the risk of overfitting and enhance the generalization ability of the model.
Through the multi-model fusion method, the risk of overfitting is reduced, the generalization ability and prediction performance of the model can be improved, and users’ credit card application intentions can be more accurately identified, providing support for the accurate selection of credit card marketing objects, and improving marketing efficiency and market share.
Smart Images
Figure CN119250857A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multi-model data processing, and in particular to a method and system for predicting credit card application intention based on multi-model fusion. Background Art
[0002] In marketing activities, accurate selection of marketing targets can improve marketing efficiency, increase customer conversion rate, improve customer satisfaction, and effectively reduce costs. However, how to accurately select marketing targets is a major problem in marketing activities. Nowadays, machine learning technology is applied to all aspects of people's lives. Using objective data for machine learning modeling and predicting customer credit card application intentions can undoubtedly provide strong technical support for accurately selecting credit card marketing targets. This prediction can help banks and financial institutions better understand the needs of potential customers, so as to adopt corresponding marketing strategies and increase the issuance and market share of credit cards. The prediction of credit card application intention is of significant value to banks and financial institutions.
[0003] In the past, intention prediction generally used a single model, such as logistic regression, support vector machine (SVM), deep learning (DNN), and ensemble tree models (LightGBM and XGBoost). However, a single model may have the risk of overfitting, limited generalization ability, sensitivity to specific types of data, and ineffective use of computing resources.
[0004] Therefore, how to identify users' willingness to apply for credit cards and provide support for the accurate selection of credit card marketing targets is a technical problem that needs to be solved urgently. Summary of the invention
[0005] The technical task of the present invention is to provide a method and system for predicting credit card application intention based on multi-model fusion to solve the problem of how to identify users' credit card application intention and provide support for the accurate selection of credit card marketing targets.
[0006] The technical task of the present invention is achieved in the following way: a method for predicting the willingness to apply for a credit card based on multi-model fusion, which adopts a parallel integration method Blending to train multiple models at the same time. By combining the predictions of multiple models, the integrated learning method Blending can smooth the errors of individual models, thereby providing better performance overall, and providing technical support for predicting the customer's willingness to apply for a credit card; the details are as follows:
[0007] Get credit card marketing sample data;
[0008] Divide the sample data into the original training set and the original testing set according to the custom ratio;
[0009] Divide the original training set into a new training set Train and a new test set Test according to a custom ratio;
[0010] Use the new training set Train to train the first layer model respectively;
[0011] Use cross-validation and Bayesian optimization algorithms to perform appropriate parameter adjustment and hyperparameter tuning, and retrain the model using the best hyperparameter combination;
[0012] Use the new test set Test for prediction, and combine the prediction outputs of each basic model into new features of the training set train;
[0013] Combine the new features of the training set train and the labels of the new test set Test into a labeled training set train; and use train to train the second-layer model;
[0014] Use cross-validation and network search methods to adjust and optimize the parameters of the second-layer model, and retrain the model using the best hyperparameter combination;
[0015] Use the trained first-layer model to predict the original test set Testing Set and output the original prediction results;
[0016] The original prediction results output by the original test set Testing Set are used as the test set of the second-layer model for prediction, and the final prediction results are output.
[0017] As a preferred method, the credit card marketing sample data is obtained as follows:
[0018] Collect relevant data of bank customers;
[0019] Pre-process the relevant data collected from bank customers: including data cleaning, deleting outliers, filling missing values, and data normalization;
[0020] Perform feature engineering on the indicator system.
[0021] Preferably, the sample data includes basic customer information, location data, traffic data, communication data, and portrait data from the operator or bank;
[0022] The sample objects are bank customers. The positive samples are customers who answered the previous credit card telephone marketing calls and the call duration was more than 10 seconds; the negative samples are customers who did not answer the previous credit card telephone marketing calls or the call duration was less than 10 seconds.
[0023] The sample characteristics are customer communication indicators, location indicators and traffic indicators on the operator side, and customer banking app and mini-program usage indicators on the bank side.
[0024] Preferably, feature engineering is performed on the indicator system as follows:
[0025] Feature selection: Calculate the variance of each feature, perform variance filtering on features whose variance is less than the set threshold, and reduce the feature dimension;
[0026] Feature construction: Construct an app usage frequency indicator based on the number of times the app is used and the usage time.
[0027] Preferably, the first layer models include LightGBM, XGBoost, DNN and SVM; LightGBM, XGBoost, DNN and SVM are trained using the new training set Train, and 5-fold cross validation and Bayesian optimization algorithm are used to perform appropriate parameter adjustment and hyperparameter tuning, and the model is retrained and saved using the best hyperparameter combination.
[0028] Preferably, the second-layer model adopts a logistic regression model, and is trained using new features composed of the prediction output combination of the first-layer model. The parameters of the logistic regression model are adjusted and tuned using 5-fold cross-validation and grid search methods, and the model is retrained and saved using the best hyperparameter combination.
[0029] As a preference, use the trained logistic regression model to predict the test set and output the final prediction results as follows:
[0030] Through the sigmoid function, a probability value between 0 and 1 is output. The formula is as follows:
[0031]
[0032] Among them, z=β0+β1x1+β2x2+...+β n x n , which represents the result of weighted summation of all features.
[0033] A credit card application intention prediction system based on multi-model fusion, the system is used to implement the credit card application intention prediction method based on multi-model fusion as mentioned above; the system comprises:
[0034] Collection module, used to obtain credit card marketing sample data;
[0035] Partition module 1 is used to divide the sample data into the original training set and the original test set according to a custom ratio;
[0036] Partition module 2 is used to divide the original training set Training Set into a new training set Train and a new test set Test according to a custom ratio;
[0037] Training module 1 is used to train the first layer model using the new training set Train;
[0038] Tuning module 1 is used to perform appropriate parameter adjustment and hyperparameter tuning using cross-validation and Bayesian optimization algorithms, and retrain the model using the best hyperparameter combination;
[0039] Prediction module 1 is used to make predictions using the new test set Test and combine the prediction outputs of each basic model into new features of the training set train;
[0040] Training module 2 is used to combine the new features of the training set train and the labels of the new test set Test into a labeled training set train, and use train to train the second-layer model;
[0041] Tuning module 2 is used to adjust and optimize the parameters of the second-layer model using cross-validation and network search methods, and retrain the model using the best hyperparameter combination;
[0042] Prediction module 2 is used to use the trained first-layer model to predict the original test set Testing Set and output the original prediction results;
[0043] Prediction module three is used to predict the original prediction results output by the original test set Testing Set as the test set of the second-layer model and output the final prediction results.
[0044] An electronic device comprising: a memory and at least one processor;
[0045] Wherein, the memory stores a computer program;
[0046] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the credit card application intention prediction method based on multi-model fusion as described above.
[0047] A computer-readable storage medium having a computer program stored therein, wherein the computer program can be executed by a processor to implement the credit card application intention prediction method based on multi-model fusion as described above.
[0048] The credit card application intention prediction method and system based on multi-model fusion of the present invention has the following advantages:
[0049] (i) The ensemble learning method of the present invention can reduce the overfitting risk of a single model by combining the predictions of multiple models, improve the generalization ability and prediction performance of the model, better adapt to various data distributions, and the diversity of the model also helps to capture more information;
[0050] (ii) The present invention is used to identify the user's willingness to apply for a credit card, and to provide support for the accurate selection of credit card marketing targets. The blending method is used to integrate multiple different types of single machine learning models, thereby improving the performance of the overall model;
[0051] (III) The present invention selects basic information data, location data, traffic data, communication data, etc. of customers from operators or banks. The indicator system is diversified, and the relationship between indicator features and labels is complex. Using only a single model to predict credit card application willingness cannot meet the processing of different data types or features;
[0052] (iv) The present invention combines different types of machine learning techniques through the Blending ensemble learning method, which can not only reduce the risk of overfitting and enhance the generalization ability of the model, but also process different types of features, reduce model bias, and improve prediction performance;
[0053] (V) The present invention first collects relevant samples, divides the sample data into an original training set Training Set and an original test set Testing Set, and divides the original training set Training Set into a new training set Train and a new test set Test; then, in the first-layer model, multiple different types of basic models are used to learn and train the new training set Train, and the trained basic models are used to predict the new test set Test to obtain multiple prediction results; secondly, these prediction results are combined into new features, and combined with the labels of the new test set Test to form the training set train of the second-layer model to train a secondary learner; finally, the first-layer model is used to predict the original test set Testing Set, and the prediction results are used as the features of the second-layer model test set and continue to be predicted using the second model. The results predicted by the second-layer model are output as the final results, which can more accurately predict the credit card application intentions of bank customers, thereby personalizing credit card marketing plans, improving marketing efficiency, and reducing marketing costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] The present invention is further described below in conjunction with the accompanying drawings.
[0055] Attached Figure 1 This is a flowchart of the credit card application intention prediction method based on multi-model fusion. DETAILED DESCRIPTION
[0056] The credit card application intention prediction method and system based on multi-model fusion of the present invention are described in detail below with reference to the drawings and specific embodiments of the specification.
[0057] Embodiment 1:
[0058] As attached Figure 1 As shown, this embodiment provides a credit card application intention prediction method based on multi-model fusion. The method adopts a parallel integration method Blending to train multiple models at the same time. By combining the predictions of multiple models, the integrated learning method Blending can smooth the errors of individual models, thereby providing better performance overall and providing technical support for predicting customers' credit card application intentions; the details are as follows:
[0059] S1. Obtain credit card marketing sample data;
[0060] S2, divide the sample data into the original training set Training Set and the original test set Testing Set according to the custom ratio;
[0061] S3, divide the original training set Training Set into a new training set Train and a new test set Test according to a custom ratio;
[0062] S4, use the new training set Train to train the first layer model respectively;
[0063] S5. Use cross-validation and Bayesian optimization algorithms to perform appropriate parameter adjustment and hyperparameter tuning, and retrain the model using the best hyperparameter combination;
[0064] S6, use the new test set Test to make predictions, and combine the prediction outputs of each basic model into new features of the training set train;
[0065] S7, combine the new features of the training set train and the labels of the new test set Test into a labeled training set train; and use train to train the second layer model;
[0066] S8. Use cross-validation and network search methods to adjust and optimize the parameters of the second-layer model, and retrain the model using the best hyperparameter combination;
[0067] S9, use the trained first-layer model to predict the original test set Testing Set and output the original prediction results;
[0068] S10: Use the original prediction results output by the original test set Testing Set as the test set of the second-layer model for prediction, and output the final prediction results.
[0069] The details of obtaining the credit card marketing sample data in step S1 of this embodiment are as follows:
[0070] S101. Collect relevant data of bank customers;
[0071] S102, pre-processing the collected bank customer data: including data cleaning, deleting outliers, filling missing values, and data normalization;
[0072] S103. Perform feature engineering on the indicator system.
[0073] The sample data in step S1 of this embodiment includes basic customer information, location data, traffic data, communication data, and portrait data from the operator or bank;
[0074] The sample objects are bank customers. The positive samples are customers who answered the previous credit card telephone marketing calls and the call duration was more than 10 seconds; the negative samples are customers who did not answer the previous credit card telephone marketing calls or the call duration was less than 10 seconds.
[0075] The sample characteristics are customer communication indicators, location indicators and traffic indicators on the operator side, and customer banking app and mini-program usage indicators on the bank side.
[0076] The feature engineering of the indicator system in step S103 of this embodiment is specifically as follows:
[0077] S10301, feature selection: calculate the variance of each feature, perform variance filtering on features whose variance is less than a set threshold, and reduce the feature dimension;
[0078] S10302. Feature construction: construct an app usage frequency index based on the number of times the app is used and the usage time.
[0079] In this embodiment, the division ratio of the training set and the test set in step S2 and step S3 is generally [0.5, 0.9].
[0080] The first layer of models in this embodiment includes LightGBM, XGBoost, DNN and SVM; LightGBM, XGBoost, DNN and SVM are trained using the new training set Train respectively, and 5-fold cross validation and Bayesian optimization algorithm are used to perform appropriate parameter adjustment and hyperparameter tuning, and the model is retrained and saved using the best hyperparameter combination.
[0081] The second-layer model in this embodiment adopts a logistic regression model, and is trained using new features composed of the prediction output combination of the first-layer model. The parameters of the logistic regression model are adjusted and tuned using a 5-fold cross-validation and grid search method, and the model is retrained and saved using the optimal hyperparameter combination.
[0082] In step S10 of this embodiment, the trained logistic regression model is used to predict the test set, and the final prediction result is output as follows:
[0083] Through the sigmoid function, a probability value between 0 and 1 is output. The formula is as follows:
[0084]
[0085] Among them, z=β0+β1x1+β2x2+...+β n x n , which represents the result of weighted summation of all features.
[0086] Embodiment 2:
[0087] This embodiment provides a credit card application intention prediction system based on multi-model fusion, which is used to implement the credit card application intention prediction method based on multi-model fusion as in Embodiment 1; the system includes:
[0088] Collection module, used to obtain credit card marketing sample data;
[0089] Partition module 1 is used to divide the sample data into the original training set and the original test set according to a custom ratio;
[0090] Partition module 2 is used to divide the original training set Training Set into a new training set Train and a new test set Test according to a custom ratio;
[0091] Training module 1 is used to train the first layer model using the new training set Train;
[0092] Tuning module 1 is used to perform appropriate parameter adjustment and hyperparameter tuning using cross-validation and Bayesian optimization algorithms, and retrain the model using the best hyperparameter combination;
[0093] Prediction module 1 is used to make predictions using the new test set Test and combine the prediction outputs of each basic model into new features of the training set train;
[0094] Training module 2 is used to combine the new features of the training set train and the labels of the new test set Test into a labeled training set train, and use train to train the second-layer model;
[0095] Tuning module 2 is used to adjust and optimize the parameters of the second-layer model using cross-validation and network search methods, and retrain the model using the best hyperparameter combination;
[0096] Prediction module 2 is used to use the trained first-layer model to predict the original test set Testing Set and output the original prediction results;
[0097] Prediction module three is used to predict the original prediction results output by the original test set Testing Set as the test set of the second-layer model and output the final prediction results.
[0098] Embodiment 3:
[0099] This embodiment also provides an electronic device, including: a memory and a processor;
[0100] Wherein, the memory stores computer-executable instructions;
[0101] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the credit card application intention prediction method based on multi-model fusion in any embodiment of the present invention.
[0102] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.
[0103] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.
[0104] Embodiment 4:
[0105] This embodiment also provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions are loaded by a processor, so that the processor executes the credit card application intention prediction method based on multi-model fusion in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.
[0106] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.
[0107] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.
[0108] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.
[0109] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.
[0110] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A credit card application intention prediction method based on multi-model fusion, characterized in that: This method uses the parallel integration method Blending to train multiple models simultaneously. By combining the predictions of multiple models, the integrated learning method Blending can smooth the errors of individual models, thereby providing better performance overall and providing technical support for predicting customers' willingness to apply for credit cards; the details are as follows: Get credit card marketing sample data; Divide the sample data into the original training set Training Set and the original test set TestingSet according to the custom ratio; Divide the original training set into a new training set Train and a new test set Test according to a custom ratio; Use the new training set Train to train the first layer model respectively; Use cross-validation and Bayesian optimization algorithms to perform appropriate parameter adjustment and hyperparameter tuning, and retrain the model using the best hyperparameter combination; Use the new test set Test for prediction, and combine the prediction outputs of each basic model into new features of the training set train; Combine the new features of the training set train and the labels of the new test set Test into a labeled training set train; and use train to train the second-layer model; Use cross-validation and network search methods to adjust and optimize the parameters of the second-layer model, and retrain the model using the best hyperparameter combination; Use the trained first-layer model to predict the original test set Testing Set and output the original prediction results; The original prediction results output by the original test set Testing Set are used as the test set of the second-layer model for prediction, and the final prediction results are output.
2. The credit card application intention prediction method based on multi-model fusion according to claim 1 is characterized in that: The details of obtaining credit card marketing sample data are as follows: Collect relevant data of bank customers; Pre-process the relevant data collected from bank customers: including data cleaning, deleting outliers, filling missing values, and data normalization; Perform feature engineering on the indicator system.
3. The credit card application intention prediction method based on multi-model fusion according to claim 1 or 2, characterized in that: Sample data includes basic customer information, location data, traffic data, communication data, and profile data from operators or banks; The sample objects are bank customers. The positive samples are customers who answered the previous credit card telephone marketing calls and the call duration was more than 10 seconds; the negative samples are customers who did not answer the previous credit card telephone marketing calls or the call duration was less than 10 seconds. The sample characteristics are customer communication indicators, location indicators and traffic indicators on the operator side, and customer banking app and mini-program usage indicators on the bank side.
4. The credit card application intention prediction method based on multi-model fusion according to claim 3 is characterized in that: The feature engineering of the indicator system is as follows: Feature selection: Calculate the variance of each feature, perform variance filtering on features whose variance is less than the set threshold, and reduce the feature dimension; Feature construction: Construct an app usage frequency indicator based on the number of times the app is used and the usage time.
5. The credit card application intention prediction method based on multi-model fusion according to claim 1 is characterized in that: The first layer of models includes LightGBM, XGBoost, DNN and SVM; LightGBM, XGBoost, DNN and SVM are trained using the new training set Train, and 5-fold cross validation and Bayesian optimization algorithm are used to make appropriate parameter adjustments and hyperparameter tuning, and the model is retrained and saved using the best hyperparameter combination.
6. The credit card application intention prediction method based on multi-model fusion according to claim 1 is characterized in that: The second-layer model uses a logistic regression model, which is trained using the new features composed of the predicted output of the first-layer model. The parameters of the logistic regression model are adjusted and optimized using 5-fold cross-validation and grid search methods. The model is retrained and saved using the best hyperparameter combination.
7. The credit card application intention prediction method based on multi-model fusion according to claim 1 is characterized in that: Use the trained logistic regression model to predict the test set and output the final prediction results as follows: Through the sigmoid function, a probability value between 0 and 1 is output. The formula is as follows: Among them, z=β0+β1x1+β2x2+...+β n x n , which represents the result of weighted summation of all features.
8. A credit card application intention prediction system based on multi-model fusion, characterized in that: The system is used to implement the credit card application intention prediction method based on multi-model fusion as described in any one of claims 1 to 7; the system comprises: Collection module, used to obtain credit card marketing sample data; Partition module 1 is used to divide the sample data into the original training set and the original test set according to a custom ratio; Partition module 2 is used to divide the original training set Training Set into a new training set Train and a new test set Test according to a custom ratio; Training module 1 is used to train the first layer model using the new training set Train; Tuning module 1 is used to perform appropriate parameter adjustment and hyperparameter tuning using cross-validation and Bayesian optimization algorithms, and retrain the model using the best hyperparameter combination; Prediction module 1 is used to make predictions using the new test set Test and combine the prediction outputs of each basic model into new features of the training set train; Training module 2 is used to combine the new features of the training set train and the labels of the new test set Test into a labeled training set train, and use train to train the second-layer model; Tuning module 2 is used to adjust and optimize the parameters of the second-layer model using cross-validation and network search methods, and retrain the model using the best hyperparameter combination; Prediction module 2 is used to use the trained first-layer model to predict the original test set Testing Set and output the original prediction results; Prediction module three is used to predict the original prediction results output by the original test set Testing Set as the test set of the second-layer model and output the final prediction results.
9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the credit card application intention prediction method based on multi-model fusion as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the credit card application intention prediction method based on multi-model fusion as described in any one of claims 1 to 7.