Customer data analysis method

Abstract

The invention discloses a customer data analysis method. The data analysis method comprises the following steps: data acquisition and integration; performing data exploration through a statistical analysis method and machine training; a Logistic regression model is established for a plurality of analysis indexes finally entering the model, the model is evaluated through ROC, KS and lifting degree statistical indexes, and meanwhile cross validation is conducted through data of different time windows; according to a customer list provided by the branch customer, relevant information of the customer is generated; according to a specific customer, providing a graph of a related transaction; and trading network customer ranking. The accuracy of analysis and evaluation is improved, a large amount of time is saved for customer obtaining evaluation and resource ratio analysis work, and customer obtaining work is more rigorous and efficient.

Description

Technical Field

[0001] The present invention relates to the field of big data analysis, and in particular to a customer data analysis method. Background Art

[0002] With the continuous development of banking business, the amount of data has increased dramatically, and traditional data processing and analysis methods are difficult to meet the needs. The emergence of big data technology has provided new opportunities for bank data analysis, but how to quickly and effectively integrate and utilize data to achieve goals such as precision marketing, risk control and management decision-making.

[0003] Existing data analysis: mainly used in enterprise internal decision support, business analysis and report generation scenarios. For example, financial analysis, sales analysis, market research, etc. Statistical analysis, data mining and other methods are mainly used to perform descriptive analysis, predictive analysis and diagnostic analysis on data. Relational databases are usually used for data storage and management. The data storage structure is relatively fixed and has limited scalability. Data management mainly relies on manual maintenance and entry by database administrators and business personnel or data entry by front-end systems.

[0004] Disadvantages of current analytical systems and methods in the industry:

[0005] The amount of data processed is relatively small, and the data sources are relatively limited, mainly structured data within the enterprise, such as transaction data and financial data in the database.

[0006] It is mainly used for decision support, business analysis and report generation within the enterprise. The application scenario is too single and simple, and cannot meet the existing large-scale precision marketing and full-life cycle marketing methods.

[0007] Analysis methods based on rules and statistical models, such as ratio analysis, trend analysis, regression analysis, etc. These methods are relatively simple and have limited ability to identify and predict complex risks. Summary of the invention

[0008] In view of the above problems, the present invention is proposed to provide a customer data analysis method that overcomes the above problems or at least partially solves the above problems.

[0009] According to one aspect of the present invention, a customer data analysis method is provided, the data analysis method comprising:

[0010] Data collection and integration;

[0011] Statistical analysis methods and machine training for data exploration;

[0012] For several analytical indicators that finally entered the model, a Logistic regression model was established, and the model was evaluated through ROC, KS and lift statistical indicators, and cross-validation was performed using data from different time windows;

[0013] Generate relevant information about customers based on the customer list provided by branch customers;

[0014] Provide graphics of relevant transactions based on specific customers;

[0015] Trading network customer rankings.

[0016] Optionally, the data collection specifically includes:

[0017] The data lake is used to complete data collection from multiple systems, and data federation technology is used to complete data collection from different databases of multiple systems. Big data is used to complete data integration and aggregation from different data channels, fields, and field types.

[0018] Optionally, the data integration specifically includes:

[0019] Acquire data from enterprise-level data warehouses and external data sources, and clean, convert and integrate them into customer historical inventory and real-time incremental transaction processing models. Create public indicators based on customer basic information, behavioral preferences, product preferences, and risk information to facilitate subsequent data mining.

[0020] Optionally, the statistical analysis method and machine training for data exploration specifically include:

[0021] Training data organization: organize customer behavior data into different sample data for the "customer acquisition model" and clean abnormal sample data. Exclude explanatory variables that have no effect on the model through statistical methods of variable dimension reduction and data reduction, and find multicollinearity between multiple explanatory variables;

[0022] Model training: select the machine learning algorithm that is suitable for the model, and use different training data to train and generate a "big customer acquisition model";

[0023] Use the deep analysis algorithm screening method to select the optimal algorithm for the "customer acquisition model".

[0024] Optionally, the method of using the deep analysis algorithm to select the optimal algorithm for the "big customer acquisition model" specifically includes:

[0025] Use SciKit-learn's algorithm selection method to preliminarily screen out multiple machine learning algorithms that meet the requirements;

[0026] For different initial screening algorithms, the same training set is used to train and generate multiple reference models. Then, comprehensive consideration and in-depth analysis are conducted through model test results, algorithm average performance value, recall rate and other indicators to determine the optimal machine learning algorithm.

[0027] Optionally, the statistical analysis method and machine training for data exploration also include: model prediction, predicting the "fund transmission intermediary" and "fund transmission tightness" models through the model, and using the SHAP method to interpret and analyze the prediction results.

[0028] Optionally, the generated customer-related information specifically includes: corporate portrait, three-level upstream data, and three-level downstream data.

[0029] Optionally, the graphics of the related transactions specifically include: graphics of related primary networks, secondary networks, and tertiary networks, which are displayed through Cytoscape.

[0030] Optionally, the transaction network customer ranking specifically includes: obtaining a corresponding overall customer ranking based on percentile ranking of customers' closeness, betweenness, number of transactions, and transaction amount.

[0031] The present invention provides a customer data analysis method, which includes: data collection and integration; statistical analysis methods and machine training for data exploration; establishing a Logistic regression model for several analysis indicators that finally enter the model, evaluating the model through ROC, KS and lift statistical indicators, and cross-validating with data from different time windows; generating customer-related information based on the customer list provided by branch customers; providing graphics of related transactions based on specific customers; and ranking transaction network customers. The accuracy of analysis and evaluation is improved, a lot of time is saved for customer acquisition evaluation and resource allocation analysis, and customer acquisition work is more rigorous and efficient.

[0032] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0034] Figure 1 A flowchart of a customer data analysis method provided by an embodiment of the present invention;

[0035] Figure 2 An integrated flow chart for obtaining different customer information from different customer acquisition channels provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of data integration provided by an embodiment of the present invention;

[0037] Figure 4 A flowchart of an optimization model algorithm provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of constructing a corporate customer transaction network view provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0040] The terms "comprises" and "having" and any variations thereof in the description embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0041] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0042] like Figure 1 As shown, a customer data analysis method includes: data collection and integration; statistical analysis methods and machine training for data exploration; establishing a Logistic regression model for several analysis indicators that finally enter the model, evaluating the model through ROC, KS and lift statistical indicators, and cross-validating with data from different time windows; generating customer-related information based on the customer list provided by branch customers; providing graphics of related transactions based on specific customers; and ranking transaction network customers.

[0043] 1. Data collection and integration

[0044] 1.1 Data collection: Currently, the industry uses data lakes to complete multi-system data collection, applies data federation technology to complete data collection from multiple systems and different databases, and cooperates with big data to complete data integration and aggregation from different data channels, fields, and field types.

[0045] like Figure 2 As shown in the figure, the integration process of obtaining different customer information from different customer acquisition channels

[0046] 1.2 Data integration: Obtain data from enterprise-level data warehouses and external data sources, and clean, convert and integrate them into customer historical data and real-time incremental transactions for processing models, and create public indicators based on customer basic information, behavioral preferences, product preferences, risk information, etc. to facilitate subsequent data mining. Figure 3 As shown in the figure, the specific data mining diagram is shown.

[0047] 2. Data exploration: Through some statistical analysis methods and machine training, the quantiles, maximum and minimum values ​​of the indicator items are analyzed, and the significance of the indicator items is verified by single-factor frequency analysis and chi-square test. Through some statistical analysis methods, the quantiles, maximum and minimum values ​​of the indicator items are analyzed, and the significance of the indicator items is verified by single-factor frequency analysis and chi-square test.

[0048] 2.1 Training Data Arrangement

[0049] For the "big customer acquisition model", customer behavior data is organized into different sample data, and abnormal sample data is cleaned. Through statistical methods of variable dimension reduction and data reduction, explanatory variables that have no effect on the model are excluded, and multicollinearity between multiple explanatory variables is found to reduce the number of features that need to be extracted, so as to reduce the workload of model training and improve the efficiency of model training.

[0050] 2.2 Model Training

[0051] Filter out the machine learning algorithms that are suitable for the model, and use different training data to train and generate a “customer acquisition model”.

[0052] The present invention uses a deep analysis algorithm screening method to select the optimal algorithm for the "customer acquisition model" in two steps.

[0053] Step 1: Use SciKit-learn's algorithm selection method to preliminarily screen out multiple machine learning algorithms that meet the requirements.

[0054] Step 2: For different initial screening algorithms, use the same training set to train and generate multiple reference models. Then, comprehensively consider and conduct in-depth analysis through indicators such as model test results, algorithm average performance value and recall rate to determine the optimal machine learning algorithm.

[0055] The present invention draws on the idea of ​​mimetic decision-making to establish a multi-model collaborative decision-making mechanism. Multiple models are selected through redundant construction to build a model decision alliance, and the sample data to be predicted are simultaneously imported into multiple models in the decision alliance. Coordinated decisions are made by comparing the output results of each model to avoid special case data from having a significant impact on the evaluation results, so as to ensure that the final conclusion is more scientific and objective.

[0056] The process of optimizing model algorithms, such as Figure 4 As shown;

[0057] 2.3 Model prediction

[0058] The model is used to make predictions on models such as "fund transmission intermediary" and "fund transmission tightness", and the SHAP method is used to explain and analyze the prediction results.

[0059] 3. Model Validation:

[0060] For the final analysis indicators that enter the model, a Logistic regression model is established. The model is evaluated through statistical indicators such as ROC, KS and lift, and cross-validated using data from different time windows. Cooperate with various marketing departments, adapt marketing strategies, select 2 to 3 institutions, circle the number of marketing leads after model scoring (for example: TOP30%, TOP50%...), conduct A / B-test, and follow up on the execution effect.

[0061] The specific implementation process takes the construction of the corporate customer transaction network view as an example:

[0062] like Figure 5 As shown in the figure, the processing of corporate transaction network includes six steps: basic data processing, transaction behavior ranking analysis, transaction network data processing, corporate customer view, customer transaction network graph analysis, and transaction network customer ranking. Specific operations include data loading, ETL processing logic, script execution order, etc. Through the screening, aggregation and processing of data, the corporate customer information table and customer-level data summary are obtained. The statistics of capital inflow and outflow of pure other-bank customers and other-bank customer accounts and our bank customers are analyzed. The indicators related to the calculation graph are constructed and analyzed; the overall ranking of customers is obtained according to the percentile ranking of the relevant indicators of customers.

[0063] 1. Basic data processing: including source data screening and filtering, core transaction code screening, elimination of some counterparties, customer name conversion and other basic data operations, unified processing for the whole bank, without branch characteristics; (see etl processing steps)

[0064] 2. Transaction behavior ranking analysis: mainly analyzes the inflow and outflow statistics of funds between other bank customers and other bank customer accounts and our bank customers.

[0065] 3. Transaction network data processing: Export the transaction relationships in the database, generate closeness and betweenness indicators through machine learning, and import the results into the database for unified processing.

[0066] Betweenness of Funds: Measures the ability of the client to act as a fund channel in the network. This indicator can measure the role of the client as a channel or conduit.

[0067] Closeness of funds transmission: measures the closeness of the client’s fund transactions with other clients in the network. This indicator measures the centrality of the client’s fund flow in the entire network.

[0068] 4. Corporate customer view: Generate relevant customer information based on the customer list provided by the branch customers, including corporate portraits, upstream three-level data, and downstream three-level data.

[0069] 5. Customer transaction network graphic analysis: Based on specific customers, provide graphics of relevant transactions, including graphics of relevant primary networks, secondary networks, and tertiary networks, and display them through Cytoscape.

[0070] 6. Trading network customer ranking: Based on the percentile ranking of customers’ closeness, betweenness, number of transactions, and transaction amount, we can get the corresponding overall customer ranking. We can issue a ranking for the entire bank or a ranking for each branch.

[0071] To help enterprises find transaction network structures that are meaningful to specific business scenarios and identify potential targets for precision marketing.

[0072] 1. Improve customer data quality by building a big customer model and customer behavior model and establishing a quality verification mechanism. Optimize the verification result data into business indicators, improve the accuracy and efficiency of data analysis, and provide strong support for industry decision-making.

[0073] 2. Achieve precision marketing through data analysis, mining and machine training, improve customer satisfaction and loyalty, strengthen risk control, reduce credit risk and losses, facilitate the diversification of business data application scenarios, and facilitate enterprises to achieve multi-dimensional decision-making.

[0074] 3. Solve the problem that the scenarios of enterprise data application are too single and simple, and cannot meet the existing problem of large-scale precision marketing and full-life cycle marketing methods, so that enterprises can more easily acquire customers and conduct precision marketing.

[0075] Beneficial effects: Promote the improvement of work productivity. Compared with the traditional working mode, the intelligent evaluation model of information innovation based on machine learning can incorporate more dimensions and constraints in the work of information innovation analysis and judgment, improve the accuracy of analysis and evaluation, save a lot of time for customer acquisition evaluation and resource allocation analysis, and make customer acquisition work more rigorous and efficient.

[0076] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A customer data analysis method, characterized in that: The data analysis method comprises: Data collection and integration; Statistical analysis methods and machine training for data exploration; For several analytical indicators that finally entered the model, a Logistic regression model was established, and the model was evaluated through ROC, KS and lift statistical indicators, and cross-validation was performed using data from different time windows; Generate relevant information about customers based on the customer list provided by branch customers; Provide graphics of relevant transactions based on specific customers; Trading network customer rankings.

2. A customer data analysis method according to claim 1, characterized in that: The data collection specifically includes: The data lake is used to complete data collection from multiple systems, and data federation technology is used to complete data collection from different databases of multiple systems. Big data is used to complete data integration and aggregation from different data channels, fields, and field types.

3. A customer data analysis method according to claim 1, characterized in that: The data integration specifically includes: Acquire data from enterprise-level data warehouses and external data sources, and clean, convert and integrate them into customer historical inventory and real-time incremental transaction processing models. Create public indicators based on customer basic information, behavioral preferences, product preferences, and risk information to facilitate subsequent data mining.

4. A customer data analysis method according to claim 1, characterized in that: The statistical analysis method and machine training for data exploration specifically include: Training data organization: organize customer behavior data into different sample data for the "customer acquisition model" and clean abnormal sample data. Exclude explanatory variables that have no effect on the model through statistical methods of variable dimension reduction and data reduction, and find multicollinearity between multiple explanatory variables; Model training: select the machine learning algorithm that is suitable for the model, and use different training data to train and generate a "big customer acquisition model"; Use the deep analysis algorithm screening method to select the optimal algorithm for the "customer acquisition model".

5. A customer data analysis method according to claim 4, characterized in that: The method of using the deep analysis algorithm to select the optimal algorithm for the "big customer acquisition model" specifically includes: Use SciKit-learn's algorithm selection method to preliminarily screen out multiple machine learning algorithms that meet the requirements; For different initial screening algorithms, the same training set is used to train and generate multiple reference models. Then, comprehensive consideration and in-depth analysis are conducted through model test results, algorithm average performance value, recall rate and other indicators to determine the optimal machine learning algorithm.

6. A customer data analysis method according to claim 1, characterized in that: The statistical analysis method and machine training for data exploration also include: model prediction, predicting the "fund transmission intermediary" and "fund transmission tightness" models through the model, and using the SHAP method to explain and analyze the prediction results.

7. A customer data analysis method according to claim 1, characterized in that: The generated customer-related information specifically includes: corporate portrait, upstream three-level data, and downstream three-level data.

8. A customer data analysis method according to claim 1, characterized in that: The graphics of the related transactions specifically include: graphics of the related primary network, secondary network, and tertiary network, which are displayed through Cytoscape.

9. A customer data analysis method according to claim 1, characterized in that: The transaction network customer ranking specifically includes: obtaining the corresponding overall customer ranking based on the percentile ranking of the customer's closeness, betweenness, number of transactions, and transaction amount.

Images