A fusion analysis method for financial big data
Through the integrated analysis method of financial big data, a multi-dimensional difference matrix and group division are constructed, differentiated marketing and risk control strategies are generated, and the data collection and analysis problems in consumer loan business are solved, and risk identification is achieved. and efficiency improvement.
Patent Information
- Application Number
- CN202410989674.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-07-23
AI Technical Summary
The lack of a unified data collection and management mechanism in consumer loan business has led to uneven data quality and difficulty in conducting comprehensive analysis and mining, which has affected business efficiency and risk control.
Using the financial big data fusion analysis method, by obtaining consumer loan business data indicators of loan intermediaries, building a multi-dimensional difference matrix, using cosine similarity algorithm and K-means clustering algorithm to divide loan intermediaries groups, and generating differentiated marketing solutions and trends Control strategy, combining orthogonal table experimental design and conditional random field algorithm, build application materials identification model and risk control approval rules, and use LightGBM and Apriori algorithms for risk prediction and early warning.
It has achieved comprehensive data collection and analysis of all links of consumer loan business, identified and controlled business risk points, improved overall efficiency and security, optimized channel management, and improved the quality and profitability of consumer loan business.
Smart Images

Figure CN118552303B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of information technology, and in particular to a method for fusion analysis of financial big data. Background Art
[0002] There is a key technical problem in the consumer loan business, that is, how to effectively collect and analyze data from all aspects of the consumer loan business to identify efficiency bottlenecks and risk points; currently, the consumer loan business involves multiple links such as application, approval, loan issuance, and repayment, and each link will generate a large amount of data, such as the applicant's personal information, credit record, vehicle model, consumer loan amount, repayment record, etc. These data are scattered in different systems and departments, lacking a unified collection and management mechanism, resulting in uneven data quality and difficulty in comprehensive analysis and mining. In addition, there are also large differences in the consumer loan business of different loan intermediaries. For example, indicators such as consumer loan business volume, customer quality, and overdue rate all need to be carefully analyzed and evaluated in order to optimize and manage the channels. However, due to the lack of standardized data collection and analysis processes, data from different channels are difficult to compare and evaluate horizontally, affecting the overall efficiency and risk control of the consumer loan business. Therefore, it is urgent to establish a complete data collection and analysis system, and through technical means such as data integration, data cleaning, and data modeling, standardize and integrate the data from each link to form a unified data view and analysis dimension. On this basis, we use big data analysis, machine learning and other technologies to conduct a comprehensive analysis and exploration of each link and channel of the consumer loan business, identify efficiency bottlenecks and risk points, and provide optimization suggestions and decision-making support, so as to improve the overall efficiency and risk control level of the consumer loan business. Summary of the invention
[0003] The present invention provides a method for fusion analysis of financial big data, which mainly includes:
[0004] Obtain the consumer loan business data indicators of loan intermediaries, including the distribution information of the number of consumer loan applications, the proportion of customer credit score ranges, the distribution of consumer loan amount ranges, the proportion of consumer loan term preferences, the proportion of differentiated mortgage product types, the proportion of repayment method preferences, the distribution of overdue rate levels and the distribution characteristics of overdue duration. Use the cosine similarity algorithm to calculate the difference values of loan intermediaries in consumer loan business data indicators and construct a multi-dimensional difference matrix. Based on the multi-dimensional difference matrix, use the K-means clustering algorithm based on Euclidean distance to cluster all loan intermediaries and obtain different loan intermediary groups. Analyze the distribution of the number of consumer loan applications and customer credit scores of loan intermediary groups. The risk level of the loan intermediary group is determined by the proportion of consumer loan terms, the proportion of differentiated mortgage product types, the proportion of repayment method preferences and the risk level of different loan intermediary groups, and the orthogonal table experimental design method is used to generate the optimal consumer loan product solution that matches the loan intermediary group, including the amount, term, interest rate and repayment method of the consumer loan; based on the optimal consumer loan product solution, the conditional random field algorithm is used to build an application material recognition model, perform structured extraction of the application materials submitted by the loan intermediary, identify key element information through named entity recognition, and compare it with the preset application template. The bank compares the information and uses the edit distance algorithm to identify missing or non-compliant information, and generates an application review report. Based on the application review report, the distribution characteristics of the customer overdue rate level and overdue duration of different loan intermediaries are obtained, and a differentiated configuration model for risk control approval rules is established. According to the customer credit score interval proportion, consumer loan amount interval distribution and mortgage product type differentiation proportion, feature selection is performed through the Gini index, and the approval rule thresholds for different loan intermediary groups are set differently. The LightGBM algorithm is used to integrate the customer historical consumer loan performance data of each loan intermediary, and the risk feature set is extracted from the customer historical consumer loan performance data through feature engineering. After processing the risk feature set with mini-max normalization, grid search is used to optimize the model hyperparameters, train the risk assessment model, and make risk predictions for the target loan intermediary's customer consumer loan applications. The prediction results are used to dynamically adjust the approval rule thresholds. According to the distribution of customer delinquency rates of each loan intermediary, loan intermediaries whose monthly delinquency rates exceed the preset thresholds and whose overdue amounts exceed the preset amounts are subject to key monitoring. An overdue warning model based on Apriori association rule mining is used to mine overdue risk signals from multi-dimensional business data. If the number of warning triggers is greater than the preset warning threshold, a warning notification is sent to the loan intermediary through the message middleware to trigger the start of the collection process.
[0005] The technical solution provided by the embodiment of the present invention may have the following beneficial effects:
[0006] The present invention provides a method for fusion analysis of financial big data, which analyzes key indicators such as the number of consumer loan applications, customer credit score ranges, consumer loan amounts and term distributions of loan intermediaries, and constructs a multi-dimensional difference matrix. Based on these matrices, loan intermediaries are grouped to identify different business models and risk levels; according to the risk levels of different loan intermediary groups, the present invention customizes the optimal consumer loan product solution for each group through an orthogonal table experimental design method; in addition, combined with data such as the consumer loan term preferences and mortgage product types of loan intermediaries, corresponding risk control strategies and marketing plans are generated; the constructed application material recognition model performs structured extraction and key information recognition on the materials submitted by loan intermediaries, detects missing or non-compliant fillings in the materials, further improves approval efficiency and accuracy, and extracts key risk features and conducts risk assessment in combination with historical consumer loan performance data of loan intermediary customers. Combined with the real-time risk prediction results, the threshold of the approval rules is adjusted dynamically to flexibly respond to the risk changes of different customer groups; the overdue warning model is used to mine overdue risk signals from business data; key monitoring is implemented on loan intermediaries whose overdue rates and overdue amounts exceed the preset thresholds, and warning notifications are sent through the message middleware to trigger the collection process; in general, the method of financial big data fusion analysis provided by the present invention realizes comprehensive data collection and analysis of each link in the consumer loan business, effectively identifies and controls business risk points, and improves the overall efficiency and security of the consumer loan business. At the same time, through the comprehensive evaluation of key indicators such as the consumer loan business volume, customer quality, and overdue rate of different loan intermediaries, channel management is optimized, high-quality channels are supported, and high-risk channels are controlled, thereby improving the quality and profitability of the overall consumer loan business. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] Figure 1 The present invention is a flowchart of a method for fusion analysis of financial big data.
[0008] Figure 2 It is a schematic diagram of a method for fusion analysis of financial big data according to the present invention.
[0009] Figure 3 This is another schematic diagram of a method for fusion analysis of financial big data according to the present invention. DETAILED DESCRIPTION
[0010] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0011] like Figure 1-3In this embodiment, a method for fusion analysis of financial big data may specifically include:
[0012] S101. Obtain the consumer loan business data indicators of loan intermediaries, including the distribution information of the number of consumer loan applications, the proportion of customer credit score ranges, the distribution of consumer loan amount ranges, the proportion of consumer loan term preferences, the proportion of differentiated mortgage product types, the proportion of repayment method preferences, the distribution of overdue rate levels and the distribution characteristics of overdue duration. Use the cosine similarity algorithm to calculate the difference values of the consumer loan business data indicators of loan intermediaries and construct a multi-dimensional difference matrix.
[0013] Obtain the consumer loan business data of the loan intermediary, including the number of consumer loan applications, customer credit score, consumer loan amount, consumer loan term, mortgage product type, repayment method, overdue rate and overdue duration; pre-process the consumer loan business data, and obtain a consumer loan business data set through data cleaning and standardization; calculate the consumer loan business data indicators of the loan intermediary by using a statistical analysis method based on the consumer loan business data set, including the distribution information of the number of consumer loan applications, the proportion of customer credit score intervals, the distribution of consumer loan amount intervals, the proportion of consumer loan term preferences, the proportion of differentiated mortgage product types, the proportion of repayment method preferences, the distribution of overdue rate levels and the distribution of overdue duration; construct a multi-dimensional consumer loan business feature vector based on the consumer loan business data indicators; use a cosine similarity algorithm to calculate the similarity between different loan intermediaries on the consumer loan business feature vector, and obtain a similarity matrix between different loan intermediaries; calculate the difference values of the loan intermediaries on the consumer loan business data indicators based on the similarity matrix, and construct a multi-dimensional difference matrix based on the difference values.
[0014] Specifically, based on the obtained consumer loan business data of loan intermediaries, big data analysis technology is used, such as using K-means clustering algorithm to divide customer credit score intervals, using Apriori association rule mining algorithm to analyze the correlation between consumer loan amount and term, etc., and statistical analysis is performed on indicators such as the distribution of the number of consumer loan applications and the proportion of customer credit score intervals to obtain the distribution characteristics of each indicator. Through data mining algorithms, such as using Word2Vec algorithm to calculate the vector representation of loan intermediaries on different indicators, analyze the differences between different loan intermediaries in terms of consumer loan amount intervals, consumer loan term preferences, and differentiated proportions of mortgage product types, and determine the unique attributes of each loan intermediary on these indicators. Using the cosine similarity algorithm, the difference values of each loan intermediary on indicators such as repayment method preference, overdue rate level distribution, and overdue duration distribution are calculated, and a multi-dimensional difference matrix is constructed to quantify the degree of difference between different loan intermediaries. According to the multi-dimensional difference matrix, the density-based DBSCAN clustering algorithm is used, combined with the characteristic data of loan intermediaries in terms of geographical location, business scale, etc., to cluster and group loan intermediaries, and identify loan intermediary groups with similar consumer loan business characteristics. For each loan intermediary group, analyze their common characteristics in terms of the number of consumer loan applications, the proportion of customer credit score intervals, and the distribution of consumer loan amount intervals, and summarize the consumer loan business preferences and risk characteristics of this group. Design differentiated consumer loan products and risk control strategies for different loan intermediary groups. The specific implementation plan includes: designing differentiated consumer loan interest rates, approval processes, quota management and other strategies for different loan intermediary groups, and establishing corresponding risk control models and rule engines to achieve automated consumer loan decision-making and risk control. For example, if the credit scores of customers in a certain group are generally high, the consumer loan conditions can be appropriately relaxed; if the overdue rate of a certain group is high, it is necessary to strengthen post-loan management and control risks. Establish a loan intermediary consumer loan business portrait system, use graph database technology to model the various attributes and associations of loan intermediaries, and realize flexible and efficient portrait query and analysis. Integrate the consumer loan business characteristics, differentiated attributes, and group information of each loan intermediary to form a comprehensive loan intermediary portrait, providing data support for subsequent precision marketing, risk management and other work. After obtaining the consumer loan business data of loan intermediaries, the K-means clustering algorithm can be used to divide the customer credit score interval. First, the customer credit score is used as the feature variable, the number of clusters is set to 5, and the cluster center is iteratively optimized to finally obtain 5 credit score intervals: 300-500, 500-600, 600-700, 700-800 and 800-900. Then, the Apriori association rule mining algorithm is used to analyze the correlation between the amount and term of consumer loans.The minimum support is set to 0.05 and the minimum confidence is set to 0.8, and the confidence of the rule "For customers with a consumer loan amount between 500,000 and 1 million, the consumer loan term is usually 3-5 years" is 0.85. Then, the Word2Vec algorithm is used to calculate the vector representation of loan intermediaries in terms of consumer loan amount range, consumer loan term preference, and differentiated proportion of mortgage product types, and the cosine similarity algorithm is used to calculate the difference between loan intermediaries. In terms of differentiated proportion of mortgage product types, the first product of loan intermediary A accounts for 0.6, the second product accounts for 0.3, and the SUV of loan intermediary B accounts for 0.4 and the third product accounts for 0.5. The cosine similarity between the two is 0.8, and the difference value is 0.2. Finally, the difference values are constructed into a multi-dimensional difference matrix, and the DBSCAN clustering algorithm is used to set the radius parameter to 0.5 and the minimum sample number to 5 to identify three loan intermediary groups with similar consumer loan business characteristics. For groups with higher credit scores, the interest rate on consumer loans can be reduced by 0.5%, and the approval time can be shortened by 1 working day; for groups with higher delinquency rates, the interest rate on consumer loans can be increased by 1%, and the credit limit management can be adjusted from loose to strict. All strategy formulation and adjustments are automatically completed through risk control models and rule engines, realizing the automation of consumer loan decision-making and risk control. Finally, the loan intermediary consumer loan business portrait system uses the Neo4j graph database for modeling, integrates various data and analysis results, and forms a loan intermediary knowledge map containing multi-dimensional attributes and associations, supporting flexible and efficient portrait query and analysis.
[0015] S102. Based on the multi-dimensional difference matrix, the K-means clustering algorithm based on Euclidean distance is used to cluster all loan intermediaries to obtain different loan intermediary groups; the distribution of the number of consumer loan applications and the proportion of customer credit score ranges of loan intermediary groups are analyzed to determine the risk level of the loan intermediary groups, and differentiated marketing plans and risk control strategies are generated according to the risk level.
[0016] According to the multi-dimensional difference matrix, the K-means clustering algorithm based on Euclidean distance is adopted to continuously update the cluster center through iterative optimization to divide all loan intermediaries into several loan intermediary groups; the proportion of loan intermediary groups in the distribution of the number of consumer loan applications and the customer credit score range is analyzed, and the business characteristics of the loan intermediary groups, including the overall business scale and customer credit level, are determined by calculating the mean, median and quartile statistics of the distribution of the number of consumer loan applications and the proportion of the customer credit score range; the historical consumer loan business data of the loan intermediary groups are analyzed, and the risk level of the loan intermediary groups is determined by comparing with the preset benchmark level, and the risk factors in the risk level are identified; the division results, business characteristics and risk factors of each loan intermediary group are integrated into the loan intermediary portrait system to form a loan intermediary clustering system, and differentiated marketing plans and risk control strategies are generated according to the loan intermediary clustering system, and the differentiated marketing plans and risk control strategies are optimized according to business development and risk prevention and control, including adjusting the approval rate threshold and increasing the frequency of post-loan management.
[0017] Specifically, according to the existing multi-dimensional difference matrix, the K-means clustering algorithm based on Euclidean distance is used to continuously update the cluster center through iterative optimization, and finally all loan intermediaries are divided into several groups with similar business characteristics. For each determined loan intermediary group, its characteristics in terms of the distribution of the number of consumer loan applications and the proportion of customer credit score ranges are deeply analyzed. By calculating the mean, median, and quartile statistics of the indicators and combining the analysis of the distribution, the overall business scale and customer credit level of the group are judged. On the basis of clarifying the business characteristics of each loan intermediary group, its historical consumer loan business data is further analyzed. By comparing with the preset benchmark level, it is judged whether the risk status of the group is higher or lower than the average level, and the key risk factors with the most prominent risk characteristics are identified, such as the overdue rate and bad debt rate risk indicators. The division results, business characteristics, and risk characteristics information of each loan intermediary group are integrated into the loan intermediary portrait system to form a complete loan intermediary grouping system as the basis for the subsequent formulation of differentiated marketing plans and risk control strategies. Differentiated risk control strategies need to balance business development and risk prevention and control on the basis of comprehensive assessment of the risk status of loan intermediaries, and formulate more detailed risk control measures and standards, including adjusting the approval rate threshold and increasing the frequency of post-loan management. The loan intermediary portrait system should also include the basic attributes of loan intermediaries and various business indicator information, providing business personnel with a comprehensive view of loan intermediary information to help them better understand and serve customers. Through the clustering and differentiated operation of loan intermediaries, strong data and strategy support is provided for subsequent loan intermediary management. When clustering loan intermediaries, the K-means clustering algorithm can be used, the number of clusters K=5 can be set, and the Euclidean distance can be used as the similarity metric. By calculating the Euclidean distance between each loan intermediary and the cluster center, it is divided into the closest category, and the cluster center is continuously updated iteratively until the clustering result is stable. For example, the multi-dimensional difference vector of loan intermediary A is (0.2, 0.5, 0.8), and the vector of loan intermediary B is (0.3, 0.6, 0.9). The Euclidean distance between the two is 0.173, which is less than the distance to other cluster centers, so they are divided into the same group. When analyzing group characteristics, key indicators such as the number of consumer loan applications, the proportion of credit score intervals, and the overdue rate can be selected to calculate their mean, median and other statistics. For group G1, the mean number of consumer loan applications is 100 per month, the median is 90 per month, the proportion of credit scores above 600 points is 80%, and the overdue rate is 3%. Based on this, it can be judged that this group is mainly small and medium-sized consumer loans, with good customer qualifications and relatively low risks. However, the overdue rate of group G2 is as high as 12%, and the bad debt rate is 3 times the industry average, so it is identified as a high-risk group.In this regard, the system automatically matches the corresponding marketing strategy and risk control strategy. For example, G1 applies the strategy combination of "active marketing, moderate approval, and routine management", while G2 applies the strategy combination of "prudent marketing, strict approval, and intensive management". These strategies can be automatically generated and optimized through algorithms such as decision trees and association rules, and dynamically adjusted in combination with effect indicators such as marketing success rate and overdue rate. Finally, the above information is summarized into the loan intermediary portrait to form a comprehensive information view for business personnel to query and use. When the salesperson visits loan intermediary X, the system automatically pops up the portrait information: the group G1, the average monthly consumer loan application is 50, the customer has good credit, it is recommended to focus on marketing 100,000-300,000 yuan consumer loan products, and give a 0.2% interest rate discount, and the expected transaction probability can reach more than 30%.
[0018] S103. Based on the consumer loan term preference ratio, mortgage product type differentiation ratio, repayment method preference ratio and risk level of different loan intermediary groups, and through the orthogonal table experimental design method, generate the optimal consumer loan product solution that matches the loan intermediary group, including the consumer loan amount, term, interest rate and repayment method.
[0019] Generate differentiated consumer loan product solutions based on the differences in consumer loan term preferences, differentiated proportions of mortgage product types, and repayment method preference characteristics among loan intermediary groups; use the orthogonal table experimental design method to generate consumer loan product solutions based on four key factors: consumer loan amount, consumer loan term, consumer loan interest rate, and repayment method, through orthogonal table combination; evaluate the matching degree and risk level of consumer loan product solutions, and screen out consumer loan product solutions whose matching degree with the characteristics of the loan intermediary group is greater than a preset threshold; if the hierarchical analysis method is used to determine the weight of each evaluation indicator, then establish an evaluation indicator hierarchy model, compare the importance of indicators pairwise, construct a pairwise comparison matrix, calculate the eigenvalues and eigenvectors of the comparison matrix, and compare the weights of the evaluation indicators. The characteristic vectors are normalized to obtain indicator weights, and consistency tests are performed to generate clear scoring rules and standards, score the consumer loan product solutions, and calculate the comprehensive scores of the consumer loan product solutions in a weighted average manner, and select solutions with scores greater than a preset threshold as recommended solutions; establish an effect evaluation mechanism for differentiated products, select key indicators such as consumer loan application volume, loan amount, delinquency rate, and customer satisfaction, set quantitative evaluation goals, build data collection and calculation processes, and regularly evaluate the performance data and customer response data of product solutions; dynamically generate the optimal consumer loan product solution based on the actual business data and personalized needs of the loan intermediary, and achieve a match between the product solution and the needs of the loan intermediary.
[0020] Specifically, in order to better meet the personalized needs of different loan intermediary groups and improve the pertinence and effectiveness of consumer loan business, it is necessary to design differentiated consumer loan product solutions based on the differences in loan intermediary groups in terms of consumer loan term preferences, differentiated proportions of mortgage product types, and repayment method preferences. By analyzing the actual business data of different loan intermediary groups, it can be found that there are significant differences in term preferences, mortgage product types, and repayment methods. For example, the consumer loan term of group A is mainly concentrated in 2-3 years, and the preference for the first product is the highest, while the consumer loan term of group B is mostly within 1 year, and they prefer the second product. These differentiated characteristics and needs provide an important basis for differentiated product design. When designing differentiated consumer loan products, the orthogonal table experimental design method can be used to generate a series of differentiated consumer loan product solutions from the four key elements of consumer loan amount, consumer loan term, consumer loan interest rate, and repayment method through orthogonal table combination. Among them, the amount of consumer loans can be set to three levels: 50,000-100,000, 100,000-200,000, and 200,000-300,000; the term of consumer loans can be set to three levels: 1 year, 2 years, and 3 years; the interest rate of consumer loans can be set to three levels: 4.5%, 5.0%, and 5.5%; the repayment method can be set to two levels: equal principal and interest, and equal principal. Through orthogonal table experiments, the workload of scheme design can be effectively reduced, and the design coverage can be comprehensive and the combination can be uniform. After generating the product scheme set, it is necessary to comprehensively evaluate the matching degree and risk level of each scheme, and select the scheme that best fits the characteristics of the loan intermediary group and has the most controllable risk. The evaluation process can adopt a weighted scoring method, taking matching degree and risk level as two evaluation dimensions, and setting several evaluation indicators respectively. For example, matching degree can include term preference matching degree, similarity of mortgage product type proportion, etc., and risk level can include capital cost, default probability, etc., and weights are allocated according to the importance of indicators. For each indicator, set clear scoring rules and standards, score each solution one by one, and calculate the comprehensive score of each solution by weighted average, and finally select the solution with the highest score as the recommended solution. In the process of implementing differentiated product solutions, we should also pay attention to the collection and analysis of customer feedback, understand customers' satisfaction evaluation and improvement suggestions for product solutions through regular visits by account managers and online questionnaires, and systematically organize and analyze feedback to timely discover deficiencies and problems in actual application of products. At the same time, it is also necessary to establish an effect evaluation mechanism for differentiated products, select key indicators such as the number of consumer loan applications, loan amount, overdue rate, customer satisfaction, set quantitative evaluation goals, build data collection and calculation processes, regularly evaluate the performance and customer response of product solutions, and iterate and optimize product solutions for problems found in the evaluation, so as to achieve continuous improvement and enhancement of differentiated consumer loan products. In addition, the matching rules and implementation processes of differentiated product solutions and loan intermediary groups should be further refined.On the one hand, the system can automatically recommend suitable differentiated product solutions based on the characteristic labels of the loan intermediary group to achieve accurate product push; on the other hand, the system can dynamically generate customized product solutions based on the actual business data and personalized needs of the loan intermediary to achieve a fine match between the product solutions and the loan intermediary needs. Through the continuous optimization and promotion of differentiated consumer loan products, the financial service needs of different loan intermediary groups can be met more accurately and efficiently, and the market competitiveness and customer stickiness can be improved. When analyzing the differentiated characteristics of different loan intermediary groups, methods such as data visualization and cluster analysis can be used. For consumer loan term preferences, a distribution map of the proportion of different loan groups in each term interval can be drawn to intuitively compare the differences; for mortgage product type preferences, the preference index of different loan groups for each product type brand can be calculated to find out the most favored product type of each group. The preference index can be calculated according to the formula of (group proportion-total proportion) / total proportion, and the value is between -1 and 1. The larger the proportion, the higher the preference. At the same time, multiple feature dimensions can be combined, and clustering algorithms such as K-means can be used to characterize the differentiated characteristics of different loan groups from multiple dimensions, providing a reference for differentiated product design. When designing an orthogonal table experiment, you can use statistical software tools such as SPSS to automatically generate an orthogonal table based on the settings of factors and levels, and calculate the uniformity and representativeness of each solution. When evaluating the matching degree and risk level of product solutions, you can use the hierarchical analysis method to determine the weight of each evaluation indicator, use the Delphi method or expert scoring method to determine the score of each solution on each indicator, and then calculate the weighted comprehensive score. The matching degree indicators include term matching degree, such as weight 0.3, mortgage product type matching degree, such as weight 0.2, and repayment method matching degree, weight 0.2. The score of each indicator is between 1 and 5 points. The weighted matching score of a certain solution = 0.3×4+0.2×5+0.2×4=3 points. Through this quantitative evaluation method, the optimal solution can be screened out more accurately. When the system automatically matches and recommends product solutions, a series of matching rules can be set. For example, for loan intermediaries with a consumer loan term preference of one year, product solutions with a term of one year and an interest rate of less than 5% can be matched; or algorithms such as association rule mining and collaborative filtering can be used to make personalized recommendations for product solutions based on the loan intermediary's historical business data and preference characteristics, thereby realizing intelligent matching of supply and demand.
[0021] S104. Based on the optimal consumer loan product solution, an application materials recognition model is constructed through the conditional random field algorithm. The application materials submitted by the loan intermediary are structured and extracted. The key element information is identified through named entity recognition and compared with the preset application template. The edit distance algorithm is used to identify missing or non-compliant parts and generate an application review report.
[0022] According to the optimal consumer loan product solution, an application material recognition model is constructed through the conditional random field algorithm; a characteristic function is defined to characterize the correlation between each key element information in the application materials, and a conditional probability model is obtained based on the correlation to realize the serialization and annotation of the key element information; for the submitted application materials, the application material recognition model is used to structure the key element information of the application materials, and the key element information is identified by using named entity recognition; the extracted key element information is compared with the preset application template, and the difference between the extracted key element information and the preset application template is calculated; if the difference exceeds the preset threshold, it is determined that there are missing or non-compliant parts, and the location and type of the missing or non-compliant parts are marked; according to the location and type of the missing or non-compliant parts, the predefined modification template is filled in, an application review report is generated, and the applicant is instructed to revise and improve the materials;
[0023] Specifically, based on the existing multi-dimensional difference matrix and the characteristics of the loan intermediary group, an intelligent application material recognition model based on the conditional random field algorithm is constructed to automatically review the consumer loan application materials submitted by loan intermediaries. The conditional random field model characterizes the dependency relationship between the key elements in the application materials by defining a set of characteristic functions, and on this basis, a conditional probability model is established to achieve serialized annotation of key elements. Compared with other machine learning models, conditional random fields can better consider contextual information and are more suitable for processing structured text data. By annotating and training a large amount of historical application material data, a conditional random field model is established that can accurately identify the key elements in the application materials, including the applicant's name, consumer loan amount, consumer loan term, mortgage product type, repayment method and other key information. When new application materials are submitted, the model can perform structured extraction of the material content, automatically locate and identify each key element information using named entity recognition technology, and convert it into a structured data format. Named entity recognition can be achieved through rule-based, dictionary-based or machine learning methods. Among them, the machine learning-based method constructs a language model to perform sequence annotation on the words in the text, and recognizes named entities such as names of people, places, and institutions, with a high accuracy rate. In the review of application materials, key elements such as the applicant's name, consumer loan amount, and term are defined as named entities, and accurate extraction is achieved through named entity recognition. The output results of the intelligent material recognition model include information such as the content, location, and type of each key element, as well as a structured representation of the material, such as JSON format. On the basis of identifying key elements, the system also needs to conduct an intelligent review of the integrity and standardization of the application materials. By comparing the extracted key element information with the standardized application template, the edit distance algorithm is used to calculate the difference between the two. If the difference exceeds a certain threshold, it is determined that there are missing or non-compliant parts in the filling, and the specific problem location and type are automatically marked. For the identified problems, the system can automatically generate standardized modification suggestions to guide the applicant to correct and improve the materials, and at the same time output the missing or erroneous information of the materials, as well as the corresponding modification suggestions, to provide a data basis for the subsequent review report generation. Based on the review of individual application materials, the system can also determine the comprehensive risk level of the applicant's application based on the historical performance of the loan intermediary group to which the applicant belongs. The system uses a weighted average method to calculate the comprehensive risk score based on the risk score of the individual application materials and the historical overdue rate of the group to which the applicant belongs, and then determines the risk level accordingly. For example, the comprehensive risk score = (individual risk score × 0.7) + (overdue rate of the group to which the applicant belongs × 0.3). If the individual risk score is 0.6 and the overdue rate of the group to which the applicant belongs is 5%, then the comprehensive risk score = (0.6 × 0.7) + (0.05 × 0.3) = 0.435.If the applicant's loan intermediary group has a high comprehensive risk level, the review standards for their application materials will also be raised accordingly, reflecting differentiated risk management strategies, such as raising information integrity requirements and adding manual review links. By combining the intelligent identification and review of application materials with the characteristics of loan intermediary groups, it is possible to quickly screen and predict risks of application materials, and improve approval efficiency and accuracy. Finally, the system automatically generates a structured application review report based on the intelligent recognition results of the application materials. The review report should include basic information of the application materials, such as the applicant, application time, consumer loan amount, etc., a list of missing or incorrect materials and modification suggestions, and the comprehensive risk level of the materials (such as high, medium and low) and the basis for judgment. When generating the report, a template + data filling method is used to design a structured report template, and then the result data of intelligent identification and risk judgment is filled in the corresponding position to realize the automatic generation of the report. This application material review mechanism based on intelligent identification technology and loan intermediary group characteristics can improve the intelligence level of consumer loan business from the source of material review, realize the automation and differentiation of the approval process, and better control the risk of consumer loans. When building a conditional random field model, the application materials can be divided into different key fields, such as the applicant's name, consumer loan amount, consumer loan term, etc. Each field constitutes an observation sequence. Then, a set of feature functions are defined, such as "when the current field is the applicant's name, whether the previous field is 'applicant'", "when the current field is the consumer loan amount, whether the field content contains numbers and units", etc., and the dependency relationship and filling rules between key fields are characterized by feature functions. The conditional random field model is trained on a large amount of annotated data, the weight parameters of the feature function are learned, and a discriminant conditional probability model is formed. The training sample size is 10,000 application materials, each with an average of 20 fields. After 10 iterations of training, the field recognition accuracy can reach more than 95%. For named entity recognition, deep learning models such as BiLSTM-CRF can be used, pre-trained on a large-scale corpus, and then fine-tuned on the application material field data to build a high-precision named entity recognizer. Using 100,000 application materials for training, it can identify numerical entities such as consumer loan amount, term, interest rate, and text entities such as bank name and collateral name, with an overall recognition accuracy of over 90%. In terms of intelligent review of materials, the edit distance algorithm can be used to compare the difference between the filled-in content and the standard template. The edit distance threshold is set to 10. If the edit distance between the actual filled-in content and the template exceeds 10, it is judged to be incorrect or missing. At the same time, for the identified error fields, predefined rules can also be used for type judgment and prompt generation, such as "the consumer loan amount exceeds the product limit" and "the repayment date format is incorrect".When judging the risk of an application, the risk score of individual materials can be obtained by taking into account multiple factors, such as credit record, income level, debt ratio, etc., through weighted summation, and then combined with risk indicators such as the overdue rate of the group to which it belongs, and a risk prediction model is established through algorithms such as logistic regression to obtain a comprehensive risk score. If the individual score is 0.7 and the group overdue rate is 10%, the comprehensive risk score = 1 / (1+e^-(0.71.5+0.10.8-0.5)) = 65%, and the risk level of the application is "high".
[0024] S105. Based on the application review report, obtain the distribution characteristics of customer delinquency rates and delinquency duration of different loan intermediaries, establish a differentiated configuration model for risk control approval rules, and perform feature selection through the Gini index based on the customer credit score range proportions, consumer loan amount range distribution, and differentiated proportions of mortgage product types, and set differentiated approval rule thresholds for different loan groups.
[0025] According to the application review report, the overdue rate level distribution and overdue duration distribution characteristics of customers of different loan intermediaries are obtained, and the characteristic data are cleaned, converted and standardized to obtain a data set; through the feature engineering method, the key features related to the overdue risk of loan intermediary customers are extracted according to the data set, including the overdue rate mean, overdue rate variance, overdue duration mean and overdue duration variance, and a differentiated configuration model for risk control approval rules is constructed; the Gini index is used to select features for the customer credit score interval proportion, consumer loan amount interval distribution and mortgage product type differentiation proportion, and the value of each feature is calculated. The Gini index is used to judge its contribution to the overdue risk, and the five features with the highest degree of differentiation of overdue risk are selected as the input of the differentiated configuration model of risk control approval rules; for different loan intermediary groups, according to the feature selection results, the K-means clustering algorithm is used to divide the customer groups, and the clustering results are mapped with the actual overdue labels to obtain the risk level of each category, including high-risk groups, medium-risk groups and low-risk groups; for groups with different risk levels, differentiated approval rule thresholds are set according to the dimensions of consumer loan amount, customer credit score, mortgage rate and repayment ability.
[0026] Specifically, according to the data analysis results in the intelligent application review report, the distribution characteristics of the customer overdue rate level and overdue duration of different loan intermediaries are obtained as the basic data for establishing a differentiated configuration model for risk control approval rules. The Gini index is used to select features for potential influencing factors such as the customer credit score interval proportion, the consumer loan amount interval distribution, and the differentiated proportion of mortgage product types. By calculating the Gini index of each feature, its contribution to the overdue risk is judged, and the top 5 features with the highest degree of differentiation of overdue risk are selected as the input variables of the model, and the results of feature selection are used as the input of the subsequent clustering algorithm. For different loan intermediary groups, according to the feature selection results, the clustering algorithm is used to divide the customer groups into high-risk groups, medium-risk groups, and low-risk groups. The clustering results are used as labels and the feature selection results are used as feature variables to train decision trees or random forest models. According to the performance of the model on the training set and the test set, the fitting effect and generalization ability of the model are evaluated, and the decision logic of the model is explained according to the feature importance ranking. Among them, the model training uses the DecisionTreeClassifier or RandomForestClassifier class in the sklearn.tree or sklearn.ensemble package, sets appropriate parameters such as the depth of the tree, the minimum number of samples for leaf nodes, etc.), and fits the model on the training set. The model evaluation uses indicators such as classification accuracy, precision, recall, and F1 value on the test set, and selects the optimal parameter combination through cross-validation. After the model training is completed, the sample distribution on different leaf nodes is extracted from the decision tree or random forest model. According to the concentration of the distribution, the threshold of each feature is determined. For example, if the proportion of credit scores below 600 points exceeds 50%, 600 points is set as the threshold for credit score access. For groups with different risk levels, differentiated approval rule thresholds are set from dimensions such as consumer loan amount, customer credit score, mortgage rate, and repayment ability. According to historical data analysis, the default rate of consumer loans exceeding 300,000 yuan is 5%, and the default rate of loans below 100,000 yuan is 1%. Therefore, 100,000 yuan is set as the amount threshold for low-risk groups, and 300,000 yuan is set as the amount threshold for high-risk groups. Through Gini index analysis, when the mortgage rate ≥ 60%, the overdue rate is 2%; when the mortgage rate < 60%, the overdue rate rises to 8%. Therefore, 60% is set as the differentiated threshold for the mortgage rate. Those below 60% are regarded as high-risk groups and subject to stricter approval standards. For example, the upper limit of the consumer loan amount for high-risk groups is 80% of that for medium-risk groups, the lower limit of the credit score is 20 points higher than that for medium-risk groups, and the mortgage rate requirement is 10 percentage points higher than that for medium-risk groups.In the differentiated configuration model of risk control approval rules, the corresponding rule thresholds are automatically generated according to the overdue performance and characteristic distribution of different loan groups through decision trees or random forest algorithms, and the thresholds are dynamically adjusted and updated according to the actual business situation to ensure the matching degree between the risk control approval rules and the risk status of loan intermediaries. For high-risk groups, in addition to differentiated thresholds, they also need to be controlled from the aspects of approval level, credit period, and guarantee conditions. For example, the consumer loan applications of high-risk groups need to be approved by higher-level approval personnel, and the approval level is raised from account managers to risk control managers, risk control directors, etc.; the consumer loan period of high-risk groups is controlled, such as the consumer loan period of ordinary groups is 2-3 years, and the high-risk groups are shortened to less than 1 year; high-risk groups are required to provide more guarantee measures, such as real estate mortgages, personal guarantees and other credit enhancement measures in addition to commodity type mortgages. In the consumer loan approval process, the system automatically obtains the risk group information of the loan intermediary to which the applicant belongs, and verifies the key elements in the application materials one by one according to the differentiated approval rules of the group, such as whether the application amount exceeds the limit of the group, whether the credit score reaches the lower limit of access, whether the mortgage rate meets the requirements, etc., to form the rule verification results. If the application materials fail to pass the rule verification, they will be automatically judged as unsuccessful, and the reasons for the failure of the rules will be pushed to the applicant to guide them to modify or supplement the materials; if the application materials pass the rule verification, combined with the manual review opinions, the final approval result is determined to achieve differentiated access control driven by rules. Through the differentiated configuration model of risk control approval rules, it is possible to implement differentiated risk control policies that match loan intermediary groups with different risk conditions, strictly control high-risk groups from the capital end, and curb risks from the source. At the same time, more relaxed approval conditions are given to high-quality loan intermediary groups to promote the healthy development of the business and support the differentiated and refined development of the business under the premise of controllable risks. When selecting features, the Gini index, a commonly used feature importance evaluation method, can be used. Assume that the target variable of overdue rate has two values, including overdue and not overdue. If a certain feature such as credit score takes a specific value, the proportion of overdue samples is 60% and the proportion of not overdue samples is 40%, then the Gini index is 1-(0.6^2+0.4^2)=0.48. The larger the Gini index, the stronger the ability of the feature to distinguish different overdue risks. By calculating the Gini index of each feature, the top 5 features are selected as the input of the clustering algorithm. When clustering the feature selection results, the K-means algorithm can be used, and the number of clusters K is set to 3, such as high, medium, and low risks, and the silhouette coefficient is used as the evaluation indicator of clustering quality. Through multiple iterations, the similarity within the cluster is the highest and the difference between clusters is the largest. For the three risk groups formed by clustering, the decision tree model is trained separately, and the model evaluation is performed using a 10-fold cross validation method.For the decision tree model of high-risk groups, a higher leaf node purity can be set to control the growth of the tree, resulting in a simple and easy-to-interpret model. From the decision tree model, it can be found that when the credit score is <650, the consumer loan amount is >150,000, and the annual income is <100,000, the overdue probability is as high as 35%, thereby generating a differentiated approval rule: "For high-risk loan intermediaries, when the applicant's credit score is less than 650 points, the consumer loan amount exceeds 150,000, and the annual income is less than 100,000, the consumer loan application will be automatically rejected and prompted to provide additional risk mitigation measures." Through the risk control rule engine, this rule can be automatically embedded in the consumer loan approval process to achieve automation and intelligence of approval.
[0027] S106. Use the LightGBM algorithm to integrate the historical consumer loan performance data of customers of various loan intermediaries, extract the risk feature set from the historical consumer loan performance data of customers through feature engineering, process the risk feature set through min-max normalization, use grid search to optimize the model hyperparameters, train the risk assessment model, and predict the risks of consumer loan applications of customers of the target loan intermediaries. Use the prediction results for dynamic adjustment of the approval rule thresholds.
[0028] Based on the LightGBM algorithm, the historical consumer loan performance data of customers of various loan intermediaries are integrated to form a risk assessment data set, and the historical consumer loan performance data of customers include basic customer information, credit information, assets and liabilities, and behavioral preference data; through the feature engineering method, the key features related to credit risk are extracted according to the risk assessment data set to construct a risk feature set; the risk feature set is subjected to minimum-maximum normalization processing to map data of different scales and distributions into a preset interval; the hyperparameters of the risk assessment model are optimized by means of grid search, and the optimal parameter combination is obtained by selecting parameters by cross-validation through presetting different parameter combinations; the consumer loan application data of customers of various loan intermediaries are input into the risk assessment model to predict the default risk of each applicant and obtain a risk probability score; according to the risk probability score, the thresholds in the approval rules are dynamically adjusted; if the customer whose risk probability score is greater than the preset probability value is marked as a high-risk customer, if the proportion of the high-risk customers exceeds the preset threshold, the credit score threshold and the upper limit of the consumer loan amount are lowered to achieve dynamic control of risks.
[0029] Specifically, a risk assessment model based on the LightGBM algorithm is used to integrate the historical consumer loan performance data of each loan intermediary to form a comprehensive risk assessment data set. At the same time, the concept of rule engine is introduced. The rule engine is an engine that makes business decisions based on preset rules. It can automatically match the corresponding processing logic and operation process according to different rule conditions. In the field of risk control, the rule engine can be combined with the risk prediction model to trigger corresponding risk control actions such as approval, rejection, manual review, and pricing increase according to the prediction results of the model, so as to realize the automation and intelligence of risk prevention and control. Through the method of feature engineering, key features related to credit risk, such as age, education, occupation, income level, debt ratio, credit card usage frequency, consumption behavior, etc., are extracted from multi-source heterogeneous data such as applicant basic information, credit status, assets and liabilities, behavioral preferences, etc., to construct a risk feature set. For example, from credit card transaction data, we can extract characteristics such as customer consumption frequency, consumption amount, consumption category, etc., which reflect the customer's consumption ability, consumption habits and lifestyle; from Internet behavior data, we can extract characteristics such as customer browsing time, website type visited, search keywords, etc., which reflect the customer's interests, living conditions and information acquisition channels. These characteristics are closely related to credit risk. The extracted risk characteristics are processed by minimum-maximum normalization, and data of different scales and distributions are mapped to the interval [0,1] to achieve data standardization and unification. The grid search method is used to optimize the hyperparameters of the LightGBM model, such as learning rate, number of trees, maximum number of samples of leaf nodes, etc. The optimal parameter combination is selected by cross-validation to obtain a personalized risk scoring model with excellent performance. LightGBM is a gradient boosting algorithm based on decision trees. It constructs multiple decision trees and weights the prediction results of the trees to form the final prediction value. Compared with the traditional GBDT algorithm, LightGBM uses a histogram-based decision tree algorithm to divide continuous features into multiple intervals, greatly reducing the number of feature values and computational complexity; secondly, it uses a leaf growth strategy with depth restrictions to prevent overfitting of the decision tree and improve the generalization ability of the model; in addition, it also supports the direct use of categorical features without One-Hot encoding, which improves the utilization efficiency of features. Through these optimizations, LightGBM has significantly improved training speed and prediction accuracy. The customer consumer loan application data of each loan intermediary is input into the trained personalized risk scoring model, and the default risk of each applicant is predicted to obtain a risk probability score between 0 and 1. The higher the score, the greater the default risk.According to the risk probability score predicted by the model, the thresholds in the approval rules are adjusted dynamically. For example, when the proportion of high-risk applicants with a predicted default probability of >0.6 exceeds 20%, the credit score entry threshold is raised from 600 to 650, and the upper limit of the consumer loan amount is lowered from 200,000 yuan to 150,000 yuan to achieve dynamic risk control. According to historical data statistics, when the proportion of high-risk customers among approved applicants exceeds 20%, the overall bad debt rate will increase by 2 percentage points, affecting capital recovery and profitability. Through simulation calculations, this dynamic threshold can control the bad debt rate within 3%, while maintaining the approval rate above 80%, achieving a good balance between risk and efficiency. The performance evaluation indicators of the personalized risk scoring model include accuracy, recall, F1 value, AUC value, etc. Among them, AUC value is the most commonly used comprehensive evaluation indicator, which represents the model's ability to sort positive and negative samples. The value is between 0.5-1, and the closer it is to 1, the stronger the model's ability to distinguish. A warning threshold of AUC value can be set, such as 0.75. When the AUC value of the model is lower than the threshold for three consecutive months, the model retraining and optimization process is triggered. Regularly monitor the prediction effect of the personalized risk scoring model, statistically analyze the difference between the default probability predicted by the model and the actual default situation, calculate the various evaluation indicators of the model, and use new business data to retrain and optimize the model in time when the model performance declines to ensure the risk prediction ability of the model. In addition to using the model for approval and access control, the personalized risk scoring model can also be combined with other risk control strategies, such as applying the prediction results of the model in consumer loan pricing and quota management. According to the customer's risk level, set differentiated consumer loan interest rates and fees, charge higher interest spreads and fees to high-risk customers, and give certain price discounts to low-risk customers to achieve risk pricing; according to the customer's risk level and consumer loan performance, dynamically adjust their consumer loan quotas, appropriately increase the consumer loan quotas of low-risk customers who have repaid on time for three consecutive periods, and reduce the consumer loan quotas of high-risk customers who have overdue more than 2 times, or even cancel their consumer loan qualifications to achieve risk limit management. Combining personalized risk scoring models with pricing, quota and other strategies to form an automated and personalized risk control solution can better balance the relationship between risk control and business development. Linking the model prediction results with the rule engine, setting the processing logic of the prediction results in the rule engine, such as automatically putting the applicant on the high-risk blacklist when the predicted default probability exceeds 0.8, prohibiting their consumer loan applications, or requiring them to provide additional collateral, etc., to form an automated risk prevention and control mechanism. In the data fusion stage, the ETL technology of the data warehouse can be used to extract and convert the customer consumer loan data scattered in various loan intermediary systems in a unified format, and load it into the training set and test set of the risk assessment model.For the configuration of the rule engine, you can use the form of decision tables or decision trees to design and maintain different rule conditions and corresponding actions in a visual way, such as "IF risk score>0.8AND debt-to-income ratio>50%THEN reject consumer loan applications". In the feature engineering stage, you can discretize continuous variables (such as age) by binning, divide the value range into several intervals (such as below 20, 20-30, 30-40, and above 40), and then use indicators such as WOE (weight of evidence) and IV (information value) to measure the risk differentiation of each interval. Taking age as an example, through binning and WOE calculation, it is found that the WOE value of customers "above 40" is 1.2, which is a strong indication of default, while the WOE value of customers "20-30" is -0.8, which is a weak indication of default. These binning features can be used as inputs to the LightGBM model for the prediction of default risk.
[0030] S107. Based on the distribution of customer overdue rate levels of each loan intermediary, key monitoring is performed on loan intermediaries whose monthly overdue rate exceeds the preset threshold and whose overdue amount exceeds the preset amount. An overdue warning model based on Apriori association rule mining is used to mine overdue risk signals from multi-dimensional business data. If the number of warning triggers is greater than the preset warning threshold, a warning notification is sent to the loan intermediary through the message middleware to trigger the start of the collection process.
[0031] According to the distribution of customer delinquency rates of various loan intermediaries, loan intermediaries whose monthly delinquency rates exceed the preset threshold and whose overdue amounts exceed the preset amounts are identified and marked as key monitoring targets and included in the monitoring scope of the early warning model; an overdue early warning model based on the Apriori association rule mining algorithm is used to mine risk signal combinations with a correlation with overdue that is greater than a preset value from the customer's multi-dimensional business data; in the data processing stage, the customer's structured data, including basic customer information, consumer loan information, repayment records, and unstructured data, including credit report information, are extracted; for the unstructured data, natural language processing technology is used to extract key information, and it is associated and integrated with the structured data to form a customer profile; based on the Apriori association rule mining algorithm and the customer profile, overdue risk combination rules are mined; based on the overdue risk combination rules, the customer's credit report is processed and the credit report is processed. The confidences of the rules are sorted, and several rules with confidences greater than a preset threshold are selected as the rule base of the overdue warning model; for each rule in the rule base, the warning threshold is set by the quantile method, and the triggering probability of the rule is divided according to the preset quantile, and the rules with a triggering probability exceeding the preset quantile are marked as high-risk rules; when the number of triggering times or the triggering ratio of the rule exceeds the preset warning threshold, a warning signal of the corresponding level is generated; in a polling manner, batch processing and scanning of the key monitored loan intermediary customer data are performed regularly, and by matching with the rules in the rule base, the customer list that triggers the warning rule is identified, and divided into different risk levels according to the overdue risk degree of the warned customers; the warning signal and the customer list are pushed to the corresponding loan intermediary in real time through the message middleware, a warning notification message is generated, and the corresponding collection process is triggered.
[0032] Specifically, according to the distribution of customer overdue rates of each loan intermediary, the loan intermediaries with monthly overdue rates exceeding 20% or overdue amounts exceeding 5 million yuan, and which have triggered this standard for three consecutive months, are identified and marked as key monitoring targets and included in the monitoring scope of the early warning model. For loan intermediaries with overdue rates exceeding 50% or overdue amounts exceeding 10 million yuan, they are blacklisted and their consumer loan business is suspended. The overdue early warning model based on the Apriori association rule mining algorithm is used to mine risk signal combinations that are highly correlated with overdue from the multi-dimensional business data of customers. The Apriori algorithm discovers the association relationship between variables in the data through two stages, including frequent item set generation and association rule generation. It is suitable for mining Boolean association rules, but the algorithm also has limitations such as high computational complexity, high rule redundancy, and high time and space consumption. Therefore, in practical applications, it is often necessary to improve and optimize the algorithm, such as using algorithms such as FP-growth and Eclat to improve efficiency, and using strategies such as pruning and compression to reduce memory consumption. By extracting multi-source data such as the customer's basic information, consumer loan information, repayment records, credit reports, etc., and following the optimized Apriori algorithm steps, a series of overdue risk combination rules are obtained, such as "age <25 & education ≤ high school & monthly income <5,000 yuan → overdue probability 65%". The mined association rules are sorted according to their support and confidence, and the top 50 rules with confidence greater than 80% are selected as the rule base of the early warning model. For each rule in the rule base, the warning threshold is set using the quantile idea, and the trigger probability of the rule is divided into quantiles of 5%, 10%, and 25%. The rules with a trigger probability exceeding the quantile of 75% are regarded as high-risk rules, and those exceeding the quantile of 90% are regarded as ultra-high-risk rules, and different warning levels are set for each. When the number of triggers or the trigger ratio of a rule exceeds the warning threshold, a warning signal of the corresponding level is generated. The polling method is adopted to batch process and scan the customer data of the key monitored loan intermediaries on a regular basis every day. By matching with the rules in the early warning rule library, the customer list that triggers the early warning is identified, and according to the overdue risk level of the early warning customers, they are divided into three levels: high, medium and low. For the identified early warning signals and customer lists, they are pushed to the corresponding loan intermediaries in real time through message middleware such as Kafka, and early warning notification messages are automatically generated to inform the loan intermediaries of the overdue risk level and detailed information of the customers, and trigger the corresponding collection process. On the loan intermediary side, the early warning message can be obtained in real time through the consumer API of Kafka, and the corresponding collection process is started according to the customer information and risk level in the message. For the early warning customers with high risk level, the collection reminder information is sent, requiring the loan intermediary to collect the customer by phone within 2 working days and feedback the collection results; for the early warning customers with medium risk level, the attention reminder information is sent, requiring the loan intermediary to investigate the customer's overdue situation within 5 working days and feedback the investigation situation.After the early warning notification is sent and the collection process is triggered, the effect should be tracked and recorded, the overdue situation of the early warning customers, the consistency between the collection feedback and the actual overdue, and the accuracy, recall rate, false alarm rate and overdue rate reduction of the model should be calculated to provide a data basis for subsequent regular evaluation. The overall effect of the early warning model should be evaluated and optimized regularly, and the evaluation indicators should be compared with the benchmark period, such as the three months before the launch. If the early warning coverage rate reaches 80%, the hit rate reaches 70%, and the overall overdue rate drops by 5 percentage points after the model is launched, it means that the effect is relatively significant. For individual cases with good early warning effects, such as a loan intermediary, after introducing the early warning model, its overdue rate of more than 90 days has dropped from 15% to 5%, and the overdue rate of less than 60 days has dropped from 30% to 10%, and the cumulative overdue amount has been reduced by 10 million yuan, it can be promoted as a demonstration. For links where the warning effect is not ideal, it is necessary to further analyze the reasons, such as whether the warning threshold setting is reasonable, whether the rule coverage is comprehensive, whether the data quality is guaranteed, whether the collection measures are effective, etc., and optimize and improve them in a targeted manner. In addition, it is also possible to consider establishing a long-term mechanism for overdue warning, and uniformly store and manage information such as the operation status of the overdue warning model, warning sending records, and collection feedback results, to form a complete overdue warning closed loop, and provide data support for subsequent model optimization and risk decision-making.
[0033] Based on the above embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification.
Claims
1. A method for fusion analysis of financial big data, characterized in that: The method comprises: Obtain the consumer loan business data indicators of loan intermediaries, including the distribution information of the number of consumer loan applications, the proportion of customer credit score ranges, the distribution of consumer loan amount ranges, the proportion of consumer loan term preferences, the proportion of differentiated mortgage product types, the proportion of repayment method preferences, the distribution of overdue rate levels and the distribution characteristics of overdue duration. Use the cosine similarity algorithm to calculate the difference values of loan intermediaries in consumer loan business data indicators and construct a multi-dimensional difference matrix. Based on the multi-dimensional difference matrix, use the K-means clustering algorithm based on Euclidean distance to cluster all loan intermediaries and obtain different loan intermediary groups. Analyze the distribution of the number of consumer loan applications and the proportion of customer credit score ranges of loan intermediary groups, determine the risk level of loan intermediary groups, and generate differentiated marketing plans and risk control strategies based on the risk level. According to the consumer loan term preference proportion, differentiated mortgage product type proportion, repayment method preference proportion and The risk level is determined by the orthogonal table experimental design method, and the optimal consumer loan product solution matching the loan intermediary group is generated, including the amount, term, interest rate and repayment method of the consumer loan. According to the optimal consumer loan product solution, the application material recognition model is constructed through the conditional random field algorithm, and the application materials submitted by the loan intermediary are structured and extracted. The key element information is identified through named entity recognition, and compared with the preset application template. The edit distance algorithm is used to identify the missing or non-compliant parts, and an application review report is generated. According to the application review report, the distribution characteristics of the customer overdue rate level and the overdue duration distribution of different loan intermediaries are obtained, and a differentiated configuration model for risk control approval rules is established. According to the customer credit score interval proportion, the consumer loan amount interval distribution and the differentiated proportion of the mortgage product type, the Gini index is used for feature selection, and the approval rule thresholds of different loan intermediary groups are set differently, including: According to the application review report, obtain the distribution characteristics of customer delinquency rates and delinquency duration of different loan intermediaries, clean, convert and standardize the characteristic data to obtain a data set; By using feature engineering methods, key features related to the overdue risk of loan intermediary customers are extracted from the data set, including the mean overdue rate, the variance of the overdue rate, the mean overdue duration, and the variance of the overdue duration, to build a differentiated configuration model for risk control approval rules; The Gini index is used to select features based on the customer credit score interval ratio, consumer loan amount interval distribution, and mortgage product type differentiation ratio. The Gini index of each feature is calculated to determine its contribution to the overdue risk, and the five features with the highest degree of differentiation of overdue risk are selected as the input of the risk control approval rule differentiation configuration model. For different loan intermediary groups, the K-means clustering algorithm is used to divide customer groups according to the feature selection results. The clustering results are mapped with the actual overdue labels to obtain the risk level of each category, including high-risk group, medium-risk group and low-risk group; For groups with different risk levels, differentiated approval rule thresholds are set according to the dimensions of consumer loan amount, customer credit score, mortgage rate and repayment ability. Specifically, in the risk control approval rule differentiated configuration model, the corresponding approval rule thresholds are automatically generated through decision tree or random forest algorithm according to the distribution characteristics of customer overdue rate level and overdue duration of different loan intermediary groups; the LightGBM algorithm is used to integrate the historical consumer loan performance data of customers of each loan intermediary, and the risk feature set is extracted from the historical consumer loan performance data of customers through feature engineering, and the risk feature set is processed through minimum-maximum normalization. After processing, grid search is used to optimize the model hyperparameters, train the risk assessment model, and predict the risks of consumer loan applications of customers of the target loan intermediaries. The prediction results are used to dynamically adjust the approval rule thresholds. According to the distribution of the delinquency rate levels of customers of each loan intermediary, key monitoring is performed on loan intermediaries whose monthly delinquency rate exceeds the preset threshold and whose overdue amount exceeds the preset amount. The overdue warning model based on Apriori association rule mining is used to mine overdue risk signals from multi-dimensional business data. If the number of warning triggers is greater than the preset warning threshold, a warning notification is sent to the loan intermediary through the message middleware to trigger the start of the collection process.
2. The method according to claim 1, wherein: The consumer loan business data indicators of the loan intermediary are obtained, including the distribution information of the number of consumer loan applications, the proportion of customer credit score intervals, the distribution of consumer loan amount intervals, the proportion of consumer loan term preferences, the proportion of differentiated mortgage product types, the proportion of repayment method preferences, the distribution of overdue rate levels and the distribution characteristics of overdue duration. The cosine similarity algorithm is used to calculate the difference values of the consumer loan business data indicators of the loan intermediary, and a multi-dimensional difference matrix is constructed, including: Obtain consumer loan business data from loan intermediaries, including the number of consumer loan applications, customer credit scores, consumer loan amounts, consumer loan terms, types of collateral, repayment methods, delinquency rates, and delinquency durations; Preprocessing the consumer loan business data to obtain a consumer loan business data set through data cleaning and standardization; Based on the consumer loan business data set, statistical analysis methods are used to calculate the consumer loan business data indicators of loan intermediaries, including the distribution information of the number of consumer loan applications, the proportion of customer credit score ranges, the distribution of consumer loan amount ranges, the proportion of consumer loan term preferences, the proportion of differentiated mortgage product types, the proportion of repayment method preferences, the distribution of overdue rate levels, and the distribution of overdue duration; Constructing a multi-dimensional consumer loan business feature vector based on the consumer loan business data indicators; Using the cosine similarity algorithm, the similarity between different loan intermediaries on the consumer loan business feature vector is calculated to obtain a similarity matrix between different loan intermediaries; According to the similarity matrix, the difference values of the loan intermediary on the consumer loan business data indicators are calculated, and a multi-dimensional difference matrix is constructed according to the difference values.
3. The method according to claim 1, wherein: According to the multi-dimensional difference matrix, the K-means clustering algorithm based on Euclidean distance is used to cluster all loan intermediaries to obtain different loan intermediary groups, analyze the distribution of the number of consumer loan applications and the proportion of customer credit score ranges of loan intermediary groups, determine the risk level of loan intermediary groups, and generate differentiated marketing plans and risk control strategies based on the risk level, including: According to the multi-dimensional difference matrix, the K-means clustering algorithm based on Euclidean distance is used to continuously update the cluster center through iterative optimization, and all loan intermediaries are divided into several loan intermediary groups; Analyze the proportion of loan intermediary groups in the distribution of the number of consumer loan applications and the range of customer credit scores, and determine the business characteristics of the loan intermediary groups, including the overall business scale and customer credit level, by calculating the mean, median and quartile statistics of the distribution of the number of consumer loan applications and the range of customer credit scores; Analyze the historical consumer loan business data of loan intermediary groups, determine the risk level of loan intermediary groups by comparing with the preset benchmark level, and identify the risk factors in the risk level; Integrate the division results, business characteristics and risk factors of each loan intermediary group into the loan intermediary portrait system to form a loan intermediary grouping system, generate differentiated marketing plans and risk control strategies based on the loan intermediary grouping system, and optimize the differentiated marketing plans and risk control strategies based on business development and risk prevention and control, including adjusting the approval rate threshold and increasing the frequency of post-loan management.
4. The method according to claim 1, wherein: The optimal consumer loan product solution matching the loan intermediary groups is generated by using the orthogonal table experimental design method based on the consumer loan term preference proportion, mortgage product type differentiation proportion, repayment method preference proportion and risk level of different loan intermediary groups, including the consumer loan amount, term, interest rate and repayment method, including: Generate differentiated consumer loan product solutions based on differences in loan intermediary groups in terms of consumer loan term preferences, differentiated proportions of mortgage product types, and repayment method preferences; The orthogonal table experimental design method is used to generate a consumer loan product plan based on the four key factors of consumer loan amount, consumer loan term, consumer loan interest rate and repayment method through orthogonal table combination; Evaluate the matching degree and risk level of consumer loan product solutions, and select consumer loan product solutions whose matching degree with the characteristics of loan intermediary groups is greater than the preset threshold; If the analytic hierarchy process is used to determine the weight of each evaluation indicator, then an evaluation indicator hierarchy model is established, the importance of indicators is compared two by two, a pairwise comparison matrix is constructed, the eigenvalues and eigenvectors of the comparison matrix are calculated, the eigenvectors are normalized, the indicator weights are obtained, a consistency test is performed, clear scoring rules and standards are generated, the consumer loan product scheme is scored, and the comprehensive score of the consumer loan product scheme is calculated by weighted average, and the scheme with a score greater than a preset threshold is selected as the recommended scheme; Establish an effect evaluation mechanism for differentiated products, select key indicators such as the number of consumer loan applications, loan amount, delinquency rate and customer satisfaction, set quantitative evaluation targets, build data collection and calculation processes, and regularly evaluate the performance data and customer feedback data of product solutions; Based on the actual business data and personalized needs of loan intermediaries, the optimal consumer loan product solution is dynamically generated to match the product solution with the needs of loan intermediaries.
5. The method according to claim 1, wherein: According to the optimal consumer loan product solution, the application materials recognition model is constructed through the conditional random field algorithm, the application materials submitted by the loan intermediary are structurally extracted, the key element information is identified through named entity recognition, and compared with the preset application template, and the edit distance algorithm is used to identify the missing or non-compliant parts, and the application review report is generated, including: According to the optimal consumer loan product solution, an application material recognition model is constructed through the conditional random field algorithm; Define a characteristic function to characterize the correlation between the key elements in the application materials, obtain a conditional probability model based on the correlation, and realize serialized annotation of the key elements; For the submitted application materials, the application materials recognition model is used to structurally extract key element information from the content of the application materials, and named entity recognition is used to recognize the key element information; Compare the extracted key element information with the preset application template, and calculate the difference between the extracted key element information and the preset application template; If the difference exceeds a preset threshold, it is determined that there is a missing or non-compliant part in the filling, and the location and type of the missing or non-compliant part are marked; Based on the location and type of missing or non-compliant information, predefined modification templates are filled in to generate an application review report, instructing the applicant to revise and improve the materials.
6. The method according to claim 1, wherein: The LightGBM algorithm is used to integrate the historical consumer loan performance data of customers of various loan intermediaries, extract the risk feature set from the historical consumer loan performance data of customers through feature engineering, process the risk feature set through minimum-maximum normalization, use grid search to optimize the model hyperparameters, train the risk assessment model, and predict the risk of consumer loan applications of customers of the target loan intermediary. The prediction results are used for the dynamic adjustment of the approval rule threshold, including: Based on the LightGBM algorithm, the historical consumer loan performance data of each loan intermediary is integrated to form a risk assessment data set. The historical consumer loan performance data of the customer includes basic information of the customer, credit information, assets and liabilities, and behavioral preference data; By means of feature engineering, key features related to credit risk are extracted from the risk assessment data set to construct a risk feature set; Performing minimum-maximum normalization on the risk feature set to map data of different scales and distributions into a preset interval; The hyperparameters of the risk assessment model are optimized by using a grid search method, and the optimal parameter combination is obtained by selecting parameters by presetting different parameter combinations and using a cross-validation method; Inputting the consumer loan application data of each loan intermediary into the risk assessment model, predicting the default risk of each applicant, and obtaining a risk probability score; Dynamically adjust various thresholds in the approval rules according to the risk probability score; If a customer's risk probability score is greater than the preset probability value, the customer will be marked as a high-risk customer. If the proportion of such high-risk customers exceeds the preset threshold, the credit score threshold and the upper limit of the consumer loan amount will be lowered to achieve dynamic risk control.
7. The method according to claim 1, wherein: According to the distribution of customer overdue rates of various loan intermediaries, the loan intermediaries whose monthly overdue rates exceed the preset threshold and whose overdue amounts exceed the preset amounts are monitored, and the overdue risk signals are mined from multi-dimensional business data using the overdue early warning model based on Apriori association rule mining. If the number of triggered early warnings exceeds the preset early warning threshold, an early warning notification is sent to the loan intermediary through the message middleware to trigger the collection process, including: Based on the distribution of customer delinquency rates of various loan intermediaries, loan intermediaries whose monthly delinquency rates exceed the preset thresholds and whose overdue amounts exceed the preset amounts are identified and marked as key monitoring targets and included in the monitoring scope of the early warning model; Adopting the overdue early warning model based on Apriori association rule mining algorithm, we can mine risk signal combinations with overdue correlation greater than the preset value from the multi-dimensional business data of customers; In the data processing phase, extract the customer's structured data, including basic customer information, consumer loan information and repayment records, as well as unstructured data, including credit report information; For the unstructured data, natural language processing technology is used to extract key information, and the information is associated and integrated with the structured data to form a customer profile; According to the Apriori association rule mining algorithm, combined with the customer profile, overdue risk combination rules are mined; Sorting the overdue risk combination rules according to their confidence levels, and selecting several rules with confidence levels greater than a preset threshold as the rule base of the overdue warning model; For each rule in the rule base, the quantile method is used to set the warning threshold, and the trigger probability of the rule is divided according to the preset quantile, and the rule whose trigger probability exceeds the preset quantile is marked as a high-risk rule; When the number of times or the triggering ratio of a rule exceeds the preset warning threshold, a warning signal of the corresponding level is generated; By adopting a polling method, the data of loan intermediary customers under key monitoring is batch processed and scanned at regular intervals, and by matching the data with the rules in the rule base, a list of customers that trigger the warning rules is identified, and the customers that are warned are divided into different risk levels according to their overdue risk levels; The warning signal and the customer list are pushed to the corresponding loan intermediary in real time through the message middleware, generating a warning notification message and triggering the corresponding collection process.
