A method of matching enterprise financing funds based on data mining
By building a corporate portrait and labeling system, data mining technology is used to solve the problems of information asymmetry and investment risks in the corporate financing process, and the fund company's accurate identification and supervision of corporate credit and operating conditions is achieved, reducing investment risks.
Patent Information
- Application Number
- CN202111609332.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-27
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2041-12-27
AI Technical Summary
During the process of corporate financing, it is difficult for existing technology to effectively explore and utilize massive enterprise data, resulting in fund companies taking time to select investment objects, information asymmetry, and relying on personal experience, increasing investment risks.
Through data mining technology, build corporate portraits, standardize and index enterprise credit data, combine RFM and unsupervised clustering algorithm to generate corporate portrait tags, and use machine learning methods to predict tags to support fund companies to query and screen potential investment objects in real time. Use sliding window filtering algorithm to update corporate portraits.
It improves the accuracy of identifying corporate credit risks and business highlights, reduces the investment risks of fund companies, improves corporate information transparency and market competitiveness, and enhances the supervision and management capabilities of fund companies.
Smart Images

Figure CN114266492B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data mining, and in particular to an enterprise financing fund matching method based on data mining. Background Art
[0002] With the increasing richness of inter-enterprise cooperation and business operations, enterprises, as economic entities, need rich enterprise information as a reference when seeking cooperation and financing. With the growth of the number of enterprises and transaction information data, the amount of data generated is also increasing, and it is growing exponentially every year. With the advent of the big data information age and the increasing development of big data processing technology, we can mine valuable data from massive and complex enterprise activity data to help securities companies, fund companies and other investors to mine and calculate all-round information about enterprises, including discovering and mining the relationship between enterprises, finding unknown relationships to promote enterprise cooperation, identifying enterprise capital behavior, and constructing enterprise risk assessment models. Automatically forming enterprise financing credit labels through data mining technology to match the needs of fund companies can significantly reduce the service costs of fund companies, shorten service time, and improve service coverage, thereby promoting more high-quality small and medium-sized enterprises to obtain financing.
[0003] For corporate portraits, it is to label corporate information, use big data technology to analyze the data of the tax system and third-party taxpayers, establish a label model system for the enterprise, classify the specific behavioral attributes of the enterprise, and finally form a diversified corporate label object. The corporate portrait model based on data mining mines the multi-source information of the enterprise, brings together multiple imaging elements such as corporate operations, integrity, risks and contributions, analyzes the company's factual attributes, financial status, operating conditions, development prospects, and profitability, and then recommends fund companies to companies with different label levels. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides an enterprise financing fund matching method based on data mining.
[0005] The present invention is based on a large amount of data on small, medium and micro enterprises, and constructs enterprise portraits through data mining methods to form enterprise financing and credit labels based on enterprise portraits; then, investment institutions such as fund companies can view the enterprise portrait label display and conveniently obtain enterprise information by query and retrieval; finally, through the update mechanism, the real-time and accuracy of enterprise data, indicators, portraits, labels and other information are guaranteed.
[0006] The technical solution of the present invention is:
[0007] A method for matching enterprise financing funds based on data mining, comprising:
[0008] S1: Enterprise credit data standardization: collect the credit information of the target enterprise according to the enterprise name, integrate and standardize the massive credit information data;
[0009] S2: Indexing of corporate credit data: Indexing of corporate credit data, deriving and screening indicators based on algorithms such as RFM and unsupervised clustering, and establishing a preliminary indicator library.
[0010] S3: Enterprise portrait construction and calculation: Build a dimension system that describes enterprise characteristics through enterprise portraits, and form enterprise portrait labels through dimension weight calculation. On the one hand, the construction of the enterprise portrait system needs to be combined with business needs, grasp the reasonable granularity, and effectively divide the dimension structure that describes the enterprise; on the other hand, the construction of enterprise labels needs to be based on machine learning methods, so that the label results have predictive and forward-looking characteristics for the future development of the enterprise.
[0011] S4: Enterprise portrait label display and retrieval: The display of enterprise portrait labels includes the display of single enterprise portrait and label and the display of enterprise group portrait and label; enterprise portrait label retrieval supports fund companies and other financial institutions to retrieve potential investment targets that meet the requirements by screening through portrait labels.
[0012] S5: Enterprise portrait tag content maintenance: In order to obtain more accurate and real-time enterprise portraits, the portrait system also needs to continuously update enterprise features according to changes in enterprise development data. Use a sliding window filtering algorithm to periodically update the enterprise portrait system.
[0013] Furthermore, the key technologies for standardization of corporate credit data and data processing steps include:
[0014] 1) Acquisition of credit information data
[0015] The main work is to complete the construction of data warehouse model, definition of data source, data extraction, conversion, cleaning and loading (ETL). The specific steps include:
[0016] Definition of data source: Since the sources of credit big data are rich and extensive, the design principles, structures, and subsequent service platforms of various systems that carry credit big data are also different. Therefore, it is necessary to uniformly manage and process interface data, library table data, and unstructured data into unified structured data that is easy to store, manage, and use. Improving the quality of data sources is also a very critical process. Through cross-comparison, rule verification and other methods, data sources with inconsistent structures and incomplete data information are transformed into data sources that meet unified structural standards and are placed in the data warehouse;
[0017] Data extraction, transformation, cleaning and loading: In the process of data extraction, transformation, cleaning and loading (ETL), an ETL strategy is established according to the characteristics of the credit data system in this modeling, including the frequency of credit data extraction, credit data granularity, credit data dimension, etc. After the completion of ETL, monitoring and tracking of the entire process are also essential steps. Among them, in the data extraction stage, the trigger update method is used for data in the form of interfaces; for data in the form of databases and tables, when the amount of public disclosure data, reward and punishment data, etc. is not large, the full update method is used. When the financial indicator data, industrial and commercial indicator data, etc. have a large number of indicators, and the regular update is fast, an incremental extraction mechanism based on data timestamps should be adopted. In the data conversion stage, the original corporate credit data extracted by the above process is converted into different data dimensions such as time, region, and industry, and then the relevant rules are used to verify the data quality and exclude data that does not meet the analysis quality requirements.
[0018] 2) Credit data governance
[0019] Detailed data from multiple data source systems and integrated summary data for analysis are stored in this center. It is a complete library logically. From a hierarchical perspective, credit data management is divided into four levels: source data layer, verification data layer, basic data layer, and product data layer:
[0020] Source data layer: It is the basis of data in other layers. The source data layer processes and stores data from various data sources in a unified and standardized manner, and updates them using appropriate strategies;
[0021] Verification data layer: When the data from the source data layer arrives, the data content is verified through the established data verification rules and verification process. The data that passes the verification is loaded into the central database, and the data that fails the verification is returned to the next layer. The erroneous data forms a data feedback message and is fed back to the management agency of the corresponding data source;
[0022] Basic data layer: mainly stores key credit business data that has passed verification. Based on performance considerations, the data that has passed verification is generally loaded into the central database in batches. The central database mainly stores the data that has passed verification, and returns the erroneous data found through post-verification to the next layer, and finally forms an error feedback message to the reporting agency.
[0023] Product data layer: After the basic data is stored and organized, it is loaded into the product data layer according to the type of service products (credit reports, fixed reports, summary queries, etc.), established data extraction logic and mining analysis, to prepare the product data required for the service for users.
[0024] 3) Application of credit data
[0025] Enterprise portrait labeling and enterprise search are the core businesses of the enterprise credit data warehouse. After establishing an enterprise credit database and conducting data governance and model building on it, the credit products provided are mainly enterprise overall portrait display, enterprise labeling and enterprise portrait label search. Enterprise portrait label search will store the enterprise label results generated based on enterprise portrait in the database in the form of No-sql data, support high-speed caching, and is suitable for fast query of enterprise portrait data. The output can be displayed through a Web page for customer acquisition analysis and investment decision-making of fund companies.
[0026] Furthermore, the enterprise credit data is indexed, and the index derivation and index selection methods are as follows:
[0027] 1) Derivation of RFM (Recency, Frequency, Monetary) indicators
[0028] RFM can achieve the segmentation of the credit status of the rated enterprise by describing the current operating ability, future development potential and performance ability of the rated enterprise. In this plan, the RFM framework is applied to the management ability, repayment ability and willingness, operation and profitability, enterprise qualifications, innovation and growth ability of the enterprise, and the characteristic derivative subject (statistical object) with enterprise financial indicators, patent statistics, administrative information, etc. as the core is defined. The specific derivation method is:
[0029] Statistical objects: corporate financial statistics, patent statistics, administrative information and other characteristics.
[0030] Statistical window: The short-term window is mostly based on a period of nearly 6 months, and the long-term window ranges from nearly 1 year to nearly 3 years.
[0031] Aggregation functions: continuous functions such as maximum, minimum, sum, mean, etc.; discrete functions such as count, frequency, etc.
[0032] Statistical variables: Continuous variables refer to variables that can take any value within a certain interval, such as the amount of unpaid taxes, etc. Discrete variables refer to variables that have a limited fixed value and can be enumerated, such as the number of tax violations, etc. New characteristic indicators are summarized and refined based on business knowledge to improve the predictive ability and relevance of characteristic indicators.
[0033] 2) Unsupervised clustering index derivation
[0034] This method mainly starts from the relationship and characteristics between the initial indicators, explores the characteristics between high-dimensional indicators that are not easily summarized by relevant business knowledge, and uses the KMeans unsupervised clustering algorithm to produce characteristic indicators with high predictability, high relevance, and high importance. For the large number of new characteristic indicators generated in this stage, they are not processed in the derivation stage and participate in the subsequent screening process. There are two specific derivation methods in this modeling:
[0035] Indicator clustering based on time series: For credit data indicators with time series, clustering and indicator value calculation are performed according to their time dimension.
[0036] Indicator clustering based on business explanation: For different credit indicators with correlation, cluster the selected credit indicator set at a specific time section, generate new feature indicators based on the clustering results, and assign corresponding label values to the rated enterprises.
[0037] 3) Index filtering
[0038] The goal of indicator filtering is to first select indicators for the data set according to certain rules. The selected features need to have the characteristics of high predictability, high stability, high business explainability, etc. Then, the selected indicators are classified according to business experience to form dimensions such as enterprise fundamentals, performance ability, debt repayment ability, growth, and operating ability. The steps of indicator filtering include:
[0039] For continuous indicators, variance filtering method is used;
[0040] For discrete indicators, use the same value filtering method;
[0041] Calculate the correlation coefficient between indicators and delete one of the indicator groups with too high correlation coefficient; calculate the correlation coefficient between the indicator and the target variable and delete the indicator with too small correlation coefficient;
[0042] The XGBoost algorithm is used to sort the importance of indicators and eliminate indicators whose importance is less than the threshold.
[0043] Furthermore, the construction and calculation of enterprise portraits include three levels: fact labels, model labels, and prediction labels, as well as the construction methods of model labels and prediction labels:
[0044] The steps to build the model label are as follows:
[0045] (1) Use the KMeans model for unsupervised clustering of enterprise indicators: The KMeans unsupervised clustering algorithm is used to generate enterprise model labels. This method mainly starts from the relationship and characteristics between the initial indicators and explores the characteristics between high-dimensional indicators that are not easily summarized by relevant business knowledge. The KMeans unsupervised clustering algorithm is used to automatically cluster time series indicators and indicators with high predictability and high business explanatory power, and the enterprise labels are summarized based on the characteristics of the cluster center indicators of the enterprise clusters formed by the clustering results.
[0046] (2) Use the association measurement model to discover enterprise association rules: Use the Apriori algorithm to mine association rules for enterprise indicators.
[0047] Association rule mining can be divided into two categories:
[0048] Enterprise risk mining: Based on the enterprise's own indicators, explore the relationship between enterprise indicators and enterprise risk performance, establish the association rules between enterprise indicator values and enterprise default risks, and form the enterprise's own risk association labels;
[0049] Mining risk associations among enterprise groups: For an enterprise set formed by multiple enterprises, the Apriori algorithm is used to mine the risk contagion rules between enterprises. By establishing the relationship between the indicators of a single enterprise and the indicators and risk performance of other defaulting enterprises in the enterprise set, a risk association label for the enterprise group is formed.
[0050] (3) Use the analytic hierarchy process to comprehensively calculate the enterprise performance score: First, the enterprise's indicators and the labels obtained through the model label are used as indicators for hierarchical serialization, that is, according to the evaluation objectives, the enterprise is divided into six dimensions: operating status, development potential, financial status, innovation ability, fundamentals, and model label results, and the indicators are classified into these six dimensions to form a progressive and ordered hierarchical model. Then, the relative importance of each level of factors in the model is quantitatively expressed, and the weight of the relative importance order of all factors at each level is determined by the discriminant matrix method. Finally, by comprehensively calculating the relative importance weights of each factor, the combined weights of the relative importance order of the lowest level (scheme level) and the highest level (overall goal) are obtained, and this model is used as the basis for calculating the enterprise performance score.
[0051] The steps for constructing the predicted labels are as follows:
[0052] (1) Data cleaning
[0053] Missing value statistics: remove indicators with all missing values, and perform statistics on the proportion of missing values in the two dimensions for the remaining indicators. Indicators with missing values greater than the threshold of 0.85 are classified as indicators with judgment and retained, and indicators with judgment and all missing indicators are eliminated.
[0054] Parity rate statistics: For the remaining indicators from the previous round, missing values are filled with float('-inf'), and the percentage of i) only one value ii) only two values iii) only two values and including -inf value and different values are calculated. The parity rate threshold is set to 0.8, and all indicators with a parity rate of 1 are removed. Indicators with a parity rate greater than 0.8 (excluding single values) are retained as pending indicators, and the remaining indicators all enter the indicator feature correlation analysis stage.
[0055] VIF collinearity analysis: For the remaining missing values of indicators, fill them with -999999 and perform VIF collinearity analysis. The threshold is set to 10. After the indicator collinearity analysis, there are no remaining indicators with collinearity.
[0056] Missing value filling: In this modeling, all missing values in the indicators are filled with a special value -999999, which is different from other existing indicator values. In the binning process, missing values are treated as independent values for analysis.
[0057] Normality test of indicators: In this module, the normality test is carried out on the distribution of indicators of the enterprise common credit model, the skewness and kurtosis of the indicator distribution are calculated, and log transformation is performed on the indicators with large skewness and kurtosis that are closer to normal distribution.
[0058] (2) Feature screening
[0059] XGBoost feature screening: In this modeling process, the XGBoost method is used to screen features. The XGBoost model randomly adjusts the model parameters and evaluates the importance of features based on the nonlinear relationship between features. In order to avoid model error problems that may occur during model training, the final feature importance evaluation result takes the mean of all model training.
[0060] WOE binning and IV value feature screening: In this modeling, the optimal binning method based on decision tree is adopted to perform WOE binning on the indicators entered into the model. The IV screening threshold is set to the corresponding threshold with the highest performance in the model comparison analysis, and the feature importance evaluation results obtained by XGBoost feature screening are combined to screen out qualified indicators.
[0061] (3) Model training
[0062] Stepwise regression
[0063] In this modeling, forward regression training based on RMSE was carried out according to the order of IV values obtained by sorting the importance of XGBoost features. The performance of the trained model was compared and analyzed through indicators such as Accuracy, AUC, and RMSE to select the optimal strategy, remove features that reduce AUC performance, and form the final model input indicators.
[0064] Model parameter optimization and prediction probability analysis
[0065] The model parameter adjustment uses the HyperOpt automatic parameter search method, the evaluation uses cross validation accuracy to measure, and the output results of finding the optimal parameters in the process of model parameter optimization are plotted.
[0066] Plotting model learning curve
[0067] In this modeling process, by drawing the learning curves of each trained model, it can be concluded that the model does not have overfitting or underfitting, so the model has a good fitting effect on the training samples.
[0068] Scorecard Conversion
[0069] The standard scorecard is converted according to the logistic regression model coefficients and the results of the woe conversion of each feature. The output of this modeling is a standard scorecard generated from the final predicted probability conversion, and this output meets the ideal requirements of the standard scorecard result, that is, the scorecard distribution diagram finally presents a trend of approximate normal distribution.
[0070] Furthermore, the display and retrieval of enterprise portrait tags, including decomposition of query data and parallel query, can decompose the burden of data processing, solve the query difficulties of high-load data, and combine the query results of each part during result processing to form the final result based on the enterprise portrait query; and based on the performance requirements of retrieval and query, the portrait system uses Elasticsearch as the search engine to realize the real-time query function of enterprise portrait.
[0071] Furthermore, the content maintenance of enterprise portrait tags includes the use of a sliding window filtering algorithm to continuously update the enterprise portrait. Whenever an update condition is triggered, it is only necessary to move the time window, that is, it is not necessary to read all historical data, only read the new data within a period of time as the calculation object. Then the new results and the intermediate results calculated and stored by the historical user data are calculated to obtain the final result within that time point and stored. Through this algorithm, the enterprise portraits of enterprises in various time periods can be flexibly stored, and the tracing and query of the evolution of enterprise portraits within a certain time window can be supported.
[0072] The beneficial effects of the present invention are
[0073] 1. Building a comprehensive and penetrating corporate portrait based on data mining models can improve the accuracy of identifying key information such as corporate credit risks and business highlights;
[0074] 2. It overcomes the problems of traditional fund companies in selecting enterprises, such as long time consumption, information asymmetry, one-sidedness and subjectivity based on personal experience, and effectively reduces investment risks for fund companies;
[0075] 3. By combining statistical analysis and machine learning algorithms, the method maintains interpretability and comprehensibility, and has the ability to be widely used, while improving the efficiency and comprehensiveness of enterprise information mining and helping fund companies improve the accuracy of enterprise evaluation;
[0076] 4. The application of the enterprise portrait system helps enterprises strengthen their credit awareness. While enhancing the transparency and openness of enterprise information, it also urges enterprises to regulate their own business behavior based on market competition principles to avoid poor management or breach of trust. The application of the enterprise portrait platform also helps fund company users strengthen their supervision and management capabilities of enterprises. Based on existing supervision experience, fund companies can conduct post-event reviews of enterprises through annual reports and other materials, timely grasp the operating conditions and credit status of enterprises, and ensure the quality of enterprise customer resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] Figure 1 It is a schematic diagram of the workflow of the present invention. DETAILED DESCRIPTION
[0078] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0079] The present invention proposes a method for matching corporate financing funds based on data mining, which uses data and mining technology to form corporate portraits and match fund companies, and can accurately and comprehensively characterize enterprises from both qualitative and quantitative perspectives. The former is qualitative, that is, the fact label of the enterprise. The preliminary research on the enterprise is carried out through the original data, and the enterprise is positioned from a macro perspective. The enterprise is qualitatively portrayed based on the inherent attributes of the enterprise, legal representative, investor, financial person in charge, enterprise site selection, transaction scope, enterprise scale and other data. For the quantitative analysis of the enterprise, that is, the enterprise model label, it can be further divided according to the focus and investment needs of different fund companies to achieve more accurate matching and improve the success rate of corporate financing.
[0080] The implementation steps mainly include:
[0081] The first step is to standardize credit data and process the original corporate credit data obtained from multiple data sources such as government data, Internet data, feedback from cooperative financial institutions, and third-party cooperation data using a corporate credit data warehouse to obtain, manage, and apply data at three levels;
[0082] The second step is to use big data processing and machine learning technology to the data in the enterprise credit data warehouse through credit data indexation, so as to form standardized enterprise credit data indicators covering the current operating ability and future development potential of the evaluated enterprises;
[0083] The third step is to build and calculate the enterprise portrait, build a dimension system that describes the characteristics of the enterprise through the enterprise portrait, and form the enterprise portrait label through dimension weight calculation. On the one hand, the construction of the enterprise portrait system needs to be combined with business needs, grasp the reasonable granularity, and effectively divide the dimension structure that describes the enterprise; on the other hand, the construction of enterprise labels needs to be based on machine learning methods, so that the label results have predictive and forward-looking characteristics for the future development of the enterprise.
[0084] The fourth step is the display and retrieval of enterprise portrait tags: the display of enterprise portrait tags includes the display of individual enterprise portraits and tags and the display of enterprise group portraits and tags; the enterprise portrait tag retrieval supports fund companies and other financial institutions to retrieve potential investment targets that meet the requirements by screening through portrait tags.
[0085] Step 5: Maintain the content of enterprise portrait tags: In order to obtain more accurate and real-time enterprise portraits, the portrait system also needs to continuously update enterprise characteristics according to changes in behavioral data. By setting reasonable trigger conditions, the enterprise portrait system can be updated periodically.
[0086] S1: Enterprise credit data standardization
[0087] The corporate credit data collected and counted in this modeling mainly come from corporate industrial and commercial data, tax data, and reward and punishment information authorized by the government; credit data and flow data of enterprises and related managers fed back by cooperating banks and other financial institutions; public opinion data and credit publicity data collected on the Internet; data of third-party cooperative operators, commercial transaction data, and judicial data obtained by third parties. Corporate credit data covers business capability data including corporate management capabilities, repayment capabilities and willingness, operational capabilities, profitability, and corporate qualifications; and development potential data including innovation capabilities and growth capabilities. The corporate credit data warehouse system architecture has the following three levels: credit data acquisition, credit data governance, and credit data application.
[0088] (1) Acquisition of credit information data
[0089] The main work is to complete the construction of data warehouse model, definition of data source, data extraction, conversion, cleaning and loading (ETL). The specific steps include:
[0090] Definition of data source: Since the sources of credit big data are rich and extensive, the design principles, structures, and subsequent service platforms of various systems that carry credit big data are also different. Therefore, it is necessary to uniformly manage and process interface data, library table data, and unstructured data into unified structured data that is easy to store, manage, and use. Improving the quality of data sources is also a very critical process. Through cross-comparison, rule verification and other methods, data sources with inconsistent structures and incomplete data information are transformed into data sources that meet unified structural standards and are placed in the data warehouse;
[0091] Data extraction, transformation, cleaning and loading: In the process of data extraction, transformation, cleaning and loading (ETL), an ETL strategy is established according to the characteristics of the credit data system in this modeling, including the frequency of credit data extraction, credit data granularity, credit data dimension, etc. After the completion of ETL, monitoring and tracking of the entire process are also essential steps. Among them, in the data extraction stage, the trigger update method is used for data in the form of interfaces; for data in the form of databases and tables, when the amount of public disclosure data, reward and punishment data, etc. is not large, the full update method is used. When the financial indicator data, industrial and commercial indicator data, etc. have a large number of indicators, and the regular update is fast, an incremental extraction mechanism based on data timestamps should be adopted. In the data conversion stage, the original corporate credit data extracted by the above process is converted into different data dimensions such as time, region, and industry, and then the relevant rules are used to verify the data quality and exclude data that does not meet the analysis quality requirements.
[0092] (2) Credit data governance
[0093] Detailed data from multiple data source systems and integrated summary data for analysis are stored in this center. It is a complete library logically. From a hierarchical perspective, credit data management is divided into four levels: source data layer, verification data layer, basic data layer, and product data layer:
[0094] Source data layer: It is the basis of data in other layers. The source data layer processes and stores data from various data sources in a unified and standardized manner, and updates them using appropriate strategies;
[0095] Verification data layer: When the data from the source data layer arrives, the data content is verified through the established data verification rules and verification process. The data that passes the verification is loaded into the central database, and the data that fails the verification is returned to the next layer. The erroneous data forms a data feedback message and is fed back to the management agency of the corresponding data source;
[0096] Basic data layer: mainly stores key credit business data that has passed verification. Based on performance considerations, the data that has passed verification is generally loaded into the central database in batches. The central database mainly stores the data that has passed verification, and returns the erroneous data found through post-verification to the next layer, and finally forms an error feedback message to the reporting agency.
[0097] Product data layer: After the basic data is stored and organized, it is loaded into the product data layer according to the type of service products (credit reports, fixed reports, summary queries, etc.), established data extraction logic and mining analysis, to prepare the product data required for the service for users.
[0098] (3) Application of credit information data
[0099] Enterprise portrait labeling and enterprise search are the core businesses of the enterprise credit data warehouse. After establishing an enterprise credit database and conducting data governance and model building on it, the credit products provided are mainly enterprise overall portrait display, enterprise labeling and enterprise portrait label search. Enterprise portrait label search will store the enterprise label results generated based on enterprise portrait in the database in the form of No-sql data, support high-speed caching, and is suitable for fast query of enterprise portrait data. The output can be displayed through a Web page for customer acquisition analysis and investment decision-making of fund companies.
[0100] S2: Enterprise credit data indexation
[0101] By indexing credit data, big data processing and machine learning technology are applied to the data in the enterprise credit data warehouse to form standardized enterprise credit data indicators covering the current operating capabilities and future development potential of the evaluated enterprises. In this stage, the RFM (Recency, Frequency, Monetary) indicator derivation and unsupervised clustering indicator derivation are mainly used.
[0102] (1) Derivation of RFM (Recency, Frequency, Monetary) indicators
[0103] RFM can achieve the segmentation of the credit status of the rated enterprise by describing the current operating ability, future development potential and performance ability of the rated enterprise. In this plan, the RFM framework is applied to the management ability, repayment ability and willingness, operation and profitability, enterprise qualifications, innovation and growth ability of the enterprise, and the characteristic derivative subject (statistical object) with enterprise financial indicators, patent statistics, administrative information, etc. as the core is defined. The specific derivation method is:
[0104] Statistical objects: corporate financial statistics, patent statistics, administrative information and other characteristics.
[0105] Statistical window: The short-term window is mostly based on a period of nearly 6 months, and the long-term window ranges from nearly 1 year to nearly 3 years.
[0106] Aggregation functions: continuous functions such as maximum, minimum, sum, mean, etc.; discrete functions such as count, frequency, etc.
[0107] Statistical variables: Continuous variables refer to variables that can take any value within a certain interval, such as the amount of unpaid taxes, etc. Discrete variables refer to variables that have a limited fixed value and can be enumerated, such as the number of tax violations, etc. New characteristic indicators are summarized and refined based on business knowledge to improve the predictive ability and relevance of characteristic indicators.
[0108] (2) Derivation of unsupervised clustering indicators
[0109] This method mainly starts from the relationship and characteristics between the initial indicators, explores the characteristics between high-dimensional indicators that are not easily summarized by relevant business knowledge, and uses the KMeans unsupervised clustering algorithm to produce characteristic indicators with high predictability, high relevance, and high importance. For the large number of new characteristic indicators generated in this stage, they are not processed in the derivation stage and participate in the subsequent screening process. There are two specific derivation methods in this modeling:
[0110] Indicator clustering based on time series: For credit data indicators with time series, clustering and indicator value calculation are performed according to their time dimension.
[0111] Indicator clustering based on business explanation: For different credit indicators with correlation, cluster the selected credit indicator set at a specific time section, generate new feature indicators based on the clustering results, and assign corresponding label values to the rated enterprises.
[0112] (3) Index filtering
[0113] The goal of indicator filtering is to first select indicators for the data set according to certain rules. The selected features need to have the characteristics of high predictability, high stability, high business explainability, etc. Then, the selected indicators are classified according to business experience to form dimensions such as enterprise fundamentals, performance ability, debt repayment ability, growth, and operating ability. The steps of indicator filtering include:
[0114] For continuous indicators, variance filtering method is used;
[0115] For discrete indicators, use the same value filtering method;
[0116] Calculate the correlation coefficient between indicators and delete one of the indicator groups with too high correlation coefficient; calculate the correlation coefficient between the indicator and the target variable and delete the indicator with too small correlation coefficient;
[0117] The XGBoost algorithm is used to sort the importance of indicators and eliminate indicators whose importance is less than the threshold.
[0118] S3: Enterprise portrait construction and calculation
[0119] To build a comprehensive and detailed corporate portrait, it is necessary to characterize the company from different dimensions. Based on whether the model is used for prediction, the labels can be divided into fact labels, model labels, and prediction labels; based on the description of the comprehensive situation of the company, the labels can be divided into corporate attributes, corporate credit attributes, corporate transaction characteristics, corporate association characteristics, corporate operating capabilities, corporate profitability, corporate evaluation information, etc. The design of the corporate portrait label system focuses on the corporate information that fund companies pay attention to when screening investment target companies.
[0120] The image label calculation modeling relies on the underlying data and belongs to the product data layer. This layer is not responsible for realizing the action on the data, but mainly realizes the processing, analysis and result output of the data that has been collected, standardized and processed into indicators. After the analysis, calculation and generation of labels, data classification, association and clustering analysis are completed. The input of the image is the ID of the query enterprise, and the output is divided into three parts: fact label, model label and prediction label. Among them, the fact label is the key enterprise information directly extracted from the indicator data and predicted by the model processing; the model label is the label formed by the KMeans, Apriori association rule mining, and hierarchical analysis model processing and calculation; the prediction label is mainly formed by the logistic regression algorithm to predict the future profitability, operating conditions and development capabilities of the enterprise, which can help fund companies make forward-looking decisions.
[0121] The steps to build the model label are as follows:
[0122] (1) Use the KMeans model for unsupervised clustering of enterprise indicators: The KMeans unsupervised clustering algorithm is used to generate enterprise model labels. This method mainly starts from the relationship and characteristics between the initial indicators and explores the characteristics between high-dimensional indicators that are not easily summarized by relevant business knowledge. The KMeans unsupervised clustering algorithm is used to automatically cluster time series indicators and indicators with high predictability and high business explanatory power, and the enterprise labels are summarized based on the characteristics of the cluster center indicators of the enterprise clusters formed by the clustering results.
[0123] (2) Use the association measurement model to discover enterprise association rules: Use the Apriori algorithm to mine association rules for enterprise indicators.
[0124] Association rule mining can be divided into two categories:
[0125] Enterprise risk mining: Based on the enterprise's own indicators, explore the relationship between enterprise indicators and enterprise risk performance, establish the association rules between enterprise indicator values and enterprise default risks, and form the enterprise's own risk association labels;
[0126] Mining risk associations among enterprise groups: For an enterprise set formed by multiple enterprises, the Apriori algorithm is used to mine the risk contagion rules between enterprises. By establishing the relationship between the indicators of a single enterprise and the indicators and risk performance of other defaulting enterprises in the enterprise set, a risk association label for the enterprise group is formed.
[0127] (3) Use the analytic hierarchy process to comprehensively calculate the enterprise performance score: First, the enterprise's indicators and the labels obtained through the model label are used as indicators for hierarchical serialization, that is, according to the evaluation objectives, the enterprise is divided into six dimensions: operating status, development potential, financial status, innovation ability, fundamentals, and model label results, and the indicators are classified into these six dimensions to form a progressive and ordered hierarchical model. Then, the relative importance of each level of factors in the model is quantitatively expressed, and the weight of the relative importance order of all factors at each level is determined by the discriminant matrix method. Finally, by comprehensively calculating the relative importance weights of each factor, the combined weights of the relative importance order of the lowest level (scheme level) and the highest level (overall goal) are obtained, and this model is used as the basis for calculating the enterprise performance score.
[0128] The steps for constructing the predicted labels are as follows:
[0129] (1) Data cleaning
[0130] Missing value statistics: remove indicators with all missing values, and perform statistics on the proportion of missing values in the two dimensions for the remaining indicators. Indicators with missing values greater than the threshold of 0.85 are classified as indicators with judgment and retained, and indicators with judgment and all missing indicators are eliminated.
[0131] Parity rate statistics: For the remaining indicators from the previous round, missing values are filled with float('-inf'), and the percentage of i) only one value ii) only two values iii) only two values and including -inf value and different values are calculated. The parity rate threshold is set to 0.8, and all indicators with a parity rate of 1 are removed. Indicators with a parity rate greater than 0.8 (excluding single values) are retained as pending indicators, and the remaining indicators all enter the indicator feature correlation analysis stage.
[0132] VIF collinearity analysis: For the remaining missing values of indicators, fill them with -999999 and perform VIF collinearity analysis. The threshold is set to 10. After the indicator collinearity analysis, there are no remaining indicators with collinearity.
[0133] Missing value filling: In this modeling, all missing values in the indicators are filled with a special value -999999, which is different from other existing indicator values. In the binning process, missing values are treated as independent values for analysis.
[0134] Normality test of indicators: In this module, the normality test is carried out on the distribution of indicators of the enterprise common credit model, the skewness and kurtosis of the indicator distribution are calculated, and log transformation is performed on the indicators with large skewness and kurtosis that are closer to normal distribution.
[0135] (2) Feature screening
[0136] XGBoost feature screening: In this modeling process, the XGBoost method is used to screen features. The XGBoost model randomly adjusts the model parameters and evaluates the importance of features based on the nonlinear relationship between features. In order to avoid model error problems that may occur during model training, the final feature importance evaluation result takes the mean of all model training.
[0137] WOE binning and IV value feature screening: In this modeling, the optimal binning method based on decision tree is adopted to perform WOE binning on the indicators entered into the model. The IV screening threshold is set to the corresponding threshold with the highest performance in the model comparison analysis, and the feature importance evaluation results obtained by XGBoost feature screening are combined to screen out qualified indicators.
[0138] (3) Model training
[0139] Stepwise regression
[0140] In this modeling, forward regression training based on RMSE was carried out according to the order of IV values obtained by sorting the importance of XGBoost features. The performance of the trained model was compared and analyzed through indicators such as Accuracy, AUC, and RMSE to select the optimal strategy, remove features that reduce AUC performance, and form the final model input indicators.
[0141] Model parameter optimization and prediction probability analysis
[0142] The model parameter adjustment uses the HyperOpt automatic parameter search method, the evaluation uses cross validation accuracy to measure, and the output results of finding the optimal parameters in the process of model parameter optimization are plotted.
[0143] Plotting model learning curve
[0144] In this modeling process, by drawing the learning curves of each trained model, it can be concluded that the model does not have overfitting or underfitting, so the model has a good fitting effect on the training samples.
[0145] Scorecard Conversion
[0146] The standard scorecard is converted according to the logistic regression model coefficients and the results of the woe conversion of each feature. The output of this modeling is a standard scorecard generated from the final predicted probability conversion, and this output meets the ideal requirements of the standard scorecard result, that is, the scorecard distribution diagram finally presents a trend of approximate normal distribution.
[0147] S4: Enterprise portrait tag display and retrieval
[0148] The usage and characteristics of enterprise portraits are mainly to query and obtain enterprise labels, which not only places high demands on the concurrent query performance of the system, but also places high demands on the system's aggregation and statistical capabilities. Decomposing query data and conducting parallel queries can decompose the burden of data processing, solve the query difficulties of high-load data, and combine the query results of each part during result processing to form the final result. Therefore, based on the performance requirements of retrieval and query, the portrait system uses Elasticsearch as the search engine to implement the real-time query function of enterprise portraits. The request is initiated by the page, and the WEB server routes the request to the ES query service. The ES query service assembles the Elasticsearch query request according to the screening conditions of the enterprise portrait, queries the Elasticsearch cluster of the portrait, and finally returns to the original path after obtaining the results. The dimensions displayed in the enterprise portrait can be defined by the user, and the results are aggregated by the query engine in real time according to various dimensions, which brings a high degree of freedom to users. Users can customize single enterprise and enterprise group analysis according to business needs.
[0149] S5: Enterprise portrait label content maintenance
[0150] The corporate portrait will change with the development of the enterprise and the passage of time. In order to obtain a more accurate corporate portrait, the portrait system also needs to continuously update the corporate characteristics according to the changes in behavioral data. The update of the corporate portrait requires setting reasonable trigger conditions, such as setting a scheduled task to periodically update the corporate portrait system. The sliding window filtering algorithm is an efficient user portrait update algorithm. Whenever the update condition is triggered, it only needs to move the time window, that is, it does not need to read all historical data, only read the new data within a period of time as the calculation object. Then the new results and the intermediate results calculated and stored by the historical user data are calculated to obtain the final results within the time point and stored. The advantage of this method is that it can reduce the amount of data read at a time and save system computing memory. Through this algorithm, the corporate portrait of the enterprise in each time period can be flexibly stored, and the tracing and query of the evolution of the corporate portrait within a certain time window can be supported. This is also an important part of the comprehensive and accurate grasp of the development of the enterprise by investment institutions such as fund companies.
[0151] The above description is only a preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, and is not used to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A method for matching enterprise financing funds based on data mining, characterized in that: The steps include: S1: Standardization of corporate credit data: Collect the credit information of target companies based on their names, integrate and standardize the massive amounts of credit information data; S2: Indexing of enterprise credit data: Indexing of enterprise credit data, deriving and screening the indicators based on RFM and unsupervised clustering algorithms, and establishing a preliminary indicator library; S3: Enterprise portrait construction and calculation: Build a dimension system to describe enterprise characteristics through enterprise portraits, and form enterprise portrait labels through dimension weight calculation; S4: Enterprise portrait tag display and retrieval: The display of enterprise portrait tags includes the display of individual enterprise portraits and tags and the display of enterprise group portraits and tags; enterprise portrait tag retrieval supports financial institutions to retrieve potential investment targets that meet the requirements by screening through portrait tags; S5: Enterprise portrait tag content maintenance: The portrait system continuously updates enterprise features according to changes in enterprise development data; a sliding window filtering algorithm is used to periodically update the enterprise portrait system; The data processing steps for enterprise credit data standardization include: Acquisition of credit information data Complete the construction of data warehouse model, definition of data source, data extraction, conversion, cleaning and loading ETL work; specific steps include: Definition of data source: uniformly manage and process interface data, database table data, and unstructured data into unified structured data that is easy to store, manage, and use; through cross-comparison and rule verification, transform data sources with inconsistent structures and incomplete data information into data that conform to unified structure standards and are placed in the data warehouse; Data extraction, conversion, cleaning and loading: In the process of data extraction, conversion, cleaning and loading ETL, an ETL strategy is established based on the characteristics of the credit data system in this modeling, including the credit data extraction frequency, credit data granularity, and credit data dimension; After the ETL is completed, monitoring and tracking the entire process is also an essential step; Among them, in the data extraction stage, the trigger update method is used for the data in the interface form; in the data in the database table form, the full update method is used for the data updated once a year or a month; for the data updated in real time, daily, and weekly, an incremental extraction mechanism based on data timestamp is used; in the data conversion stage, the original corporate credit data extracted by the above process is converted into data dimensions based on time, region, and industry, and then the relevant rules are used to verify the data quality and exclude the data that does not meet the analysis quality requirements; Credit data governance The detailed data from more than one data source system and the integrated summary data for analysis are stored in the central database; it is a complete library logically; from the perspective of hierarchy, credit data management is divided into four levels: source data layer, verification data layer, basic data layer, and product data layer: Source data layer: It is the basis of data in other layers. The source data layer processes and stores data from various data sources in a unified and standardized manner, and updates them using appropriate strategies. Verification data layer: When the data from the source data layer arrives, the data content is verified through the established data verification rules and verification process. The data that passes the verification is loaded into the central database, and the data that fails the verification is returned to the next layer. The erroneous data forms a data feedback message and is fed back to the management agency of the corresponding data source; Basic data layer: stores key credit reporting business data that has passed verification; the central database stores the data that has passed verification, and returns the erroneous data found through post-verification to the next layer, and finally forms an error feedback message to the reporting agency; Product data layer: After the basic data is stored and sorted, it is loaded into the product data layer according to the types of service products, established data extraction logic and mining analysis, so as to prepare the product data required for the service for users; Application of credit data After establishing a corporate credit database and conducting data governance and model building on it, the credit products provided are the overall corporate portrait display, corporate tags and corporate portrait tag retrieval; the corporate portrait tag retrieval will store the corporate tag results generated based on the corporate portrait in the database in the form of No-SQL data, support high-speed caching, and is suitable for fast query of corporate portrait data. The output records can be displayed through Web pages for customer acquisition analysis and investment decision-making of fund companies.
2. The method according to claim 1, characterized in that S2: Enterprise credit data indexation Use RFM indicator derivation and unsupervised clustering indicator derivation; R stands for Recency, F stands for Frequency, and M stands for Monetary; RFM indicator derivatives The specific derivation method is: Statistical objects: enterprise financial statistics, patent statistics, administrative information; Statistical window: The short-term window is based on the last 6 months, and the long-term window is from the last 1 year to the last 3 years; Aggregate functions: Continuous functions include maximum, minimum, and mean; Statistical variables: Continuous variables are variables that can take any value within a specified interval; Discrete variables refer to variables with finite fixed values and can be enumerated; new characteristic indicators are summarized and refined based on business knowledge to improve the predictive ability and relevance of characteristic indicators; Unsupervised clustering index derivation Based on the relationship and characteristics between the initial indicators, the KMeans unsupervised clustering algorithm is used to produce characteristic indicators; The new feature indicators generated in this stage are not processed in the derivation stage and are involved in the subsequent screening process. There are two specific derivation methods in this modeling: Indicator clustering based on time series: For credit data indicators with time series, clustering and indicator value calculation are performed according to their time dimension; Indicator clustering based on business explanation: for different credit indicators with correlation, cluster the selected credit indicator set at a specific time section, generate new characteristic indicators based on the clustering results, and label the rated enterprises with corresponding label values; Metric Filtering The goal of indicator filtering is to first select indicators for the data set according to the rules, and then classify the selected indicators according to business experience to form the dimensions of enterprise fundamentals, performance ability, debt repayment ability, growth, and operating ability; The steps of indicator filtering include: For continuous indicators, variance filtering method is used; For discrete indicators, use the same value filtering method; Calculate the correlation coefficient between indicators and delete one of the indicator groups with too high correlation coefficient; calculate the correlation coefficient between the indicator and the target variable and delete the indicator with too small correlation coefficient; The XGBoost algorithm is used to sort the importance of indicators and eliminate indicators whose importance is less than the threshold.
3. The method according to claim 1, characterized in that S3: Enterprise portrait construction and calculation, including three levels: fact labels, model labels, and prediction labels, as well as the construction methods of model labels and prediction labels.
4. The method according to claim 3, characterized in that The steps to build the model label are as follows: (1) Use the KMeans model to perform unsupervised clustering of enterprise indicators; use the KMeans unsupervised clustering algorithm to automatically cluster the time series indicators and the top ten indicators ranked by indicator importance using the XGBoost algorithm in the enterprise credit data indexing step, and summarize the enterprise labels based on the cluster center indicator characteristics of the enterprise clusters formed by the clustering results; (2) Use the association measurement model to discover enterprise association rules: Use the Apriori algorithm to mine association rules for enterprise indicators; (3) Use the analytic hierarchy process to comprehensively calculate the enterprise performance score: First, the enterprise's indicators and the labels obtained through model labels are used as indicators for hierarchical serialization. That is, according to the evaluation objectives, the enterprise is divided into six dimensions: operating status, development potential, financial status, innovation ability, fundamentals, and model label results. The indicators are then classified into these six dimensions to form a progressive and ordered hierarchical model. Then, the relative importance of each level of factors in the model is quantitatively expressed, and the discriminant matrix method is used to determine the weights of the relative importance order of all factors at each level. Finally, by comprehensively calculating the relative importance weights of each factor, the combined weights of the relative importance order of the lowest level plan layer and the highest level overall goal are obtained. This model is used as the basis for calculating the enterprise performance score.
5. The method according to claim 4, characterized in that Association rule mining can be divided into two categories: Enterprise risk mining: Based on the enterprise's own indicators, explore the relationship between enterprise indicators and enterprise risk performance, establish the association rules between enterprise indicator values and enterprise default risks, and form the enterprise's own risk association labels; Enterprise group risk association mining: For an enterprise set consisting of more than one enterprise, the Apriori algorithm is used to mine the risk contagion rules between enterprises. By establishing the relationship between the indicators of a single enterprise and the indicators and risk performance of other defaulting enterprises in the enterprise set, a risk association label for the enterprise group is formed.
6. The method according to claim 3, characterized in that The steps for constructing the predicted labels are as follows: (1) Data cleaning Missing value statistics: remove indicators with all missing values, and count the proportion of missing values in the two dimensions for the remaining indicators. Indicators with missing values greater than the threshold of 0.85 are classified as indicators with judgment and retained, and indicators with judgment and all missing indicators are removed; Parity rate statistics: For the remaining indicators from the previous round, missing values are filled with float ('-inf'), and the percentages of i) only one value, ii) only two values, iii) only two values and including -inf values, and different values are calculated respectively; the parity rate threshold is set to 0.8, and all indicators with a parity rate of 1 are removed, and indicators with a parity rate greater than 0.8 are retained as pending indicators, and the remaining indicators are all included in the indicator feature correlation analysis stage; VIF collinearity analysis: For the remaining missing values of indicators, fill them with -999999 and perform VIF collinearity analysis. The threshold is set to 10. After the indicator collinearity analysis, there are no remaining indicators with collinearity. Missing value filling: In this modeling, all missing values in the indicators are filled with a special value -999999 that is different from other existing indicator values. In the binning process, missing values are treated as independent values for processing and analysis; Normality test of indicators: A normality test was conducted on the distribution of indicators of the enterprise general credit model, and the skewness and kurtosis of the indicator distribution were calculated. Log transformation was performed on indicators with large skewness and kurtosis that were biased towards normal distribution; (2) Feature screening XGBoost feature screening: In this modeling process, the XGBoost method is used to screen features. The XGBoost model randomly adjusts model parameters and evaluates feature importance based on the nonlinear relationship between features. The final feature importance evaluation result takes the mean of all model trainings. WOE binning and IV value feature screening: In this modeling, the optimal binning method based on decision tree is used to perform WOE binning on the indicators entered into the model. The IV screening threshold is set to the corresponding threshold with the highest performance in the model comparison analysis, and the feature importance evaluation results obtained by XGBoost feature screening are combined to screen out qualified indicators; (3) Model training Stepwise regression In this modeling, forward regression training based on RMSE was performed according to the order of IV values obtained by sorting the importance of XGBoost features, and the performance of the trained model was compared and analyzed by the Accuracy, AUC, and RMSE indicators to select the optimal strategy, remove the features that reduce the AUC performance, and form the final modeling indicators; Model parameter optimization and prediction probability analysis The model parameter adjustment uses the HyperOpt automatic parameter search method, and the evaluation uses cross validation accuracy to measure and plot the output results of finding the optimal parameters in the process of model parameter optimization. Plotting model learning curve In this modeling process, by drawing the learning curves of each trained model, it can be concluded that the model does not have overfitting or underfitting; Scorecard Conversion The standard scorecard is converted according to the logistic regression model coefficients and the results of woe conversion of each feature; the output result of this modeling is the standard scorecard generated from the final predicted probability conversion, and this output result meets the ideal requirements of the standard scorecard results, that is, the scorecard distribution diagram finally presents a trend of approximate normal distribution.
7. The method according to claim 1, characterized in that S4: Enterprise portrait tag display and retrieval, including decomposing the query data and conducting parallel queries, combining the query results of each part during result processing, and forming the final result based on the enterprise portrait query; Based on the performance requirements of retrieval and query, the portrait system uses Elasticsearch as the search engine to realize the real-time query function of enterprise portraits.
8. The method according to claim 1, characterized in that S5: Maintenance of enterprise portrait label content, including continuous updating of enterprise portraits using a sliding window filtering algorithm; whenever an update condition is triggered, only the time window needs to be moved, that is, there is no need to read all historical data, only the new data within a period of time is read as the calculation object; then the new results are calculated and stored with the intermediate results of historical user data to obtain the final result within that time point and store it.
