Data processing method and system based on heterogeneous data fusion

Abstract

The invention provides a data processing method and system based on heterogeneous data fusion, and is applied to the technical field of automatic data processing. The method comprises the steps that a service application request for a target object is responded, multiple data calling requests are initiated in parallel, multi-dimensional data of the target object are obtained from heterogeneous data sources, and the heterogeneous data sources have different data modes and communication protocols; performing heterogeneous data fusion processing on the multi-dimensional data to generate an information fusion table; mapping the information fusion table into a high-dimensional feature matrix; performing dimension reduction processing on the high-dimensional feature matrix by adopting principal component analysis, and extracting a low-dimensional key sensitive feature set representing core features of the target object; and inputting the low-dimensional key sensitive feature set into the risk assessment model, and outputting a risk score of the target object. The problems that an existing data processing mode is difficult to ensure the accuracy and efficiency of service processing and cannot meet the automatic processing requirement under a large-scale service scene are solved.

Description

Technical Field

[0001] This invention relates to the field of automated data processing technology, and specifically to a data processing method and system based on heterogeneous data fusion. Background Technology

[0002] In the context of the deep integration of information technology and traditional industries, various business approval processes increasingly rely on multi-dimensional data support. Currently, financial institutions and enterprise management platforms often need to retrieve heterogeneous data such as user business data, performance records, and salary information from multiple independent systems for risk assessment when processing applications for loans, credit, onboarding, and internal approvals. These data sources are often scattered across different departments or third-party platforms. Existing methods often involve aggregating different data sources to merge them, but this simple aggregation approach often results in excessive data volume, low data processing efficiency, inability to meet the real-time requirements of business applications, and difficulty in guaranteeing data quality. Furthermore, existing risk assessment models struggle to adapt to the characteristics of multi-source heterogeneous data, further impacting the accuracy and efficiency of business processing and failing to meet the automated processing needs of large-scale business scenarios. Summary of the Invention

[0003] To overcome the above-mentioned technical problems, the present invention provides a data processing method and system based on heterogeneous data fusion.

[0004] On the one hand, the present invention provides a data processing method based on heterogeneous data fusion, comprising: In response to a business application request for a target object, multiple data call requests are initiated in parallel to obtain multidimensional data of the target object from heterogeneous data sources. The multidimensional data includes basic information, business data, work performance appraisal data, and salary data. The heterogeneous data sources have different data modes and communication protocols. Heterogeneous data fusion processing is performed on the acquired multidimensional data to generate an information fusion table; the information fusion table is mapped into a high-dimensional feature matrix; principal component analysis is used to reduce the dimensionality of the high-dimensional feature matrix to eliminate multicollinearity among features and extract a low-dimensional key sensitive feature set that characterizes the core features of the target object. The low-dimensional key sensitive feature set is input into the risk assessment model, and the risk score of the target object is output; wherein, the risk assessment model is an ensemble learning model trained on the historical data feature set after dimensionality reduction by principal component analysis.

[0005] Optionally, in response to a business request for the target object, multiple data access requests may be initiated in parallel, including: In response to a business application request for a target object, multiple data call requests are initiated in parallel through a distributed data access middleware. The distributed data access middleware includes a dynamic protocol adaptation module and a data format conversion engine. The dynamic protocol adaptation module adapts to different communication protocols of each heterogeneous data source by using a driver or functional component that matches each heterogeneous data source. The data format conversion engine is used to perform timestamp alignment and structure conversion on the acquired time-series data and non-time-series data with different modes.

[0006] Optionally, the step of performing heterogeneous data fusion processing on the acquired multidimensional data to generate an information fusion table includes: The multidimensional data is cleaned, standardized, and integrated using distributed computing nodes to generate an information fusion table.

[0007] Optionally, the association integration includes: Using the target field as the entity identifier, the system calculates cosine similarity to match the same entity object in different data sources and performs data deduplication. If the content of the same field is inconsistent in different data sources, a weighted voting strategy is used to merge the content of the same field in different data sources and the result of the merge calculation is used as the field value of the field in the information fusion table. The weighted voting strategy is determined based on the correlation between the evaluation target and different data sources, and the correlation is determined based on the Pearson correlation coefficient between the feature values ​​of different data sources and the risk indicators of the evaluation target.

[0008] Optionally, principal component analysis is used to reduce the dimensionality of the high-dimensional feature matrix to eliminate multicollinearity among features and extract a low-dimensional set of key sensitive features characterizing the core features of the target object, including: The data in the high-dimensional feature matrix are standardized to obtain a standardized matrix; Based on the eigenvalue decomposition of the covariance matrix of the standardized matrix, the low-dimensional key sensitive feature set of the target object is selected by the cumulative variance contribution rate.

[0009] Optionally, before inputting the low-dimensional key sensitive feature set into the risk assessment model and outputting the risk score of the target object, the method further includes: Based on the correlation between the pre-set candidate risk factors and the risk assessment results in historical assessment cases of different objects, each assessment risk factor is selected. Based on the correlation between each assessment risk factor and the low-dimensional key sensitive feature set, a corresponding risk calculation module is constructed for each assessment risk factor; An ensemble learning model is trained using historical assessment cases from different subjects to obtain a risk assessment module; The risk assessment model is obtained based on each risk calculation module and the risk assessment module.

[0010] Optionally, the low-dimensional key sensitive feature set is input into the risk assessment model to output a risk score for the target object, including: The low-dimensional key sensitive feature set is input into each risk calculation module to calculate risk factors and obtain the corresponding assessment risk factor values. The risk assessment module inputs the values ​​of each risk assessment factor to perform risk assessment and obtains the risk score of the target object. Specifically, a trigger mechanism is added at the entry point of the business process to automatically invoke the risk assessment model for risk assessment; the risk assessment is based on the correlation coefficient between different risk assessment factors.

[0011] Optionally, the risk assessment module is a decision tree model; the step of inputting the values ​​of each risk assessment factor into the risk assessment module to perform risk assessment and obtain the risk score of the target object includes: For each node of the decision tree model, the classification threshold of the node and the correlation coefficient between the node and the risk assessment factors corresponding to the adjacent nodes are used to select a path for each input risk assessment factor value until the leaf node of the decision tree model is reached, so as to obtain the risk score of the business application request. Each node corresponds to a risk assessment factor.

[0012] Optionally, after outputting the risk score of the target object, the method further includes: An approval decision signal is generated based on the risk score and returned to the business application initiation terminal to complete the automated approval process.

[0013] Optionally, after outputting the risk score of the target object, the method further includes: Based on the business identifier of the business application request, the approval result of the business application request is associated with the corresponding assessment risk factor value; Statistical analysis is performed on the deviation between risk scores and corresponding approval results based on the assessed risk factor values. The model parameters of each risk calculation module and the risk assessment module are then optimized and adjusted based on the statistical analysis results.

[0014] On the other hand, the present invention also provides a data processing system based on heterogeneous data fusion, comprising: The data acquisition module is used to respond to business application requests for a target object by initiating multiple data call requests in parallel, and to obtain multidimensional data of the target object from heterogeneous data sources. The multidimensional data includes basic information, business data, work performance appraisal data, and salary data. The heterogeneous data sources have different data modes and communication protocols. The data processing module is used to perform heterogeneous data fusion processing on the acquired multidimensional data to generate an information fusion table; map the information fusion table into a high-dimensional feature matrix; and use principal component analysis to perform dimensionality reduction processing on the high-dimensional feature matrix to eliminate multicollinearity between features and extract a low-dimensional key sensitive feature set that characterizes the core features of the target object. The evaluation module is used to input the low-dimensional key sensitive feature set into the risk assessment model and output the risk score of the target object; wherein, the risk assessment model is an ensemble learning model trained on the historical data feature set after dimensionality reduction by principal component analysis.

[0015] Optionally, the data acquisition module is specifically used for: In response to a business application request for a target object, multiple data call requests are initiated in parallel through a distributed data access middleware. The distributed data access middleware includes a dynamic protocol adaptation module and a data format conversion engine. The dynamic protocol adaptation module adapts to different communication protocols of each heterogeneous data source by using a driver or functional component that matches each heterogeneous data source. The data format conversion engine is used to perform timestamp alignment and structure conversion on the acquired time-series data and non-time-series data with different modes.

[0016] Optionally, the data processing module includes: The data fusion submodule is used to clean, standardize, and integrate the multidimensional data through distributed computing nodes to generate an information fusion table.

[0017] Optionally, the data fusion submodule is specifically used for: Using the target field as the entity identifier, the system calculates cosine similarity to match the same entity object in different data sources and performs data deduplication. If the content of the same field is inconsistent in different data sources, a weighted voting strategy is used to merge the content of the same field in different data sources and the result of the merge calculation is used as the field value of the field in the information fusion table. The weighted voting strategy is determined based on the correlation between the evaluation target and different data sources, and the correlation is determined based on the Pearson correlation coefficient between the feature values ​​of different data sources and the risk indicators of the evaluation target.

[0018] Optionally, the data processing module includes a dimensionality reduction submodule, which is used for: The data in the high-dimensional feature matrix are standardized to obtain a standardized matrix; Based on the eigenvalue decomposition of the covariance matrix of the standardized matrix, the low-dimensional key sensitive feature set of the target object is selected by the cumulative variance contribution rate.

[0019] Optionally, it also includes a model building module, which is used for: Based on the correlation between the pre-set candidate risk factors and the risk assessment results in historical assessment cases of different objects, each assessment risk factor is selected. Based on the correlation between each assessment risk factor and the low-dimensional key sensitive feature set, a corresponding risk calculation module is constructed for each assessment risk factor; An ensemble learning model is trained using historical assessment cases from different subjects to obtain a risk assessment module; The risk assessment model is obtained based on each risk calculation module and the risk assessment module.

[0020] Optionally, the evaluation module includes: The risk factor calculation submodule is used to input the low-dimensional key sensitive feature set into each risk calculation module to calculate the risk factors and obtain the corresponding assessment risk factor values. The assessment submodule is used to input the values ​​of each assessment risk factor into the risk assessment module to conduct risk assessment and obtain the risk score of the target object; Specifically, a trigger mechanism is added at the entry point of the business process to automatically invoke the risk assessment model for risk assessment; the risk assessment is based on the correlation coefficient between different risk assessment factors.

[0021] Optionally, the risk assessment module is a decision tree model; the assessment submodule is specifically used for: For each node of the decision tree model, the classification threshold of the node and the correlation coefficient between the node and the risk assessment factors corresponding to the adjacent nodes are used to select a path for each input risk assessment factor value until the leaf node of the decision tree model is reached, so as to obtain the risk score of the target object. Each node corresponds to a risk assessment factor.

[0022] Optionally, it also includes: The approval module is used to generate an approval decision signal based on the risk score and return the approval decision signal to the business application initiation terminal to complete the automated approval process.

[0023] Optionally, it also includes an optimization module, which is used for: Based on the business identifier of the business application request, the approval result of the business application request is associated with the corresponding assessment risk factor value; Statistical analysis is performed on the deviation between risk scores and corresponding approval results based on the assessed risk factor values. The model parameters of each risk calculation module and the risk assessment module are then optimized and adjusted based on the statistical analysis results.

[0024] On the other hand, the present invention also provides an electronic device, comprising: at least one processor and a memory; the memory and the processor are connected via a bus; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the method described in any of the foregoing is implemented.

[0025] On the other hand, the present invention also provides a readable storage medium having an executable program stored thereon, wherein when the executable program is executed, it implements the method described in any one of the above.

[0026] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a data processing method and system based on heterogeneous data fusion. Employing a parallel invocation strategy, it can simultaneously initiate requests to multiple data sources with different data patterns and communication protocols. Compared to serial invocation, this significantly shortens data retrieval time, enabling real-time business responses. By generating an information fusion table through heterogeneous data fusion processing, it solves the quality problems caused by differences in data patterns among heterogeneous data sources. By mapping the information fusion table to a high-dimensional feature matrix and performing principal component analysis to reduce the dimensionality of the high-dimensional feature matrix, it achieves quantifiable transformation of the fusion table. This retains key information representing the target object in the data while eliminating multicollinearity between features, reducing redundant data, improving data processing efficiency, and ensuring data validity.

[0027] This invention uses an ensemble learning model trained on a historical data feature set reduced by principal component analysis as a risk assessment model. By matching the training input with a low-dimensional key sensitive feature set, the model's adaptability to multi-source heterogeneous data features is improved. The low-dimensional key sensitive feature set is then input into the risk assessment model. By combining low-dimensional features with ensemble learning, the model's computational load is reduced and the assessment speed is improved while accurately uncovering the risk correlation patterns of the target object. This enables automated risk assessment in large-scale business scenarios. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a data processing method based on heterogeneous data fusion according to the present invention. Figure 2This is a block diagram of an electronic device according to the present invention. Detailed Implementation

[0029] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0030] Example 1 This invention provides a data processing method based on heterogeneous data fusion, the schematic diagram of which is shown below. Figure 1 As shown, the method includes: Step S110: In response to a business application request for the target object, multiple data call requests are initiated in parallel to obtain multidimensional data of the target object from heterogeneous data sources. The multidimensional data includes basic information, business data, work performance appraisal data, and salary data. The heterogeneous data sources have different data modes and communication protocols. Step S120: Heterogeneous data fusion processing is performed on the acquired multidimensional data to generate an information fusion table; the information fusion table is mapped to a high-dimensional feature matrix; principal component analysis is used to reduce the dimensionality of the high-dimensional feature matrix to eliminate multicollinearity between features and extract a low-dimensional key sensitive feature set that characterizes the core features of the target object. Step S130: Input the low-dimensional key sensitive feature set into the risk assessment model and output the risk score of the target object; wherein, the risk assessment model is an ensemble learning model trained on the historical data feature set after dimensionality reduction by principal component analysis.

[0031] In this example implementation, the target object is the subject of the business application request, that is, the person who needs to handle the business or the person who submits the business application. The application type of the business application request can include various applications in the financial credit field or employee onboarding applications and various internal approval applications in the enterprise service field (such as contract approval, order execution approval, income certificate approval, and overseas travel approval). Heterogeneous data sources can include relational databases, non-relational databases, and file servers (FTP / SFTP). Basic information can include name, date of employment, company, department, position, job level, contact information, and job change records. For contract approval, the corresponding business data may include contract number, contract type, contract content, etc.; for order execution approval, the business data includes basic order information (such as order number, order date, order type, order status, order priority, etc.), transaction-related party information (such as customer information, internal responsible person information), and financial and pricing information (such as price list, product details and pricing, additional fees, payment terms, estimated profit, etc.); for income verification approval and overseas travel approval, the corresponding business data may be the relevant business data of the requesting party in recent months. For example, when the requesting party is a salesperson, the business data may be their sales data in recent months. Work performance appraisal data may include performance bonuses, performance indicators, etc. Salary data may include employee basic salary, bonuses, allowances, salary adjustment records, etc. The information fusion table is a structured table after heterogeneous data fusion, which may contain two dimensions: rows and columns. Row dimensions can correspond to a target object (such as a user applying for a loan or an employee applying for employment); column dimensions can correspond to an attribute field of the target object (such as "age" or "education level" in basic information, "historical loan count" in business data, "quarterly assessment score" in performance data, and "average monthly income" in salary data). The process of mapping the information fusion table to a high-dimensional feature matrix can include non-numerical field transformation (such as converting non-numerical fields to numerical values ​​through one-hot encoding or label encoding), feature dimension expansion, and normalization. Feature dimension expansion is used to complete implicit features. Specifically, it can derive implicit features from the original data based on business logic, such as deriving the income-age ratio based on basic information and salary data to reflect income efficiency, or deriving cumulative income capacity based on average monthly income and years of service; it can also derive features such as historical loan count / quarterly assessment score based on business data and performance data to correlate business risk with performance, or the standard deviation of quarterly assessment scores to characterize performance stability; it can also perform cross-field combination derivation, such as whether someone has a high level of education and high income, etc. More implicit features can be mined through feature dimension expansion. Normalization is used to eliminate the influence of dimensions and ensure the comparability of matrix features. High-dimensional feature matrices can be generated by arranging samples and features.Risk assessment models can be ensemble learning models trained on historical data feature sets after dimensionality reduction through principal component analysis, such as decision trees (e.g., CART, ID3), random forests, support vector machines (SVM), logistic regression, and neural network models. For example, risk assessment reports can be automatically generated based on the output of the constructed risk assessment model (e.g., risk scores).

[0032] In some example implementations, in response to a business request for the target object, step S110 initiates multiple data retrieval requests in parallel, including: In response to a business application request for a target object, multiple data call requests are initiated in parallel through a distributed data access middleware. The distributed data access middleware includes a dynamic protocol adaptation module and a data format conversion engine. The dynamic protocol adaptation module adapts to different communication protocols of each heterogeneous data source by using a driver or functional component that matches each heterogeneous data source. The data format conversion engine is used to perform timestamp alignment and structure conversion on the acquired time-series data and non-time-series data with different modes.

[0033] In this example implementation, a distributed data access middleware specifically designed for parallel calls can be developed. This middleware incorporates a dynamic protocol adaptation module and a data format conversion engine. The dynamic protocol adaptation module adapts to the different communication protocols of each heterogeneous data source by using drivers or functional components matched to each data source. For example, it adapts to the SQL protocol of relational databases using a pre-defined JDBC / ODBC driver library, the BSON protocol using MongoDB JavaDriver, and the TCP / IP protocol of FTP / SFTP using Apache Commons Net components, thus resolving communication protocol differences between different data sources. The data format conversion engine performs timestamp alignment and structured transformation on the acquired time-series and non-time-series data with different patterns. For example, it uniformly converts the acquired multidimensional data into JSON format, achieving format unification for heterogeneous data. This example, by developing a distributed data access middleware for parallel calls to acquire heterogeneous data, can significantly improve data acquisition speed, addressing the latency issue in the data retrieval stage of business applications from the source and meeting real-time requirements.

[0034] In some example implementations, step S120, which involves performing heterogeneous data fusion processing on the acquired multidimensional data to generate an information fusion table, includes: The multidimensional data is cleaned, standardized, and integrated using distributed computing nodes to generate an information fusion table.

[0035] In this example implementation, distributed computing nodes are used to perform cleaning, standardization, and correlation integration operations on multidimensional data. Cleaning operations can include outlier handling, missing value imputation, and removal of duplicate and invalid data. Specifically, outlier handling involves detecting outliers using statistical analysis or visualization methods. For example, for numeric fields, statistical methods (such as mean ± 3 standard deviation, quartile rules of box plots, etc.) can be used to detect outliers. For categorical fields, outliers can be detected by checking whether the field value is within a predefined range of legal values. If an outlier is clearly unreasonable and cannot be corrected, the record containing the outlier can be directly deleted. If the outlier may be due to a data entry error, it can be corrected to a reasonable value according to business logic. For example, a clearly excessively high salary value can be corrected to the average or median of the field. Formatting consistency is ensured: all data fields have consistent formats. For example, date fields should be uniformly converted to a standard date format; numeric fields should ensure there are no extra spaces or special characters, which can be handled using string methods such as trim() and replace(). Missing value imputation: Check for missing values ​​in the data. If the missing field is crucial for subsequent analysis, records containing missing values ​​can be directly deleted. Otherwise, imputation methods can be used. For numeric fields, missing values ​​can be filled with the mean, median, or mode. For categorical fields, missing values ​​can be filled with the most frequent value or default value. For date fields, a reasonable default value (such as the current date or a fixed date) can be filled based on business logic. For example, removing extreme values ​​that are clearly outside the reasonable range from salary data, and completing or removing records with missing key fields in basic information to ensure the initial validity of the data. Because data formats and units differ across data sources—for example, transaction amounts recorded in different systems within business data might be measured in "yuan" and "ten thousand yuan" respectively, and performance evaluation data might be presented in either a percentage or a grade system—standardization converts these data into a unified, predefined format and unit, ensuring comparability and integrability across different sources. Association integration involves linking cleaned and standardized data according to a predefined logic. For instance, based on the unique identifier of a target object (such as an ID number or employee ID), its corresponding basic information, business data, performance data, and salary data can be linked to the same data entry, forming a unified and complete information fusion table. Data processing via distributed computing nodes fully leverages the parallel computing capabilities of the distributed architecture, significantly improving the efficiency of data cleaning, standardization, and association integration, avoiding the slow processing speed of traditional single-node methods when dealing with massive amounts of heterogeneous data. Simultaneously, association integration establishes effective relationships between data, ensuring high-quality and structurally sound information fusion table data, laying a solid foundation for subsequent processing.

[0036] For example, the association integration includes: Using the target field as the entity identifier, the system calculates cosine similarity to match the same entity object in different data sources and performs data deduplication. If the content of the same field is inconsistent in different data sources, a weighted voting strategy is used to merge the content of the same field in different data sources and the result of the merge calculation is used as the field value of the field in the information fusion table. The weighted voting strategy is determined based on the correlation between the evaluation target and different data sources, and the correlation is determined based on the Pearson correlation coefficient between the feature values ​​of different data sources and the risk indicators of the evaluation target.

[0037] In this example implementation, the target field is typically a field that uniquely identifies an entity, such as the target entity's ID number or employee number. In a financial loan scenario, this might be a customer's unique credit number, or in an internal enterprise approval scenario, it might be an employee's work number. Cosine similarity calculation converts the feature data associated with the target field from different data sources (such as name and address in basic information, and transaction record identifiers in business data) into vector form. Then, it calculates the cosine similarity between these vectors. When the similarity reaches a preset threshold, the corresponding records are determined to belong to the same entity. Subsequently, duplicate records are deduplicated, retaining the record with the most complete and accurate information, or non-conflicting information from multiple records is merged, removing redundant content. The weighted voting strategy is determined based on the degree of association between the evaluation target and different data sources. This degree of association is derived by calculating the Pearson correlation coefficient between the feature values ​​of different data sources and the risk indicators of the evaluation target. For example, in risk assessment for loan business, a risk indicator might be the customer's delinquency rate. In this case, Pearson correlation coefficients are calculated between the delinquency rate and characteristic values ​​from different data sources such as the bank's internal credit system, third-party credit reporting platforms, and customer transaction records (e.g., historical delinquency counts in the credit system, credit scores from credit reporting platforms, and average monthly income / expense difference in transaction records). For income verification applications, a risk indicator might be the employee's turnover probability. In this case, Pearson correlation coefficients are calculated between the employee's turnover probability and characteristic values ​​from different data sources such as the payroll system, performance appraisal system, and corporate human resource management platform (e.g., historical salary adjustments in the payroll system, performance bonuses in the performance appraisal system, and education level and age in the corporate human resource management platform). The larger the absolute value of the correlation coefficient, the stronger the correlation between the data source and the assessment target, and the greater its weight in the weighted voting. By configuring risk indicators and characteristic values ​​for different business applications, the degree of correlation between different data sources and the assessment target is determined. This example uses cosine similarity calculation to accurately match and deduplicate the same entity object, avoiding data redundancy caused by repeated storage of the same entity information in different data sources, while ensuring the uniqueness and consistency of the same entity data. The application of a weighted voting strategy fully considers the correlation between different data sources and the evaluation target, making the field content fusion results more targeted and accurate, and improving the data quality of the information fusion table after association and integration.

[0038] In some example implementations, step S120 employs principal component analysis to reduce the dimensionality of the high-dimensional feature matrix, thereby eliminating multicollinearity among features and extracting a low-dimensional set of key sensitive features characterizing the core features of the target object, including: The data in the high-dimensional feature matrix are standardized to obtain a standardized matrix; Based on the eigenvalue decomposition of the covariance matrix of the standardized matrix, the low-dimensional key sensitive feature set of the target object is selected by the cumulative variance contribution rate.

[0039] In this example implementation, standardization can unify data with different feature types and varying magnitudes and numerical ranges. For instance, standardization converts the data for each feature into a standard normal distribution. This can be achieved by subtracting the mean of the feature from each data point and then dividing by the standard deviation of that feature. This ensures that all feature data in the high-dimensional feature matrix are of the same magnitude, eliminating the interference of magnitude differences on subsequent analysis and yielding a standardized matrix. The elements in the covariance matrix reflect the degree of linear correlation between different features. Eigenvalue decomposition of the covariance matrix yields eigenvalues ​​and corresponding eigenvectors. The eigenvalue represents the degree to which the principal component represented by the eigenvector can explain data variation; the larger the eigenvalue, the stronger the explanatory power of the corresponding principal component for data variation. Then, the principal components are sorted in descending order of eigenvalues, and the variance contribution rate (i.e., the ratio of the eigenvalue of this principal component to the sum of the eigenvalues ​​of all principal components) and the cumulative variance contribution rate (i.e., the sum of the variance contribution rates of the first k principal components) of each principal component are calculated. When the cumulative variance contribution rate reaches a threshold, principal component selection is stopped. At this point, the selected first k principal components can effectively reduce the feature dimensionality while retaining most of the data information. The eigenvectors corresponding to the selected k principal components are used as projection matrices, and the standardized matrix is ​​projected onto these projection matrices to obtain a low-dimensional key sensitive feature set representing the core features of the target object. This example, by using covariance matrix eigenvalue decomposition and cumulative variance contribution rate to select principal components, not only effectively eliminates multicollinearity among features and avoids redundant features interfering with subsequent risk assessment model training and prediction, but also significantly reduces feature dimensionality, reduces data processing volume and model computational complexity, and improves data processing efficiency and model running speed. Meanwhile, the selected low-dimensional key sensitive feature set retains the core information in the original high-dimensional data, which can accurately characterize the key features of the target object and provide high-quality feature data for subsequent risk scoring. This effectively solves the problems of complex high-dimensional data processing and the impact of multicollinearity between features on the accuracy of assessment in existing technologies.

[0040] In some example implementations, before inputting the low-dimensional key sensitive feature set into the risk assessment model and outputting the risk score of the target object, the method further includes: Based on the correlation between the pre-set candidate risk factors and the risk assessment results in historical assessment cases of different objects, each assessment risk factor is selected. Based on the correlation between each assessment risk factor and the low-dimensional key sensitive feature set, a corresponding risk calculation module is constructed for each assessment risk factor; An ensemble learning model is trained using historical assessment cases from different subjects to obtain a risk assessment module; The risk assessment model is obtained based on each risk calculation module and the risk assessment module.

[0041] In this example implementation, candidate risk factors differ across different business scenarios. For instance, for financial loan business, candidate risk factors may include the target's income level, debt situation, credit history, job stability, and past business defaults. For income verification application business, candidate risk factors may include salary adjustment risk factors (the percentage of salary adjustment), job change risk factors (the ratio of the number of times an employee changes jobs within a certain period to their years of service), performance change risk factors (the trend of employee performance rating changes), employee turnover risk factors (such as the weighted sum of employee leave frequency and satisfaction survey results), and salary level differentiation factors (comparing the difference between employee salaries and the market average). For overseas travel application business, candidate risk factors may include background risk factors, travel risk factors, and expense budget risk factors. Among these, background risk factors can be determined based on the employee's personal background information, such as whether the employee has a criminal record or is involved in legal disputes, which will affect the review of overseas travel applications. Background risk factors can be assessed through qualitative analysis. A mapping relationship can be established in advance between different combinations of background parameters (such as whether there are negative records, whether there are legal disputes, etc.) and background risk factor values. The risk level can be determined through reviewing internal employee files and conducting external background investigations. For example, if an employee has a negative record or legal disputes, the background risk factor value can be set to a higher value (e.g., 1); otherwise, a lower value (e.g., 0). The travel risk factor characterizes whether an employee's travel arrangements are detailed, reasonable, and meet actual needs. For example, excessively long travel times, too many destinations, or unreasonable travel arrangements may arouse suspicion from the auditing department. This risk factor is also mainly assessed through qualitative analysis, and can be comprehensively evaluated based on factors such as the reasonableness of the itinerary, the number of destinations, and travel time. A mapping relationship can be established in advance between different travel parameters (distance to destinations, mode of transportation, number of destinations, travel time, etc.) and travel risk factor values, and the travel risk factor is determined through this mapping relationship. For example, if the travel arrangements are reasonable, the risk factor value can be set to a lower value (e.g., 0.2); if the travel arrangements are unreasonable, the risk factor value can be set to a higher value (e.g., 0.8). The expense budget risk factor is used to characterize whether the budget for employee overseas travel expenses is reasonable and complies with the company's financial regulations. Excessive expenses or expenses not in line with the budget may lead to approval failure or require further explanation of the source of funds. This risk factor can be assessed through quantitative analysis. A budget threshold can be set; if the actual budget exceeds this threshold, the risk factor value increases. For example, the expense budget risk factor value can be determined by the percentage by which the actual budget exceeds the set threshold. If the actual budget is less than or equal to the budget threshold, the risk factor value is 0; if the actual budget exceeds the budget threshold, the risk factor value is the result calculated above.This example collects a sufficient number of historical assessment cases, extracts candidate risk factor data and corresponding risk assessment results (such as approval status, risk level, etc.) from each case, and then uses statistical methods (such as Spearman rank correlation analysis) to calculate the correlation between each candidate risk factor and the risk assessment results. When the correlation between a candidate risk factor and the risk assessment results reaches a preset threshold, it indicates that the candidate risk factor has a significant impact on the risk assessment results, and it is selected as the assessment risk factor. Based on the association between each assessment risk factor and the low-dimensional key sensitive feature set, corresponding calculation logic and formulas are designed to construct a risk calculation module. Each risk calculation module is specifically used to calculate the corresponding assessment risk factor value based on the relevant feature data in the low-dimensional key sensitive feature set.

[0042] Next, historical data of each selected risk factor and corresponding risk assessment results are extracted from historical assessment cases of different objects as training data. The training data is divided into a training set and a validation set. The ensemble learning model is trained using the training set. During training, each base learner learns the mapping relationship between the risk factor and the risk assessment result based on the training data. Then, the prediction results of multiple base learners are integrated using a preset ensemble strategy (such as voting, weighted average, stacking, etc.) to form the prediction result of the ensemble learning model. The model's prediction performance is evaluated using the validation set. A corresponding risk assessment model can be trained for each business scenario. For example, if the risk assessment module is a decision tree, for a risk factor A, Gini impurity is used as the feature selection criterion. Gini impurity measures the purity of the dataset; the smaller the value, the purer the dataset. For example, the decision tree for income verification applications is constructed: this decision tree is used to assess the risk of employee income verification applications. The dataset / training set contains the following features: salary adjustment risk factor (continuous feature), job change risk factor (discrete feature), performance change risk factor (discrete feature), and employee turnover risk factor (discrete feature). (1) Select the best feature: Assuming Gini impurity is chosen as the feature selection criterion, calculate the Gini impurity of each feature. It is found that "salary adjustment risk factor" has the smallest Gini impurity, so it is chosen as the best feature. (2) Split the data: For the continuous feature "salary adjustment risk factor", find the best split point t. Assume that the best split point t is 0.2 (representing 20% ​​of the salary adjustment risk factor). Based on this split point, divide the training set into two subsets: subset 1 (salary adjustment risk factor ≤ 0.2) and subset 2 (salary adjustment risk factor > 0.2). (3) Generate a node: Generate a node in the decision tree, recording the split feature as "salary adjustment risk factor", and the split conditions as "≤ 0.2" and ">0.2". (4) Recursive subset: Repeat the above process for subset 1 and subset 2 respectively. In subset 1, assuming that "job change risk factor" becomes the best feature, continue to split the data and generate new nodes. In subset 2, assuming a “salary level differentiation factor”, the data is divided according to the salary level differentiation factor and new nodes are generated. (5) Iteratively construct the tree structure: Continue to recursively execute the above steps for each generated subset until the stopping condition is met (such as the records in the subset all belong to the same category or the preset maximum tree depth is reached), and obtain the decision tree model of the income verification application. Finally, based on each risk calculation module and risk assessment module, a complete risk assessment model is formed. Each risk calculation module is responsible for converting the low-dimensional key sensitive feature set into the corresponding assessment risk factor value, and the risk assessment module is responsible for outputting the final risk score based on these assessment risk factor values.This example solves the problem of low accuracy and reliability of risk assessment caused by poor model adaptability by screening risk factors that are highly correlated with the risk assessment results and combining various risk calculation modules and risk assessment modules.

[0043] In some example implementations, S130 inputs the low-dimensional key sensitive feature set into the risk assessment model and outputs a risk score for the target object, including: The low-dimensional key sensitive feature set is input into each risk calculation module to calculate risk factors and obtain the corresponding assessment risk factor values. The risk assessment module inputs the values ​​of each risk assessment factor to perform risk assessment and obtains the risk score of the target object. Specifically, a trigger mechanism is added at the entry point of the business process to automatically invoke the risk assessment model for risk assessment; the risk assessment is based on the correlation coefficient between different risk assessment factors.

[0044] In this example implementation, after the low-dimensional key sensitive feature set is input into the risk assessment model, the model will allocate the feature data related to each assessment risk factor to the corresponding risk calculation module. For example, the credit status assessment risk factor value is obtained by weighting the feature values ​​of historical credit default records and credit score feature values. Different assessment risk factors may have certain correlations. For example, there is usually a positive correlation between income level assessment risk factors and repayment ability assessment risk factors. This correlation will affect the risk assessment result. Therefore, it is necessary to first calculate the correlation coefficient between different assessment risk factors, such as calculating the Pearson correlation coefficient based on historical data of each assessment risk factor in historical assessment cases, and inputting it along with the assessment risk factor values ​​into the risk assessment module for risk assessment. The trigger mechanism added at the business process entry point can automatically call the risk assessment model, starting the risk assessment process without manual intervention.

[0045] For example, the risk assessment module is a decision tree model; the step of inputting the values ​​of each risk assessment factor and the correlation coefficients between each risk assessment factor into the risk assessment module to perform risk assessment and obtain the risk score of the business application request includes: For each node of the decision tree model, the classification threshold of the node and the correlation coefficient between the node and the risk assessment factors corresponding to the adjacent nodes are used to select a path for each input risk assessment factor value until the leaf node of the decision tree model is reached, so as to obtain the risk score of the business application request. Each node corresponds to an assessment risk factor.

[0046] In this example implementation, when the risk assessment module is a decision tree model, the values ​​of each assessment risk factor and the correlation coefficients between them are input into the risk assessment module for risk assessment. The decision tree model consists of a root node, internal nodes, and leaf nodes. Each node corresponds to an assessment risk factor (i.e., each node's corresponding feature is an assessment risk factor). The root node is the starting point of the model, internal nodes are used to judge and branch the assessment risk factor values, and leaf nodes represent the final risk score. During the assessment process, for each node in the decision tree model, the classification threshold of that node and the correlation coefficients between that node and the corresponding risk assessment factors of adjacent nodes are used to select a path for each input assessment risk factor value. Starting from the root node, the assessment risk factor corresponding to the root node has a preset classification threshold, which is determined based on historical assessment case data during model training and is used to judge the input assessment risk factor value. At the same time, there is a correlation coefficient between the root node and the corresponding features (i.e., the corresponding assessment risk factors) of the adjacent next-level nodes (child nodes). This coefficient reflects the degree of association between the assessment risk factors corresponding to the root node and the assessment risk factors corresponding to the child nodes. When selecting a path, the system not only compares the risk factor value corresponding to the root node with the classification threshold (selecting a specific child node path if the value is greater than the threshold, and another if it's less than or equal to the threshold), but also adjusts the path selection based on correlation coefficients. For example, if the correlation coefficient between the risk factor corresponding to the root node and the risk factor corresponding to a child node is high, it indicates a close relationship between the two. When the risk factor value of the root node is close to the classification threshold, the system is more inclined to select that child node path to ensure that the path selection fully considers the relationships between factors and avoids path bias caused by judging a single factor. Following this method, path selection is performed sequentially for each node, starting from the root node, until a leaf node is reached, resulting in a risk score. This example inputs both the risk factor value and the correlation coefficient between factors into the risk assessment module, enabling the module to comprehensively consider the impact of each factor and their interrelationships on risk. This avoids the bias caused by using factor values ​​alone and improves the reliability of the risk score.

[0047] In some example implementations, after outputting the risk score of the target object, the method further includes: An approval decision signal is generated based on the risk score and returned to the business application initiation terminal to complete the automated approval process.

[0048] In this example implementation, the risk assessment model outputs a risk score, which is converted into a risk level based on a preset threshold. For example, low risk: riskScore < 0.3, medium risk: 0.3 ≤ riskScore < 0.7, high risk: riskScore ≥ 0.7. If the risk assessment result is low risk, approval is granted, and the matching of the business application request items can be triggered. Based on the matched items, a corresponding business voucher is generated, and based on the business voucher, an approval decision signal is generated. If the risk assessment result is medium risk, supplementary review is required, and an approval decision signal is generated based on the supplementary review result. If the risk assessment result is high risk, approval is rejected, and an approval decision signal is generated based on the review result.

[0049] In addition, a risk assessment report can be automatically generated based on the assessment results. The report content includes a report title, report date, risk score, risk level, risk factor analysis, and recommended measures. Report title: for example, "Employee Risk Assessment Report"; Report date: the date the report was generated; Employee information: including basic information such as employee name, position, and date of employment; Risk factor analysis: listing the value of each risk factor and its impact on the risk assessment; Risk score / level: displaying the risk score or risk level output by the model; Recommended measures: providing corresponding recommendations based on the risk level (Low risk: no additional action required; Medium risk: further review recommended; High risk: additional approval process recommended). The report can also be exported in HTML, PDF, or Excel format, providing visual charts (such as bar charts and line charts) to intuitively display risk factors. Specific implementation methods include using HTML tags (such as...). <h1>、 、

[0050]

[0051]

[0052]

[0053]

[0054]

[0055]

[0056]

[0057]

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] Figure 2

[0066]

[0067]

[0068]

[0069] Figure 1 Figure 1

[0070] Figure 1 Figure 1

[0071] Figure 1 Figure 1

[0072] The report structure is built, and a PDF file is generated using a Java PDF library (such as iText or Apache PDFBox), and an Excel file is generated using a Java Excel library (such as Apache POI). The report can be sent to relevant personnel via Java email API (such as JavaMail) or integrated into the company's approval process for easy viewing and processing by relevant personnel. For example, the risk factors and approval results of an employee's income certificate application in this invention are shown in Table 1: In Table 1, Approval ID 0001: Salary adjustment risk factor (0.05), Job change risk factor (low), Performance change risk factor (stable), Employee turnover risk factor (low), Salary level differentiation factor (0.05), Approval result (passed); Approval ID 0002: Salary adjustment risk factor (0.25), Job change risk factor (medium), Performance change risk factor (fluctuating), Employee turnover risk factor (low), Salary level differentiation factor (0.1), Approval result (passed). For Approval ID 0003: Salary adjustment risk factor (0.3), Job change risk factor (high), Performance change risk factor (stable), Employee turnover risk factor (high), Salary level differentiation factor (0.15), Approval result (requires additional review); Approval ID 0004: Salary adjustment risk factor (0.4), Job change risk factor (high), Performance change risk factor (volatile), Employee turnover risk factor (high), Salary level differentiation factor (0.2), Approval result (rejected). In some example implementations, after outputting the risk score of the target object, the method further includes: associating the approval result of the business application request with the corresponding assessment risk factor value based on the business identifier of the business application request; performing statistical analysis on the deviation between the risk score and the corresponding approval result based on the assessment risk factor value; and optimizing and adjusting the model parameters of each risk calculation module and the risk assessment module based on the statistical analysis results. In this example implementation, each step in the entire approval process can be tagged using the business identifier of the business application request as a unique identifier, thereby associating the approval result with the assessment risk factor.Statistical analysis is performed on the approval results to calculate the consistency ratio (accuracy rate) and deviation ratio (error rate) between the model's predicted risks and the actual approval results. The impact of each risk factor on the model's predictions is analyzed, and risk factors with significant prediction deviations are identified. Based on feedback data and changes in corporate policies, decisions are made regarding whether to add or delete certain risk factors. If new factors are found to have a significant impact on risk assessment, they can be added to the model as new risk factors. If some risk factors have large prediction deviations and contribute little to the model, they can be considered for deletion. The weights of risk factors are adjusted based on feedback data to optimize the model's predictive performance. If a risk factor has a small prediction deviation and contributes significantly to the model, its weight can be appropriately increased; if a risk factor has a large prediction deviation and contributes little to the model, its weight can be appropriately decreased. New data can also be used to optimize and adjust the model to improve the reliability of the approval results. This invention achieves "efficient, high-quality, and highly adaptable" heterogeneous data processing and risk assessment through a full-process design of parallel data invocation, heterogeneous data fusion, principal component analysis dimensionality reduction, and adaptive integrated model evaluation. This supports various business scenarios that rely on multi-dimensional data, such as financial loans and internal corporate approvals. First, parallel processing improves data retrieval speed to meet real-time business requirements. Then, heterogeneous fusion processing and principal component analysis (PCA) optimize data quality and reduce data volume, improving processing efficiency. Finally, an integrated model trained on historical dimensionality-reduced data adapts to the characteristics of multi-source heterogeneous data, balancing accuracy and speed to meet the needs of large-scale automated business processing. This system can handle large-scale business processing in various scenarios, enabling data linkage and real-time interaction, ensuring the accuracy and reliability of approval results. Example 2, based on the same inventive concept, also provides a data processing system based on heterogeneous data fusion, comprising: a data acquisition module, used to respond to a business application request for a target object, initiate multiple data call requests in parallel, and acquire multidimensional data of the target object from heterogeneous data sources respectively, wherein the multidimensional data includes basic information, business data, work performance appraisal data, and salary data; wherein the heterogeneous data sources have different data modes and communication protocols; a data processing module, used to perform heterogeneous data fusion processing on the acquired multidimensional data to generate an information fusion table; map the information fusion table into a high-dimensional feature matrix; use principal component analysis to perform dimensionality reduction processing on the high-dimensional feature matrix to eliminate multicollinearity between features and extract a low-dimensional key sensitive feature set representing the core features of the target object; and an evaluation module, used to input the low-dimensional key sensitive feature set into a risk assessment model and output a risk score for the target object; wherein the risk assessment model is an ensemble learning model trained on a historical data feature set after dimensionality reduction by principal component analysis.In one possible implementation, the data acquisition module is specifically used to: respond to a business application request for a target object, initiate multiple data call requests in parallel through a distributed data access middleware; wherein, the distributed data access middleware includes a dynamic protocol adaptation module and a data format conversion engine, the dynamic protocol adaptation module adapts to different communication protocols of each heterogeneous data source by using a driver or functional component matched to each heterogeneous data source; the data format conversion engine is used to perform timestamp alignment and structure conversion on the acquired time-series data and non-time-series data with different patterns. In one possible implementation, the data processing module includes: a data fusion submodule, used to clean, standardize, and integrate the multidimensional data through distributed computing nodes to generate an information fusion table. In one possible implementation, the data fusion submodule is specifically used to: use the target field as the entity identifier, match the same entity object in different data sources using cosine similarity calculation, and perform data deduplication; if the same field in different data sources has inconsistent field content, use a weighted voting strategy to perform fusion calculation on the field content of the same field in different data sources, and use the fusion calculation result as the field value of that field in the information fusion table; wherein, the weighted voting strategy is determined based on the degree of correlation between the evaluation target and different data sources, and the degree of correlation is determined based on the Pearson correlation coefficient between the feature values ​​of different data sources and the risk indicators of the evaluation target. In one possible implementation, the data processing module includes a dimensionality reduction submodule, which is used to: standardize the data in the high-dimensional feature matrix to obtain a standardized matrix; perform feature decomposition on the covariance matrix of the standardized matrix, and filter out the low-dimensional key sensitive feature set of the target object by the cumulative variance contribution rate. In one possible implementation, a model building module is further included. This module is used to: filter out assessment risk factors based on the correlation between preset candidate risk factors and risk assessment results in historical assessment cases of different objects; construct a corresponding risk calculation module for each assessment risk factor based on the association between each assessment risk factor and the low-dimensional key sensitive feature set; train an ensemble learning model using historical assessment cases of different objects to obtain a risk assessment module; and obtain the risk assessment model based on each risk calculation module and the risk assessment module. In another possible implementation, the assessment module includes: a risk factor calculation submodule, used to input the low-dimensional key sensitive feature set into each risk calculation module to calculate risk factors and obtain corresponding assessment risk factor values; and an assessment submodule, used to input each assessment risk factor value into the risk assessment module to perform risk assessment and obtain a risk score for the target object; wherein a trigger mechanism is added at the business process entry point to automatically invoke the risk assessment model for risk assessment; and the risk assessment is based on the correlation coefficient between different assessment risk factors.In one possible implementation, the risk assessment module is a decision tree model; the assessment sub-module is specifically used to: for each node of the decision tree model, use the classification threshold of the node and the correlation coefficient between the node and the corresponding risk assessment factors of adjacent nodes to perform path selection on the input assessment risk factor values ​​until the leaf node of the decision tree model is reached, to obtain the risk score of the target object; wherein, each node corresponds to one assessment risk factor. In one possible implementation, it also includes: an approval module, used to generate an approval decision signal based on the risk score, and return the approval decision signal to the business application initiating terminal to complete the automated approval process. In one possible implementation, it also includes an optimization module, which is used to: associate the approval result of the business application request with the corresponding assessment risk factor value based on the business identifier of the business application request; perform statistical analysis on the deviation between the risk score and the corresponding approval result based on the assessment risk factor value, and optimize and adjust the model parameters of each risk calculation module and the risk assessment module based on the statistical analysis results. As shown in Embodiment 3, the present invention also provides an electronic device, which may be a computer device, a microcontroller device, a smart mobile device, etc. The electronic device in this embodiment may include a processor, a memory, a transceiver component, etc. The memory, processor, and transceiver components are connected via a bus. The memory can be used to store executable programs, and an exemplary executable program may include instructions. The processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be retrieved and / or modified when instructions are executed. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, and is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the storage medium to implement the corresponding method flow or corresponding function, so as to implement the steps of a data processing method based on heterogeneous data fusion in the above embodiments. Embodiment 4 Based on the same inventive concept, the present invention also provides a readable storage medium, specifically an electronic device readable storage medium (Memory), which is a memory device in an electronic device used to store programs and data. It is understood that the storage medium here can include both the built-in storage medium in electronic devices and the extended storage medium supported by electronic devices.The storage medium provides storage space, which stores the terminal's operating system. Furthermore, this storage space also contains one or more instructions suitable for loading and execution by a processor. These instructions can be one or more executable programs (including program code). It should be noted that the storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk drive. Loading and executing one or more instructions stored in the storage medium by the processor can implement the steps of a data processing method based on heterogeneous data fusion in the above embodiments. Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk drives, CD-ROMs, optical storage, etc.) containing computer-usable program code. The present invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowcharts and / or blocks. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in the flowcharts and / or blocks. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in the flowcharts and / or blocks.Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit its scope of protection. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that after reading the present invention, they can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the application, but these changes, modifications or equivalent substitutions are all within the scope of protection of the claims pending approval. < / h1>

Claims

1. A data processing method based on heterogeneous data fusion, characterized in that, Comprise: In response to a business application request for a target object, a plurality of data call requests are initiated in parallel to obtain multi-dimensional data of the target object from heterogeneous data sources, respectively, the multi-dimensional data comprising basic information, business data, work performance evaluation data and salary data; wherein the heterogeneous data sources have different data modes and communication protocols; The obtained multi-dimensional data is subjected to heterogeneous data fusion processing to generate an information fusion table; the information fusion table is mapped to a high-dimensional feature matrix; principal component analysis is used to reduce the dimension of the high-dimensional feature matrix to eliminate the multicollinearity between features and extract a low-dimensional key sensitive feature set representing the core features of the target object; The low-dimensional key sensitive feature set is input into a risk assessment model to output a risk score of the target object; wherein the risk assessment model is an ensemble learning model trained based on the historical data feature set after dimensionality reduction by principal component analysis.

2. The method of claim 1, wherein, In response to a business application request for a target object, a plurality of data call requests are initiated in parallel, comprising: In response to a business application request for a target object, a plurality of data call requests are initiated in parallel through a distributed data access middleware; Wherein the distributed data access middleware comprises a dynamic protocol adaptation module and a data format conversion engine, the dynamic protocol adaptation module uses a driver or functional component matched with each heterogeneous data source to adapt to the different communication protocols of each heterogeneous data source; the data format conversion engine is used to align timestamps and structure convert the obtained time series data and non-time series data with different modes.

3. The method of claim 1, wherein, The obtained multi-dimensional data is subjected to heterogeneous data fusion processing to generate an information fusion table, comprising: The multi-dimensional data is cleaned, standardized and associated and integrated by a distributed computing node to generate an information fusion table.

4. The method of claim 3, wherein, The associated integration comprises: Taking a target field as an entity identifier, the same entity object in different data sources is matched by cosine similarity calculation, and data deduplication is performed; If the field contents of the same field in different data sources are inconsistent, a weighted voting strategy is used to fuse and calculate the field contents of the same field in different data sources, and the fusion calculation result is taken as the field value of the field in the information fusion table; Wherein, the weighted voting strategy is determined based on the evaluation target and the association degree of different data sources, and the association degree is determined based on the Pearson correlation coefficient of the characteristic value of different data sources and the risk index of the evaluation target.

5. The method of claim 1, wherein, The high-dimensional feature matrix is subjected to dimensionality reduction processing by principal component analysis to eliminate the multicollinearity between features and extract a low-dimensional key sensitive feature set representing the core features of the target object, comprising: The data in the high-dimensional feature matrix is subjected to standardization processing to obtain a standardized matrix; Based on the covariance matrix of the standardized matrix, feature decomposition is performed to screen out a low-dimensional key sensitive feature set of the target object through cumulative variance contribution rate.

6. The method of claim 1, wherein, Before the low-dimensional key sensitive feature set is input into the risk assessment model to output the risk score of the target object, it further comprises: Screening each evaluation risk factor based on the correlation between each candidate risk factor and the risk evaluation result in the preset historical evaluation cases of different objects; Constructing a corresponding risk calculation module for each evaluation risk factor based on the association between each evaluation risk factor and the low-dimensional key sensitive feature set; Training an integrated learning model using the historical evaluation cases of different objects to obtain a risk evaluation module; Obtaining the risk evaluation model based on each risk calculation module and the risk evaluation module.

7. The method of claim 6, wherein, Inputting the low-dimensional key sensitive feature set into the risk evaluation model to output the risk score of the target object, including: Inputting the low-dimensional key sensitive feature set into each risk calculation module to calculate the corresponding evaluation risk factor value; Inputting each evaluation risk factor value into the risk evaluation module to perform risk evaluation and obtain the risk score of the target object; Wherein, the risk evaluation model is automatically called to perform risk evaluation by adding a trigger mechanism at the entrance of the business process; the risk evaluation is based on the correlation coefficient between different evaluation risk factors.

8. The method of claim 7, wherein, The risk evaluation module is a decision tree model; the risk evaluation module is inputted with each evaluation risk factor value to perform risk evaluation and obtain the risk score of the target object, including: For each node of the decision tree model, the classification threshold of the node and the correlation coefficient between the node and the adjacent node corresponding to the risk evaluation factor are used to select the path of the input evaluation risk factor value until the leaf node of the decision tree model is reached, and the risk score of the business application request is obtained; Wherein, each node corresponds to an evaluation risk factor.

9. The method of claim 1, wherein, After outputting the risk score of the target object, it further includes: Generating an approval decision signal based on the risk score and returning the approval decision signal to the business application initiation terminal to complete the automatic approval process.

10. The method of claim 9, wherein, After outputting the risk score of the target object, it further includes: Based on the business identifier of the business application request, the approval result of the business application request is associated with the corresponding evaluation risk factor value; Based on the statistical analysis of the deviation between the risk score based on the evaluation risk factor value and the corresponding approval result, the model parameters of each risk calculation module and the risk evaluation module are optimized and adjusted based on the statistical analysis result.

Images