Intelligent risk control method, device and equipment for multi-source data fusion and storage medium
Through high-dimensional tensor conversion and reconstruction technology, combined with multi-dimensional decomposition and deep neural network, the existing multi-source data fusion method is solved, and the problem of inefficient and insufficient accuracy is insufficient when processing high-dimensional and sparse large-scale data is achieved with high efficiency and high efficiency , accurate risk prediction and decision-making recommendation generation.
Patent Information
- Application Number
- CN202411554622.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2024-11-29
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing multi-source data fusion methods are inefficient and insufficient in processing large-scale data with high-dimensional, sparse, dynamically changing, and difficult to meet real-time requirements, and difficult to convert complex data analysis results into understandable and executable decisions. suggestion.
Using high-dimensional tensor transformation and reconstruction technology, combining multi-dimensional decomposition and self-organizing mapping, a risk propagation network is built, risk level prediction is carried out through cost-sensitive deep neural networks, and similarity matching and decision-making reasoning are carried out in combination with historical case databases. , generate digital risk control decision-making suggestions.
It improves data utilization and analysis dimensions, enhances the accuracy and real-timeness of data analysis, reduces false alarm rates and missed alarm rates, and provides interpretable intelligent decision-making suggestions to adapt to the needs of different application scenarios.
Smart Images

Figure CN119046647A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data intelligent risk control technology, and in particular to an intelligent risk control method, device, equipment and storage medium for multi-source data fusion. Background Art
[0002] In the era of big data, all industries are facing the challenge of processing massive, multi-source, heterogeneous data. Traditional data analysis methods are often limited to a single data source or simple data fusion technology, which makes it difficult to fully explore the complex correlation information contained in multi-source heterogeneous data. This limitation leads to inefficient use of data value and affects the accuracy and timeliness of decision-making.
[0003] Existing multi-source data fusion methods have problems such as low efficiency and insufficient precision when processing high-dimensional, sparse, and dynamically changing large-scale data. In particular, traditional methods are difficult to meet the needs of rapid response when facing application scenarios with high real-time requirements. On the other hand, the interpretability and operability of data analysis results are also important challenges currently faced. How to transform complex data analysis results into understandable and executable decision recommendations has become an urgent problem to be solved. Summary of the invention
[0004] The main purpose of the present invention is to provide an intelligent risk control method, device, equipment and storage medium for multi-source data fusion, which can make full use of big data and artificial intelligence technology to achieve all-round, multi-dimensional analysis and early warning, thereby improving data application capabilities and risk decision-making efficiency.
[0005] To achieve the above object, the present invention provides an intelligent risk control method for multi-source data fusion, comprising the following steps: Based on the big data platform, multi-source heterogeneous data to be processed are collected, and the multi-source heterogeneous data are preprocessed and converted into high-dimensional tensors to obtain target high-dimensional tensors; Reconstructing the target high-dimensional tensor to obtain a fused reconstructed tensor; Performing multi-dimensional decomposition and feature extraction on the fused reconstructed tensor to obtain a risk propagation network, and performing cluster analysis and feature mapping on the risk propagation network to obtain a risk feature mapping set; Inputting the risk feature mapping set into a cost-sensitive deep neural network integration model to predict the risk level, and obtaining a probability distribution of multi-category risk levels; The warning threshold is set according to the probability distribution of the multi-category risk levels, and similarity matching and decision reasoning are performed in combination with the historical case database to obtain digital risk control decision recommendations.
[0006] The present invention also provides an intelligent risk control device for multi-source data fusion, comprising: An acquisition module is used to acquire multi-source heterogeneous data to be processed based on a big data platform, and to perform preprocessing and high-dimensional tensor conversion on the multi-source heterogeneous data to obtain a target high-dimensional tensor; A reconstruction module, used for reconstructing the target high-dimensional tensor to obtain a fused reconstructed tensor; A mapping module, used for performing multi-dimensional decomposition and feature extraction on the fused reconstructed tensor to obtain a risk propagation network, and performing cluster analysis and feature mapping on the risk propagation network to obtain a risk feature mapping set; A prediction module, used for inputting the risk feature mapping set into a cost-sensitive deep neural network integrated model to predict the risk level and obtain a probability distribution of multi-category risk levels; The decision-making module is used to set the warning threshold according to the probability distribution of the multi-category risk levels, and perform similarity matching and decision reasoning in combination with the historical case database to obtain digital risk control decision recommendations.
[0007] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.
[0008] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.
[0009] In summary, in the technical solution provided by the present invention, through high-dimensional tensor conversion and reconstruction technology, efficient fusion of multi-source heterogeneous data is achieved, data utilization and analysis dimension are improved, and multi-dimensional decomposition and self-organizing mapping technology are adopted to accurately capture the complex relationships and potential patterns in the data, and to build a more sophisticated and comprehensive feature mapping network. Based on the cost-sensitive deep neural network integration model, dynamic prediction of multi-category problems is achieved, and the accuracy and real-time performance of the analysis results are improved. Through historical case similarity matching and multi-criteria decision analysis, interpretable intelligent decision suggestions are provided for complex problems, which can process high-dimensional, sparse large-scale data, and continuously optimize model performance through adaptive learning to meet the needs of different application scenarios. The use of tensor decomposition and parallel computing technology significantly improves the efficiency of large-scale data processing and analysis, and meets the needs of real-time analysis. Through multi-dimensional feature analysis and cost-sensitive learning, the false alarm rate and missed alarm rate are reduced, and the accuracy of anomaly detection and early warning is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] Figure 1 It is a schematic diagram of the steps of an intelligent risk control method for multi-source data fusion in one embodiment of the present invention; Figure 2It is a structural block diagram of an intelligent risk control device for multi-source data fusion in one embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.
[0011] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0012] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0013] Reference Figure 1 This embodiment provides an intelligent risk control method for multi-source data fusion, including the following steps: S1, based on the big data platform, collects multi-source heterogeneous data to be processed, and performs preprocessing and high-dimensional tensor conversion on the multi-source heterogeneous data to obtain the target high-dimensional tensor; In this embodiment, the multi-source heterogeneous data to be processed is collected based on the big data platform, and the multi-source heterogeneous data is cleaned to remove the noise, redundancy and incomplete parts in the multi-source heterogeneous data to obtain the cleaned data set. The numerical data in the cleaned data set is standardized, and the data of different scales are converted into unified standardized numerical data by scaling the data range to ensure the compatibility between different data types in subsequent model training and analysis. At the same time, the categorical data in the cleaned data set is vectorized, and the categorical variables are converted into numerical vector representations to obtain vectorized categorical data. The text data in the cleaned data set is segmented, and the continuous text is converted into separate words or phrases, and meaningless stop words are removed to optimize the quality of the text data. Feature extraction is performed on the text data after segmentation and stop word removal, and representative text features are extracted and converted into text feature vectors. Standardized numerical data, vectorized categorical data and text feature vectors are integrated to obtain a standard multi-source data set. The standard multi-source data set is reorganized based on a multidimensional array structure, and various types of data are organized according to different attribute dimensions to construct an initial high-dimensional tensor. In a high-dimensional tensor, each dimension represents a data attribute, such as time, location, category, etc. Tucker decomposition and CP decomposition are performed on the initial high-dimensional tensor to effectively reduce the dimension of the data and obtain a high-dimensional tensor after dimensionality reduction. Based on the tensor completion algorithm, the missing elements in the high-dimensional tensor after dimensionality reduction are estimated and filled to obtain the target high-dimensional tensor.
[0014] S2, reconstruct the target high-dimensional tensor to obtain a fused reconstructed tensor; Specifically, singular value decomposition is performed on the target high-dimensional tensor, and the original high-dimensional tensor is decomposed into a singular value set and corresponding left and right singular vectors. By decomposition, the most important feature information in the data is extracted. After decomposition, a first singular value set is obtained. The first singular value set is truncated based on a preset threshold function, some smaller singular values are discarded, and only those singular values with greater contributions are retained to obtain a second singular value set. The second singular value set is shrunk using the p-shrinkage operator, which is a common method for noise reduction and sparse processing. By shrinking, the influence of noise and redundant information can be reduced, and a third singular value set is obtained. Based on the third singular value set and the corresponding left and right singular vectors, the high-dimensional tensor is reconstructed to obtain an initial reconstructed tensor. The residual between the initial reconstructed tensor and the target high-dimensional tensor is calculated, and the residual is represented as an initial residual tensor. In order to reduce errors and improve reconstruction accuracy, the initial residual tensor is subjected to nuclear norm regularization. Nuclear norm regularization is an effective regularization method that can suppress overfitting and ensure the stability of the model during tensor reconstruction. After regularization, a regularized residual tensor is obtained. The regularized residual tensor is iteratively optimized based on the alternating direction multiplier method. During the optimization process, the optimal solution is gradually approached through multiple iterative calculations to obtain the optimized residual tensor. The optimized residual tensor is added to the initial reconstructed tensor to obtain the updated reconstructed tensor. It is judged whether the updated reconstructed tensor meets the preset convergence condition. The convergence condition is set to ensure that the optimization of the model reaches a stable state, that is, the residual is within an acceptable range. If the updated reconstructed tensor does not meet the convergence condition, the reconstructed tensor will be regarded as a new target high-dimensional tensor, and the previous singular value decomposition, truncation, p-shrinkage operator processing, residual calculation and regularization, alternating direction multiplier method optimization and other steps are repeated until the convergence condition is met. When the reconstructed tensor finally reaches convergence, a fused reconstructed tensor is obtained.
[0015] S3, performing multi-dimensional decomposition and feature extraction on the fused reconstructed tensor to obtain a risk propagation network, and performing cluster analysis and feature mapping on the risk propagation network to obtain a risk feature mapping set; It should be noted that the fused reconstructed tensor is decomposed into tensors to represent high-dimensional data as multiple factor matrices, which represent the internal structure of data in different dimensions. By analyzing the factor matrix, the eigenvector of each risk factor is extracted to form a risk factor feature set. The risk factor feature set is subjected to dimensionality reduction processing to simplify the high-dimensional data into a more operational risk feature matrix. The correlation coefficient matrix between risk factors is calculated based on the risk feature matrix after dimensionality reduction to reveal the mutual relationship between different risk factors, and the initial risk association matrix is obtained by calculation. The initial risk association matrix is subjected to threshold filtering and weak correlation removal to obtain the filtered risk association matrix. The risk factor network diagram is constructed based on the filtered risk association matrix to form the initial risk network. The network diagram represents different risk factors with nodes, and the edges represent the association between them. The initial risk network is topologically analyzed to identify the key nodes and edges, and the risk core set is obtained. The risk core set reflects the core factors that play an important role in the propagation and impact of system risks in the entire network. The influence intensity between nodes is calculated based on the core nodes to obtain the risk influence intensity matrix, which describes the degree of mutual influence of different risk factors in the network. A graph embedding analysis is performed on the risk impact intensity matrix. By mapping high-dimensional risk factors into low-dimensional space, the embedding vector of the risk factors is obtained. Low-dimensional embedding can simplify the network structure while retaining key risk propagation information. Based on the embedding vector, a weighted directed graph is constructed, in which each node represents a risk factor, and the edge weights between nodes represent the coupling strength between risk factors. Through the weighted directed graph, a comprehensive risk propagation network is obtained to describe the transmission and impact path of risks between different factors. Cluster analysis of the risk propagation network is performed based on the self-organizing map model. Through unsupervised learning methods, similar risk factors are clustered together to form different risk categories. At the same time, feature mapping is performed based on the clustering results to generate a risk feature mapping set containing different risk categories and features.
[0016] The risk propagation network is feature extracted to extract node attributes and edge weight information in the network, and these information are converted into feature vectors to form a risk network feature matrix. The matrix contains the specific features of each node in the network and the relationships between the nodes. The risk network feature matrix is input into the self-organizing map model for processing. The model maps the features of each risk node in an unsupervised learning manner, maps the high-dimensional data to a two-dimensional plane, and obtains the initial risk feature distribution. On this two-dimensional plane, the spatial position of each node reflects the similarity of their features in the risk propagation network. Based on the initial risk feature distribution, the distance relationship between adjacent nodes is analyzed, and the similarity between nodes is judged by calculating their distance on the plane. According to the distance relationship between nodes, the boundary of risk clustering is determined, and nodes close to each other are divided into the same category to obtain the division result of risk clustering. Through this process, risk clusters with similar features are identified, and the boundaries between different risk clusters are clarified. These risk clusters represent a collection of risk factors with similar propagation characteristics in the network. The coordinates of the center point of each cluster in the risk cluster division result are calculated. The center point can represent the center position of each cluster to form a risk cluster center set. Based on the risk cluster center set, the Euclidean distance between each risk cluster is calculated, which reflects the mutual relationship and similarity between different risk clusters. By calculating the Euclidean distance, the topological structure of the risk cluster is constructed, so as to better understand the spatial layout and mutual influence between different clusters. The high-order risk clusters are identified in the risk cluster topological structure, and a multi-level risk cluster system is obtained, which reflects the hierarchical relationship between risk clusters at different levels. The multi-level risk cluster system is calculated to obtain characteristic indicators such as intra-cluster variance, inter-cluster distance and silhouette coefficient of each level of risk cluster. The intra-cluster variance measures the consistency of node characteristics within the same cluster, the inter-cluster distance reflects the separation between different clusters, and the silhouette coefficient is used to evaluate the effect and rationality of clustering. These indicators together constitute the basis for evaluating the characteristics of risk clusters. Based on the risk cluster characteristic indicators, the importance of each risk cluster is ranked. By analyzing the influence of each cluster in the entire network, the most important core risk cluster set is identified. The core risk cluster usually plays a key role in the risk propagation network and plays an important role in the overall risk propagation and control. The core risk cluster set is combined with the topological structure of the entire risk propagation network to construct a risk feature mapping set.
[0017] S4, inputting the risk feature mapping set into the cost-sensitive deep neural network ensemble model to predict the risk level and obtain the probability distribution of multi-category risk levels; Specifically, the risk feature mapping set is subjected to feature combination and normalization, and features from different sources are standardized to ensure that different features are calculated at the same scale, thereby obtaining a standardized risk feature input vector. The standardized risk feature input vector is respectively passed into multiple sub-network models in the cost-sensitive deep neural network integration model for processing, and the sub-network models include a fully connected neural network, a convolutional neural network, and a recurrent neural network. Different types of neural network models are used to represent and process features from multiple angles. The fully connected neural network processes the input standardized features through a multi-layer perceptron, introduces nonlinearity using the ReLU activation function in each layer, and prevents overfitting through Dropout regularization to generate the first set of initial feature representations. The convolutional neural network extracts features through multi-layer convolution, pooling operations, and batch normalization, which can capture local patterns and global features of the data and generate the second set of initial feature representations. The recurrent neural network captures temporal dependencies through its unique network structure, enabling it to process data with time series attributes and generate the third set of initial feature representations. Each set of initial feature representations is weighted through the cost-sensitive layer of each sub-network model. The weighting process is carried out according to the preset cost matrix. Samples of different categories are assigned different weights according to the importance of their categories, so that the model can improve the recognition ability of high-cost samples when processing unbalanced samples. In this way, multiple sets of cost-weighted features are obtained. Each set of cost-weighted features is input into the deep neural network part of each sub-network model for processing. Through this process, the model can perform more in-depth feature learning on the cost-weighted features and generate multiple sets of deep feature representations. In order to improve the accuracy of prediction, the attention mechanism is applied to multiple sets of deep feature representations, and the importance weights of each sub-network model output are calculated to ensure that the model can focus on the most important feature combination and generate attention-weighted feature combinations, which can highlight those features that have the most influence in risk level prediction. The attention-weighted feature combination is input into the integration layer of the cost-sensitive deep neural network integration model for fusion. In the integration layer, the features of different sub-networks are integrated into a comprehensive feature representation. The comprehensive feature representation is passed to the cost-sensitive fully connected layer, where risk level mapping is performed and the model is optimized through the cost-sensitive loss function. The cost-sensitive loss function ensures that high-cost misclassifications can be effectively punished during the training process, thereby improving the model's ability to distinguish risk levels. The Softmax function is used to perform probability normalization on the integrated prediction results, converting the original results output by the model into a probability distribution of multi-category risk levels, and obtaining the probability value of each risk category.
[0018] S5, sets warning thresholds based on the probability distribution of multi-category risk levels, and combines historical case databases for similarity matching and decision reasoning to obtain digital risk control decision recommendations.
[0019] In this embodiment, the probability distribution of multi-category risk levels is analyzed, the ROC curve (receiver operating characteristic curve) is used to evaluate the classification performance of the model, and the optimal classification threshold is calculated based on the Youden index to obtain a set of risk warning threshold sets. The risk warning threshold set is compared with the current risk state to identify those high-risk categories that exceed the threshold, and thus generate a risk warning signal. Based on the risk warning signal, cases similar to the current risk state are retrieved from the historical case database. The similarity between the current risk state and the historical case is calculated by cosine similarity, and a set of historical cases that are most similar to the current situation is screened out to construct a similar case set. In order to improve the accuracy of the analysis, the similar case set is clustered, and the cases are grouped based on similarity to form case clustering results. In each cluster, the key core features and decision elements are extracted, and a preliminary decision rule set is constructed based on this. The decision rule set includes the historical decision experience most relevant to the current risk state. According to the extracted initial decision rule set, a decision tree model is constructed. Through this model, the current risk state is inferred to generate a set of candidate decision solutions. Each solution set is based on the decision-making patterns in historical cases and combines the current risk characteristics to provide multiple possible response strategies for the current problem. Perform multi-criteria decision analysis on the candidate decision solution set to obtain a decision solution scoring matrix to quantify the performance of each decision solution. Use the hierarchical analysis method to calculate the weight of each evaluation indicator, and combine the scoring matrix to comprehensively score each candidate solution to obtain the optimal decision solution. Combine the optimal decision solution with the risk warning signal to generate a comprehensive digital risk control decision recommendation. The recommendation includes a specific risk level, describes the reasons for triggering the warning, and the corresponding response measures.
[0020] In one example, multi-source heterogeneous data to be processed is collected based on a big data platform, and the multi-source heterogeneous data is preprocessed and converted into high-dimensional tensors to obtain a target high-dimensional tensor, including: collecting multi-source heterogeneous data to be processed based on a big data platform, and performing data cleaning on the multi-source heterogeneous data to obtain a cleaned data set; standardizing the numerical data in the cleaned data set to obtain standardized numerical data, and vectorizing the categorical data in the cleaned data set to obtain vectorized categorical data; segmenting the text data in the cleaned data set and removing stop words Word processing is performed to obtain preprocessed text data, and feature extraction is performed on the preprocessed text data to obtain text feature vectors; standardized numerical data, vectorized classification data and text feature vectors are integrated to obtain a standard multi-source data set; based on a multidimensional array structure, the standard multi-source data set is reorganized into a high-dimensional tensor, in which each dimension represents a data attribute, to obtain an initial high-dimensional tensor; Tucker decomposition and CP decomposition are performed on the initial high-dimensional tensor to obtain a reduced-dimensional high-dimensional tensor, and the missing elements in the reduced-dimensional high-dimensional tensor are estimated and filled based on a tensor completion algorithm to obtain a target high-dimensional tensor.
[0021] In this example, multi-source heterogeneous data is collected from a big data platform. These data come from different sources, with different formats, types, and structures, and may include numerical data, categorical data, and text data. Data cleaning is performed on the multi-source heterogeneous data to process missing values, noise data, redundant information, etc., to obtain a cleaned data set. The numerical data in the cleaned data set is standardized to map data of different dimensions to the same range. Usually, the values are adjusted to zero mean and unit variance to obtain standardized numerical data. The cleaned categorical data (such as the user's gender, occupation, region, etc.) is vectorized. A common method for vectorization is "one-hot encoding". Through vectorization, categorical data is converted into a numerical representation that can be understood by the machine learning model. Text data (such as user comments, transaction descriptions, etc.) is segmented to break long texts into meaningful words or phrases. After the segmentation is completed, stop words are removed, that is, words that have no practical meaning to the text content are deleted. After segmentation and stop word removal, the preprocessed text is obtained. Perform feature extraction on the preprocessed text data, and use TF-IDF (term frequency-inverse document frequency) or word vectors (such as Word2Vec) to convert the text data into vector form to obtain a text feature vector that can represent the text content. Integrate the standardized numerical data, vectorized classification data, and text feature vectors to obtain a standard multi-source data set. Use a multi-dimensional array structure to reorganize the standard multi-source data set into a high-dimensional tensor. In this process, each data attribute is regarded as a dimension. For example, assuming there are four attributes of the user's age, income, region, and comment text, then this multi-source data set can be reorganized into a four-dimensional tensor, with each dimension corresponding to these attributes. The representation of a high-dimensional tensor is: ; in, represents the initial high-dimensional tensor, are the size of each dimension, representing the value range of different data attributes. The constructed tensor can fully express the multidimensional relationship of multi-source data. The initial high-dimensional tensor is subjected to Tucker decomposition and CP decomposition to obtain a reduced-dimensional high-dimensional tensor. Tucker decomposition is to decompose a high-dimensional tensor into a core tensor and multiple factor matrices. The core tensor represents compact multi-dimensional features, and the factor matrix represents the feature distribution in each dimension; while CP decomposition represents a high-dimensional tensor as the sum of multiple rank tensors. Through these dimensionality reduction methods, high-dimensional tensors are compressed into low-dimensional forms to reduce computational complexity. The missing elements in the reduced-dimensional high-dimensional tensor are estimated and filled based on the tensor completion algorithm. The tensor completion algorithm analyzes the structure and pattern of the existing data, infers the value of the missing part, and generates a complete reduced-dimensional high-dimensional tensor. The target high-dimensional tensor obtained through this process can accurately represent the core information of multi-source data while retaining the structural relationship of the data.
[0022] Among them, the initial high-dimensional tensor is subjected to Tucker decomposition and CP decomposition to obtain a reduced-dimensional high-dimensional tensor, and the missing elements in the reduced-dimensional high-dimensional tensor are estimated and filled based on the tensor completion algorithm to obtain the target high-dimensional tensor, which specifically includes: applying a high-order singular value decomposition algorithm to the initial high-dimensional tensor to obtain the core tensor and target factor matrix of the tensor; calculating the rank of the tensor based on the core tensor and the target factor matrix, and setting a truncation threshold to obtain the dimensionality reduction parameters; truncating the core tensor according to the dimensionality reduction parameters to retain the main feature information to obtain a compressed core tensor; using the compressed core tensor and the original target factor matrix to reconstruct the tensor to obtain the reduced-dimensional tensor after Tucker decomposition; applying alternating least squares to the reduced-dimensional tensor after Tucker decomposition. Multiplication, iteratively calculate the factor vector of each dimension to obtain the factor matrix set of CP decomposition; calculate the core tensor based on the factor matrix set of CP decomposition, and compare it with the original tensor to obtain the reconstruction error; set the iterative stopping condition according to the reconstruction error, if the stopping condition is not met, return to the alternating least squares step until the final CP decomposition result is obtained; weighted fusion of the results of Tucker decomposition and CP decomposition to obtain a comprehensive reduced-dimensional high-dimensional tensor; apply the tensor completion algorithm based on low-rank constraints to the comprehensive reduced-dimensional high-dimensional tensor, construct the tensor nuclear norm minimization optimization problem, and obtain the completion optimization model; use the alternating direction multiplier method to solve the completion optimization model, iteratively update the tensor element values until convergence, and obtain the complete target high-dimensional tensor.
[0023] Specifically, a high-order singular value decomposition algorithm is applied to the initial high-dimensional tensor to decompose the tensor into multiple factor matrices and a core tensor. Assume that the initial high-dimensional tensor is , where each Represents the tensor in The high-order singular value decomposition algorithm performs singular value decomposition on the pattern matrix of each dimension to obtain the corresponding factor matrix and form a core tensor The high-order singular value decomposition process can be expressed as: ; in, is the core tensor, It is dimensional factor matrix, Represents the tensor in Through high-order singular value decomposition, we get the core tensor and multiple target factor matrices , these factor matrices preserve the main features of the tensor in each dimension. Based on the core tensor and the target factor matrix Calculate the rank of the tensor. The rank of the tensor is represented here as the number of singular values in each dimension, reflecting the data complexity of the tensor in all directions. In order to reduce the computational complexity and retain the main information in the data, a truncation threshold is set, and the core tensor is truncated based on the threshold. The truncation process actually retains the main singular value information in the core tensor and ignores smaller singular values. The truncated core tensor can be regarded as a compressed tensor in which only the main feature information is retained. The compressed core tensor and the original target factor matrix are used to reconstruct the tensor to obtain the reduced dimensionality tensor after Tucker decomposition. In this way, the main features of the initial high-dimensional tensor are retained while reducing the dimensionality and complexity of the data. Perform CP decomposition on the reduced dimensionality tensor after Tucker decomposition. CP decomposition is a decomposition method that represents a tensor as the sum of multiple rank tensors. Its representation is: ; in, is a scalar, It is The factor vector in dimension, represents the outer product operation, is the tensor rank. In order to solve the CP decomposition, the alternating least squares method is applied for iterative calculation, and the factor matrix on each dimension is optimized respectively until the factor vectors of each dimension converge, and the factor matrix set of CP decomposition is obtained. After completing the CP decomposition, the core tensor is calculated based on the factor matrix set. This core tensor is compared with the original tensor to calculate the reconstruction error. The calculation formula of the reconstruction error is:
[0024] Where e represents the reconstruction error, is the original tensor, is the reconstructed tensor, Represents the Frobenius norm, which measures the difference between two tensors. By analyzing the reconstruction error, the iteration stop condition is set. If the reconstruction error does not reach the preset threshold, the alternating least squares method step is returned to continue iteratively optimizing the factor matrix until the reconstruction error is small enough. When the CP decomposition achieves the ideal result, the results of Tucker decomposition and CP decomposition are weightedly fused to obtain a comprehensive reduced-dimensional high-dimensional tensor. A tensor completion algorithm based on low-rank constraints is applied to the comprehensive reduced-dimensional high-dimensional tensor. The goal of tensor completion is to fill in the missing data that may exist in the dimensionality reduction process. This goal is achieved by constructing a tensor nuclear norm minimization optimization problem. Minimization of the nuclear norm helps to find the optimal solution for the low-rank tensor. The form of the optimization problem can be expressed as: ; in, is the nuclear norm of the tensor, represents the set of indices of observations, is the observed tensor element. The solution to this optimization problem is the optimal estimate of the low-rank tensor. In order to solve this optimization problem, the alternating direction method of multipliers (ADMM) is used to solve it. ADMM is an efficient optimization algorithm that can effectively handle large tensor completion problems by decomposing the original problem into multiple sub-problems and solving them alternately. In the ADMM process, the element values of the tensor are iteratively updated until the optimization problem converges. A complete target high-dimensional tensor is obtained, and all missing data elements are completed while maintaining the low-rank characteristics and main structure of the tensor.
[0025] In one example, a target high-dimensional tensor is reconstructed to obtain a fused reconstructed tensor, including: A1: performing singular value decomposition on the target high-dimensional tensor to obtain a first singular value set and corresponding left and right singular vectors, and truncating the first singular value set based on a preset threshold function to obtain a second singular value set; A2: shrinking the second singular value set using a p-shrinkage operator to obtain a third singular value set, and reconstructing a tensor based on the third singular value set and the corresponding left and right singular vectors to obtain an initial reconstructed tensor; A3: calculating the initial reconstructed tensor and the target A4: Iteratively optimize the regularized residual tensor based on the alternating direction multiplier method to obtain the optimized residual tensor, and add the optimized residual tensor to the initial reconstruction tensor to obtain the updated reconstruction tensor; A5: Determine whether the updated reconstruction tensor meets the convergence condition. If not, use the updated reconstruction tensor as the new target high-dimensional tensor, and repeat the above steps A1-A4 until the convergence condition is met to obtain the fused reconstruction tensor.
[0026] In this example, the target high-dimensional tensor is subjected to singular value decomposition to obtain its first singular value set. And the corresponding left and right singular vector sets and . Decomposing the structural features of the tensor into different components simplifies the complexity of the data and extracts the most important features of the tensor. After decomposition, the resulting singular value set usually contains a large number of small values. These small singular values represent noise or unimportant information in the data. Based on the preset threshold function , truncate the first singular value set, retain the singular values greater than the threshold, ignore the part less than the threshold, and obtain the truncated second singular value set The truncation operation effectively reduces the redundancy of data and retains the main feature information by reducing unimportant singular values. Threshold function The choice of can be determined based on the application scenario and experience. For example, the energy retention rate can be selected as the basis to ensure that a certain proportion of the characteristic energy is retained. In order to improve the robustness and noise resistance of the model, the p-shrinkage operator is used to reduce the second singular value set Perform shrinkage. p-shrinkage is a common singular value processing method that shrinks all singular values according to a certain ratio to reduce the impact of noise. The form of p-shrinkage can be expressed as: ; in, is the third singular value set after shrinkage, is the shrinkage parameter, which is used to control the degree of shrinkage. This step reduces the impact of small fluctuations on reconstruction by smoothing the changes in singular values, thus ensuring the robustness of the reconstructed tensor. And the corresponding set of left and right singular vectors and , reconstruct the initial reconstructed tensor The initial reconstructed tensor is obtained by recombining the truncated and shrunk singular values with the singular vectors, representing an approximate reconstruction of the original target high-dimensional tensor. Calculate the initial reconstructed tensor With the target high-dimensional tensor The calculation formula of the residual between is: ; in, is the initial residual tensor, representing the difference between the original tensor and the reconstructed tensor. Perform nuclear norm regularization. Nuclear norm regularization is an effective denoising and compression method that can reduce potential overfitting problems in the reconstruction process by penalizing the nuclear norm of the residual. The nuclear norm is defined as the sum of singular values, that is: ; in, is the singular value of the residual tensor. By minimizing the nuclear norm of the residual, the noise in the data is suppressed and a more accurate reconstruction result is obtained. The residual tensor after regularization is the regularized residual tensor The regularized residual tensor is iteratively optimized based on the alternating direction multiplier method. The alternating direction multiplier method is an efficient optimization algorithm that decomposes a complex optimization problem into multiple sub-problems and achieves the goal of global optimization by alternately solving each sub-problem. In this process, the residual tensor is optimized and then the optimized residual tensor is compared with the initial reconstruction tensor. Add together to get the updated reconstructed tensor The updated reconstructed tensor retains the main information of the initial reconstructed tensor and corrects the errors in the reconstruction through optimization. Whether the convergence condition is met. Usually, the convergence condition can be set according to the size of the reconstruction error, for example, a permissible error threshold can be set. , when the updated reconstruction error is less than the threshold, the algorithm is considered to have converged. If the updated reconstruction tensor does not meet the convergence condition, it is used as the new target high-dimensional tensor and returns to step A1 to continue singular value decomposition, truncation, contraction and iterative optimization until the preset convergence condition is met. After multiple iterative optimizations, the final fused reconstruction tensor is It is the result of multiple updates.
[0027] The method comprises: applying the p-shrinkage operator to shrink the second singular value set to obtain a third singular value set, and reconstructing the tensor based on the third singular value set and the corresponding left and right singular vectors to obtain an initial reconstructed tensor, including: applying the p-shrinkage operator to each singular value in the second singular value set to obtain a shrunken singular value; performing a nonlinear transformation on the shrunken singular value based on a preset p value and a threshold parameter to obtain a transformed singular value; normalizing the transformed singular value to obtain a normalized singular value; performing a matrix multiplication operation on the normalized singular value and the corresponding left and right singular vectors to obtain a preliminary reconstructed matrix; and performing a matrix multiplication operation on the normalized singular value and the corresponding left and right singular vectors to obtain a preliminary reconstructed matrix. The preliminary reconstruction matrix applies the inverse tensor expansion operation to obtain the reconstructed tensor; the Frobenius norm between the reconstructed tensor and the target high-dimensional tensor is calculated to obtain the reconstruction error; the parameters of the p-shrinkage operator are adaptively adjusted based on the reconstruction error to obtain the updated p-shrinkage operator parameters; the above steps are repeated using the updated p-shrinkage operator parameters to obtain the optimized reconstructed tensor; a low-rank constraint is applied to the optimized reconstructed tensor, and it is optimized through an iterative soft threshold algorithm to obtain a low-rank reconstructed tensor; the low-rank reconstructed tensor is combined with the original left and right singular vectors to obtain the initial reconstructed tensor.
[0028] Specifically, the p-shrinkage operator is applied to each singular value in the second singular value set. The p-shrinkage operator is an effective tool for reducing noise and improving robustness. By shrinking each singular value, important features are retained and the influence of noise on the reconstruction process is weakened. The basic formula of p-shrinkage is: ; in, Indicates singular values, represents the singular value after shrinkage, is the shrinkage parameter, which determines the strength of the shrinkage. For each singular value, if it is less than the preset p value, the singular value will be compressed to zero, which is equivalent to ignoring those unimportant components. Through the shrinkage process, the shrunken singular value set is obtained , these singular values are smoother than the original singular values. Based on the preset p-value and threshold parameters, the shrunken singular values are nonlinearly transformed. By applying a specific nonlinear function, the distribution of singular values is adjusted. Common nonlinear transformations can include exponential decay or logarithmic functions to enhance the expressiveness of singular values. For example, the following forms of nonlinear transformations can be applied: ; in, are the transformed singular values, is a parameter that controls the strength of the nonlinear transformation. This transformation can reduce the influence of smaller singular values while retaining the main information of the singular values, making the reconstruction process more effective. Normalization is performed to map all singular values to a uniform scale to prevent instability caused by large differences in different singular values during the reconstruction process. The normalization formula is: ; in, is the normalized singular value. Perform matrix multiplication on the normalized singular value and the corresponding left and right singular vectors to obtain the initial reconstruction matrix. Assume that the left and right singular vectors are and , the calculation of the preliminary reconstruction matrix can be expressed as: ; in, is the initial reconstruction matrix, is the normalized singular value diagonal matrix, and are the transpose of the left and right singular vector matrices respectively. The preliminary reconstruction matrix reflects the characteristics of the target high-dimensional tensor in the main direction. Apply the inverse tensor expansion operation to convert the preliminary reconstruction matrix back to a high-dimensional tensor. The tensor expansion operation refers to expanding the multi-dimensional tensor into a matrix form, while the inverse tensor expansion is to remap the matrix back to the form of a multi-dimensional tensor. Through the inverse tensor expansion, the preliminary reconstruction matrix is converted to the reconstruction tensor , the reconstructed tensor is an approximate reconstruction of the target high-dimensional tensor. In order to evaluate the effect of the reconstruction, the reconstructed tensor is calculated With the target high-dimensional tensor The Frobenius norm between them. The Frobenius norm is used to measure the difference between two tensors, and its formula is: ; in, and Represent the values of the target tensor and the reconstructed tensor on each element respectively. The smaller the value of the Frobenius norm, the smaller the difference between the reconstructed tensor and the target tensor. By calculating the Frobenius norm, the reconstruction error is obtained to judge the quality of the current reconstruction. Based on the reconstruction error, the parameters of the p-shrinkage operator are adaptively adjusted. The purpose of adaptive adjustment is to dynamically adjust the p value according to the error during the reconstruction process, so that the shrinkage process is more flexible. For example, if the current reconstruction error is large, the p value can be appropriately reduced to retain more singular value information; if the reconstruction error is small, the p value can be increased to further compress the noise part. The adjusted p-shrinkage parameters are used for the next round of reconstruction. Using the updated p-shrinkage operator parameters, repeat the above steps, perform singular value shrinkage, nonlinear transformation, normalization, matrix multiplication and tensor expansion operations again, until the reconstruction error reaches the preset convergence condition, and obtain the optimized reconstructed tensor. Apply low-rank constraint processing to the optimized reconstructed tensor. Low-rank constraint helps to further reduce the redundant information of the tensor, so that the rank of the tensor remains small and ensures the structure of the data. In this process, the low-rank reconstruction tensor is optimized by iterative soft thresholding algorithm. The role of the soft thresholding algorithm is to perform sparse processing on the singular values so that small singular values are gradually shrunk to zero, thereby retaining the main features. The low-rank reconstruction tensor is combined with the original left and right singular vectors U and V to obtain the final initial reconstruction tensor.
[0029] In one example, a fused reconstructed tensor is subjected to multi-dimensional decomposition and feature extraction to obtain a risk propagation network, and the risk propagation network is subjected to cluster analysis and feature mapping to obtain a risk feature mapping set, including: performing tensor decomposition on the fused reconstructed tensor to obtain multiple factor matrices, and extracting feature vectors of each risk factor based on the multiple factor matrices to obtain a risk factor feature set; performing dimensionality reduction processing on the risk factor feature set to obtain a risk feature matrix after dimensionality reduction, and calculating a correlation coefficient matrix between risk factors based on the risk feature matrix after dimensionality reduction to obtain an initial risk association matrix; performing threshold filtering and weak correlation removal on the initial risk association matrix to obtain a filtered risk association matrix. matrix, and build a risk factor network diagram based on the filtered risk association matrix to obtain the initial risk network; perform topological analysis on the initial risk network to identify key nodes and edges, obtain the risk core set, and calculate the impact intensity between nodes based on the risk core set to obtain the risk impact intensity matrix; perform graph embedding analysis on the risk impact intensity matrix, map the risk factors to a low-dimensional space, obtain the risk factor embedding vector, and build a weighted directed graph based on the risk factor embedding vector, in which the nodes represent the risk factors and the edge weights represent the coupling intensity, to obtain the risk propagation network; perform clustering analysis and feature mapping on the risk propagation network based on the self-organizing mapping model to obtain a risk feature mapping set.
[0030] In this example, the fused reconstructed tensor is decomposed to extract feature information in different dimensions. , where each Represents the tensor in To handle this high-dimensional structure, common tensor decomposition methods such as Tucker decomposition or CP decomposition are used. Using Tucker decomposition, the tensor is decomposed into a core tensor And multiple factor matrices: ; in, is the core tensor, is a factor matrix for each dimension. The main features in different dimensions are captured, and the column vector in each factor matrix is the feature vector of the data in the corresponding dimension. Through this process, the feature vector of each risk factor in the fused reconstruction tensor is extracted to form a risk factor feature set. The risk factor feature set is subjected to dimensionality reduction. Common dimensionality reduction methods include PCA (principal component analysis), which maximizes the variance of the data by finding a set of orthogonal principal components. Through PCA dimensionality reduction, the high-dimensional feature vector is reduced to a low-dimensional feature matrix, which can be expressed as: ; in, is the risk factor feature set, is the reduced-dimensional projection matrix, is the risk feature matrix after dimensionality reduction. The feature matrix after dimensionality reduction retains the main risk factor information while greatly reducing the impact of redundant data. Based on the risk feature matrix after dimensionality reduction, the correlation coefficient matrix between risk factors is calculated. The correlation coefficient matrix is used to measure the linear correlation between different risk factors, and its calculation formula is: ; in, There are two variables and The covariance of and are their standard deviations respectively. Through this process, the initial risk association matrix is obtained, which reflects the strength of the relationship between risk factors. In order to simplify the analysis, the initial risk association matrix is threshold filtered and weak correlation is removed. A threshold is set. If the correlation coefficient between two risk factors is lower than the threshold, they are considered irrelevant or weakly correlated and are removed from the association matrix. The filtered risk association matrix only retains strongly correlated risk factors, effectively reducing the complexity of the analysis. Based on the filtered risk association matrix, a risk factor network diagram is constructed. In this network diagram, each node represents a risk factor, and the edges between nodes represent the strong correlation between risk factors. The network diagram can be represented in the form of a graph as ,in is a collection of risk factor nodes, is the edge between risk factors. Through this network diagram, we can preliminarily observe the interconnection relationship between risk factors and form an initial risk network. After constructing the initial risk network, we can perform a topological analysis on the network. Topological analysis helps to identify key nodes and edges in the network, which often play an important role in the risk propagation of the entire network. Key nodes can be identified through common graph theory indicators such as degree centrality and betweenness centrality. Degree centrality measures the number of connections of a node, while betweenness centrality measures the control power of a node in the network. Through topological analysis, we get a set of risk core nodes, which are the most important risk factors in risk propagation. Based on the risk core set, we calculate the influence strength between nodes in the network. The influence strength matrix quantifies the propagation effect between risk factors. Nodes with greater influence strength mean more decisiveness in the risk propagation process. The calculation of the influence strength matrix can be based on the edge weights between nodes: ; in, It is a slave node To Node The edge weight represents the coupling strength between the two. Graph embedding analysis is performed on the risk impact intensity matrix to map risk factors from high-dimensional space to low-dimensional space. Graph embedding technology, such as the node embedding algorithm (Node2Vec), is used to represent the nodes of the high-dimensional network graph as low-dimensional vectors by retaining the local and global structure of the network. The resulting risk factor embedding vector is expressed as: ; in, Is a node The low-dimensional embedding vector of the risk factors is obtained by this embedding analysis. Through this embedding analysis, the risk factors are represented in the low-dimensional space while retaining their relative relationships in the high-dimensional network. Based on the risk factor embedding vectors, a weighted directed graph is constructed. The nodes represent the risk factors, while the edge weights represent the coupling strength between the risk factors. The construction of the weighted directed graph helps to better understand the direction and intensity of risk propagation and obtain a complete risk propagation network. The network is clustered and feature mapped using the self-organizing map model (SOM). SOM is an unsupervised learning model that can map high-dimensional data to two-dimensional space, thereby achieving data visualization and clustering. By performing SOM clustering analysis on the nodes in the risk propagation network, risk clusters with similar characteristics are identified. These clusters represent risk factors that are associated in the risk propagation process. Through the clustering results of the self-organizing map model, a set of risk feature mappings is obtained.
[0031] In one example, cluster analysis and feature mapping are performed on the risk propagation network based on the self-organizing map model to obtain a risk feature mapping set, including: extracting features from the risk propagation network, converting node attributes and edge weights into feature vectors, and obtaining a risk network feature matrix; inputting the risk network feature matrix into the self-organizing map model, performing feature mapping on each risk node, and obtaining an initial risk feature distribution on a two-dimensional plane; analyzing the distance relationship between adjacent nodes based on the initial risk feature distribution to obtain a risk clustering boundary, and clustering the mapping results according to the risk clustering boundary to identify risk clusters with similar features and obtain risk clusters. Cluster division results; calculate the coordinates of the center point of each cluster in the risk cluster division results to obtain the risk cluster center set, and calculate the Euclidean distance between each risk cluster based on the risk cluster center set to construct a risk cluster topological structure; identify high-order risk clusters in the risk cluster topological structure to obtain a multi-level risk cluster system, and calculate the intra-cluster variance, inter-cluster distance and silhouette coefficient of risk clusters at each level in the multi-level risk cluster system to obtain risk cluster characteristic indicators; rank the importance of each risk cluster based on the risk cluster characteristic indicators to obtain a core risk cluster set, and combine the core risk cluster set with the topological structure of the risk propagation network to construct a risk characteristic mapping set.
[0032] In this example, feature extraction is performed on node attributes and edge weights in the risk propagation network. Assume that the risk propagation network can be represented as a directed weighted graph ,in represents a set of nodes, representing different risk factors, represents the edge set, which represents the correlation or mutual influence between risk factors. The weight of the edge usually reflects the risk transmission intensity between two nodes. For each node, extract its attributes (such as risk characteristics, connectivity, etc.) and edge weight, convert this information into a feature vector, and obtain a risk network feature matrix. Assume that nodes, the risk network characteristic matrix can be expressed as: ; in, Is a node The feature vector of contains the attributes of the nodes and the weight information of their edges. Input the self-organizing map model (SOM) to perform feature mapping on each risk node. The self-organizing map model is an unsupervised neural network that maps high-dimensional data to a low-dimensional space, usually a two-dimensional plane, by maintaining the topological structure of the data. After SOM processing, each risk node is mapped to a two-dimensional plane to form an initial risk feature distribution. Based on the initial risk feature distribution, the distance relationship between adjacent nodes is analyzed to obtain the risk clustering boundary. The SOM model can map similar nodes to similar positions by preserving topology. The similarity between nodes is determined based on the Euclidean distance between nodes. The calculation formula is: ; in, Is a node and nodes The Euclidean distance in feature space, and The nodes are and nodes By setting a distance threshold, if the distance between nodes is less than the threshold, they are classified into the same cluster. Based on this process, the boundaries of the risk cluster are identified, and the mapping results are clustered to obtain multiple risk clusters with similar characteristics. Each risk cluster represents a group of nodes with similar characteristics in the risk propagation network. The coordinates of the center point of each risk cluster are calculated to obtain the risk cluster center set. The center point of the risk cluster is the average position of all nodes in the cluster on the two-dimensional plane, and the calculation formula is: ; in, Risk Cluster The center point of is the number of nodes in the cluster, are the coordinates of the nodes in the cluster. Through this process, the center set of all risk clusters is obtained. Based on the center set of risk clusters, the Euclidean distance between each risk cluster is calculated to construct the risk cluster topology. The Euclidean distance measures the similarity between different risk clusters. The closer the distance, the more similar they are in the feature space. A risk cluster topology graph is constructed based on the distance relationship. Each edge in the graph represents the association between risk clusters, and the weight of the edge is determined by the Euclidean distance between clusters. The risk cluster topology structure is used to identify high-order risk clusters and obtain a multi-level risk cluster system. The multi-level risk cluster system is achieved by further dividing clusters into sub-clusters or aggregating similar clusters into large clusters. Such a system is constructed using a hierarchical clustering algorithm (such as agglomerative hierarchical clustering). Each level represents a risk cluster group of different levels. The bottom-level cluster contains nodes with very similar characteristics, and the top-level contains clusters with more extensive characteristics. In order to evaluate the quality of each cluster, the intra-cluster variance, inter-cluster distance and silhouette coefficient of each level of clusters in the multi-level risk cluster system are calculated. The intra-cluster variance is used to measure the compactness of the cluster. The smaller the intra-cluster variance, the more similar the nodes in the cluster are. The inter-cluster distance reflects the separation between different clusters. The larger the distance, the higher the distinction between clusters. The silhouette coefficient combines the intra-cluster compactness and inter-cluster separation to evaluate the overall effect of clustering. The formula is: ; in, Is a node The silhouette coefficient, Is a node The average distance within the cluster, Is a node The minimum average distance with other clusters. The value of the silhouette coefficient ranges from -1 to 1, and the closer it is to 1, the better the clustering effect. Based on the risk cluster characteristic indicators (intra-cluster variance, inter-cluster distance and silhouette coefficient), the importance of each risk cluster is ranked. Risk clusters with higher importance tend to have higher compactness (low variance) and larger inter-cluster distance (high discrimination). These risk clusters play a core role in the network, and finally a set of core risk clusters is obtained. The core risk cluster set is combined with the topological structure of the risk propagation network to construct a risk feature mapping set. By combining the center point of the cluster, the inter-cluster association and the topological structure, a complete risk propagation map is formed. The core risk cluster set can help identify risk factors that play an important role in risk propagation in the entire network, and thus provide a decision-making basis for risk management.
[0033] In one example, a risk feature mapping set is input into a cost-sensitive deep neural network integration model to predict the risk level and obtain a probability distribution of multi-category risk levels, including: combining and normalizing the risk feature mapping set to obtain a standardized risk feature input vector; respectively passing the standardized risk feature input vector into multiple sub-network models based on the cost-sensitive deep neural network integration model for feature processing to obtain multiple groups of initial feature representations; the multiple sub-network models include a fully connected neural network, a convolutional neural network, and a recurrent neural network; the fully connected neural network is processed by a multi-layer perceptron, and each layer uses a ReLU activation function and Dropout regularization to obtain a first group of initial feature representations; the convolutional neural network extracts local and global features through multi-layer convolution, pooling, and batch normalization operations to obtain a second group of initial feature representations; the recurrent neural network obtains a third group of initial feature representations by capturing temporal dependencies; The initial feature representation of each group is weighted by the cost-sensitive layer of each sub-network model, and different weights are assigned to samples of different categories according to the preset cost matrix to obtain multiple groups of cost-weighted features; each group of cost-weighted features is input into the corresponding deep neural network of each sub-network model for processing to obtain multiple groups of deep feature representations; the attention mechanism is analyzed on the multiple groups of deep feature representations, and the importance weight of each sub-network model output is calculated to obtain the attention-weighted feature combination; the attention-weighted feature combination is input into the integration layer of the cost-sensitive deep neural network integration model for fusion to obtain the comprehensive feature representation, and the comprehensive feature representation is mapped to the risk level through the cost-sensitive fully connected layer based on the cost-sensitive deep neural network integration model, and optimized using the cost-sensitive loss function to obtain the integrated prediction result; the integrated prediction result is probability normalized by the Softmax function to obtain the probability distribution of multi-category risk levels.
[0034] In this example, the risk feature mapping set is feature combined and normalized. The purpose of normalization is to map the data range of different features to the same scale to avoid unstable model training due to different feature dimensions. Through normalization, a standardized risk feature input vector is obtained to ensure that different features have similar numerical ranges, which is convenient for the neural network model to perform feature processing. The standardized risk feature input vector is passed into multiple sub-network models based on the cost-sensitive deep neural network integration model for feature processing. The integrated model consists of multiple sub-networks, including fully connected neural networks, convolutional neural networks, and recurrent neural networks. Different sub-network models are good at processing different types of features and provide multi-level feature representations. The fully connected neural network processes the standardized feature input vector through a multi-layer perceptron. Each layer uses the ReLU activation function, which is a common nonlinear activation function used to introduce nonlinearity so that the model can learn complex feature relationships. The definition of ReLU is: ; In addition, each layer of the multi-layer perceptron also uses Dropout regularization to prevent overfitting. Dropout regularization reduces the model's dependence on specific features and improves the model's generalization ability by randomly discarding a portion of neurons. Through this process, the first set of initial feature representations is obtained. Convolutional neural networks extract local features from input data through multi-layer convolution operations. The convolution operation can capture local patterns in the data by sliding windows. The convolution formula is: ; in, is the local region of the input data, is the convolution kernel. Pooling operations (such as maximum pooling) can reduce the size of the feature map, reduce the amount of calculation, and retain important features. The convolutional neural network normalizes the output of the convolution layer through batch normalization to improve the training speed and stability of the model. After convolution, pooling, and batch normalization, the second set of initial feature representations is obtained. The third sub-network model is a recurrent neural network, which is used to process time series features. The recurrent neural network can capture the temporal dependencies between features through its cyclic structure. The recurrent neural network stores and transmits time series information through hidden state vectors, and its recursive formula is: ; in, is the hidden state at the current time step, is the hidden state of the previous time step, is the input of the current time step. In this way, the recurrent neural network can capture the time dependency of risk factors and obtain the third set of initial feature representations. Each set of initial feature representations is weighted by the cost-sensitive layer of each sub-network model. The cost-sensitive layer assigns different weights to samples of different categories according to the preset cost matrix, increasing the sensitivity of the model to high-cost misclassification. If some categories have higher risks or their misclassification costs are greater, they are assigned higher weights through the cost matrix to obtain multiple sets of cost-weighted features. Each set of cost-weighted features is input into the deep neural network of each sub-network model for processing to obtain multiple sets of deep feature representations. The deep feature representation is further refined through a deeper network structure to capture higher-order feature information. In order to comprehensively utilize the feature representations of different sub-network models, the attention mechanism is applied to analyze multiple sets of deep feature representations. The attention mechanism can highlight the features that are more important for risk prediction by calculating the importance weights of the output of each sub-network model. The weight calculation formula of the attention mechanism is: ; in, is the model’s attention score, is the weight of the sub-network model output. Through the attention mechanism, the attention-weighted feature combination is obtained, which can adaptively focus on the features that are more critical to risk prediction. The attention-weighted feature combination is input into the integration layer of the cost-sensitive deep neural network integration model for fusion. The function of the integration layer is to effectively combine the feature representations of different sub-networks to generate a comprehensive feature representation. This fusion strategy ensures that the advantages of each sub-network can be fully utilized in the overall model. The comprehensive feature representation of the integration layer is mapped to the risk level through the cost-sensitive fully connected layer. The cost-sensitive fully connected layer optimizes the classification results of the model by introducing a cost-sensitive loss function, so that the model can more effectively distinguish different risk levels. The cost-sensitive loss function can be defined as: ; in, The samples are classified into classes The probability of is the cost matrix. By optimizing the cost-sensitive loss function, the model can reduce high-cost misclassification and improve the accuracy of overall risk level prediction. The Softmax function is used to perform probability normalization on the integrated prediction results to obtain the probability distribution of multi-category risk levels. The formula of the Softmax function is: ; in, is the score output by the neural network, Is the sample belonging to the category Through Softmax processing, the model can output the probability distribution of each risk category and provide more fine-grained risk assessment results.
[0035] In one example, warning thresholds are set according to the probability distribution of multi-category risk levels, and similarity matching and decision reasoning are performed in combination with the historical case database to obtain digital risk control decision recommendations, including: analyzing the probability distribution of multi-category risk levels, calculating the optimal classification threshold based on the ROC curve and Youden index, and obtaining a set of risk warning thresholds; comparing the risk warning threshold set with the current risk status, identifying high-risk categories that exceed the threshold, and obtaining a risk warning signal; retrieving similar cases from the historical case database based on the risk warning signal, and using cosine similarity to calculate the similarity between the current risk status and historical cases to obtain a set of similar cases; The collection is clustered and grouped for similar cases to obtain case clustering results, and the core features and decision-making elements of each cluster are extracted based on the case clustering results to construct an initial decision rule set; a decision tree model is constructed according to the initial decision rule set, and the current risk status is inferred based on the decision tree model to generate a set of candidate decision plans; a multi-criteria decision analysis is performed on the candidate decision plan set to obtain a decision plan scoring matrix, and the weight of each evaluation indicator is calculated based on the hierarchical analysis method, and a comprehensive score is performed in combination with the decision plan scoring matrix to obtain the optimal decision plan; the optimal decision plan is combined with the risk warning signal to generate a digital risk control decision recommendation that includes risk level, warning reason and response measures.
[0036] In this example, the probability distribution of multi-category risk levels is analyzed. The probability distribution reflects the risk level of each risk category. In order to extract useful warning information from it, the ROC curve and Youden index are used to calculate the optimal classification threshold. The ROC curve evaluates the performance of the classifier by plotting the relationship between the true positive rate (TPR) and the false positive rate (FPR), while the Youden index is used to measure the optimal performance of the classifier at different thresholds. The calculation formula is: ; Among them, TPR is the proportion of high-risk samples correctly identified by the classifier, and FPR is the proportion of samples incorrectly classified as high-risk. By calculating the Youden index, the classification threshold that makes the classifier most accurate is found, and a set of risk warning threshold sets is generated. The risk warning threshold set is compared with the current risk status. For each risk level, if the probability of the current risk exceeds the corresponding threshold, it can be identified as a high-risk category. Identifying these risk categories that exceed the threshold generates a risk warning signal, indicating that serious risks may exist in the current state. Based on the risk warning signal, similar risk cases are retrieved from the historical case database. In order to find the historical cases with the most reference value, cosine similarity is used to measure the similarity between the current risk status and historical cases. The calculation formula for cosine similarity is: ; in, and Represent the feature vectors of the current risk status and historical cases respectively, · represents the dot product of the vectors, and is the modulus of the vector. Through this calculation, a similar case set is obtained, which contains historical cases similar to the current risk situation. After obtaining the similar case set, these similar cases are clustered and structured information is extracted from them. The similar case set is clustered and grouped to obtain case clustering results, and each cluster contains cases with similar characteristics. Based on the clustering results, the core features and decision elements of each cluster are extracted. These elements represent the key strategies and parameters when dealing with specific types of risks. Through this process, an initial decision rule set is constructed. Based on the extracted initial decision rule set, a decision tree model is constructed. Decision tree is a commonly used classification and decision-making tool. It recursively divides data into different subsets to form a tree structure, thereby helping decision makers choose the best action path. Each node of the decision tree model represents a decision point, and each branch represents a different possible value of a risk factor. By training the decision tree model, a set of candidate decision solutions is generated based on the current risk status. Each candidate solution represents a possible risk response measure, which helps decision makers understand different response paths. For the generated set of candidate decision solutions, multi-criteria decision analysis is performed. Multi-criteria decision analysis is a method to evaluate the pros and cons of multiple solutions. It helps to select the best solution by comprehensively considering multiple evaluation indicators. In this process, a decision solution scoring matrix is constructed. Each row in the matrix represents a candidate solution, and each column represents an evaluation indicator (such as cost, risk reduction, implementation difficulty, etc.). The weight of each evaluation indicator is calculated using the hierarchical analysis method. The hierarchical analysis method compares the importance of each indicator by constructing a comparison matrix and finally determines the relative weight of each indicator. The optimal decision solution is combined with the risk warning signal generated previously to generate a digital risk control decision recommendation containing risk level, warning reason and countermeasures. The risk level describes the risk level in the current state, the warning reason indicates the specific factors or conditions that trigger the warning, and the countermeasures provide specific operational suggestions based on the optimal decision solution. The entire process is data-driven, combining historical data, similar cases and multi-criteria decision analysis to ensure that the generated decision recommendations are scientifically based and practical.
[0037] Reference Figure 2 This embodiment provides an intelligent risk control device for multi-source data fusion, including: Acquisition module 1 is used to collect multi-source heterogeneous data to be processed based on the big data platform, and pre-process and convert the multi-source heterogeneous data into high-dimensional tensors to obtain target high-dimensional tensors; Reconstruction module 2, used to reconstruct the target high-dimensional tensor to obtain a fused reconstructed tensor; Mapping module 3 is used to perform multi-dimensional decomposition and feature extraction on the fused reconstructed tensor to obtain a risk propagation network, and to perform cluster analysis and feature mapping on the risk propagation network to obtain a risk feature mapping set; Prediction module 4, used for inputting the risk feature mapping set into the cost-sensitive deep neural network integration model to predict the risk level and obtain the probability distribution of multi-category risk levels; Decision module 5 is used to set warning thresholds according to the probability distribution of multi-category risk levels, and to perform similarity matching and decision reasoning in combination with the historical case database to obtain digital risk control decision recommendations.
[0038] In this embodiment, for the specific implementation of each unit in the above device embodiment, please refer to the above method embodiment, which will not be repeated here.
[0039] Reference Figure 3 In an embodiment of the present invention, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.
[0040] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.
[0041] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.
[0042] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.
[0043] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, device, article or method including the element.
[0044] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation based on the contents of the present invention description and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. An intelligent risk control method based on multi-source data fusion, characterized in that: The following steps are involved: Based on the big data platform, multi-source heterogeneous data to be processed are collected, and the multi-source heterogeneous data are preprocessed and converted into high-dimensional tensors to obtain target high-dimensional tensors; Reconstructing the target high-dimensional tensor to obtain a fused reconstructed tensor; Performing multi-dimensional decomposition and feature extraction on the fused reconstructed tensor to obtain a risk propagation network, and performing cluster analysis and feature mapping on the risk propagation network to obtain a risk feature mapping set; Inputting the risk feature mapping set into a cost-sensitive deep neural network integration model to predict the risk level, and obtaining a probability distribution of multi-category risk levels; The warning threshold is set according to the probability distribution of the multi-category risk levels, and similarity matching and decision reasoning are performed in combination with the historical case database to obtain digital risk control decision recommendations.
2. The intelligent risk control method based on multi-source data fusion according to claim 1 is characterized in that: The method collects the multi-source heterogeneous data to be processed based on the big data platform, and performs preprocessing and high-dimensional tensor conversion on the multi-source heterogeneous data to obtain the target high-dimensional tensor, including: Collecting multi-source heterogeneous data to be processed based on the big data platform, and performing data cleaning on the multi-source heterogeneous data to obtain a cleaned data set; Standardizing the numerical data in the cleaned data set to obtain standardized numerical data, and vectorizing the categorical data in the cleaned data set to obtain vectorized categorical data; Performing word segmentation and stop word removal processing on the text data in the cleaned data set to obtain preprocessed text data, and performing feature extraction on the preprocessed text data to obtain a text feature vector; Integrate the standardized numerical data, the vectorized classification data and the text feature vector to obtain a standard multi-source data set; Reorganizing the standard multi-source data set into a high-dimensional tensor based on a multi-dimensional array structure, wherein each dimension represents a data attribute, to obtain an initial high-dimensional tensor; The initial high-dimensional tensor is subjected to Tucker decomposition and CP decomposition to obtain a reduced-dimensional high-dimensional tensor, and the missing elements in the reduced-dimensional high-dimensional tensor are estimated and filled based on a tensor completion algorithm to obtain a target high-dimensional tensor.
3. The intelligent risk control method based on multi-source data fusion according to claim 1 is characterized in that: The step of reconstructing the target high-dimensional tensor to obtain a fused reconstructed tensor includes: A1: performing singular value decomposition on the target high-dimensional tensor to obtain a first singular value set and corresponding left and right singular vectors, and truncating the first singular value set based on a preset threshold function to obtain a second singular value set; A2: shrinking the second singular value set using a p-shrinkage operator to obtain a third singular value set, and reconstructing a tensor based on the third singular value set and corresponding left and right singular vectors to obtain an initial reconstructed tensor; A3: Calculate the residual between the initial reconstructed tensor and the target high-dimensional tensor to obtain an initial residual tensor, and perform nuclear norm regularization on the initial residual tensor to obtain a regularized residual tensor; A4: iteratively optimizing the regularized residual tensor based on an alternating direction multiplier method to obtain an optimized residual tensor, and adding the optimized residual tensor to the initial reconstruction tensor to obtain an updated reconstruction tensor; A5: Determine whether the updated reconstructed tensor meets the convergence condition. If not, use the updated reconstructed tensor as the new target high-dimensional tensor and repeat the above steps A1-A4 until the convergence condition is met to obtain a fused reconstructed tensor.
4. The intelligent risk control method based on multi-source data fusion according to claim 1 is characterized in that: The multi-dimensional decomposition and feature extraction of the fused reconstructed tensor are performed to obtain a risk propagation network, and cluster analysis and feature mapping are performed on the risk propagation network to obtain a risk feature mapping set, including: Performing tensor decomposition on the fused reconstructed tensor to obtain a plurality of factor matrices, and extracting a feature vector of each risk factor based on the plurality of factor matrices to obtain a risk factor feature set; Performing dimensionality reduction processing on the risk factor feature set to obtain a risk feature matrix after dimensionality reduction, and calculating a correlation coefficient matrix between risk factors based on the risk feature matrix after dimensionality reduction to obtain an initial risk association matrix; Performing threshold filtering and weak correlation removal on the initial risk association matrix to obtain a filtered risk association matrix, and constructing a risk factor network diagram based on the filtered risk association matrix to obtain an initial risk network; Performing topological analysis on the initial risk network, identifying key nodes and edges, obtaining a risk core set, and calculating the impact strength between nodes based on the risk core set to obtain a risk impact strength matrix; Performing graph embedding analysis on the risk impact intensity matrix, mapping risk factors to a low-dimensional space, obtaining risk factor embedding vectors, and constructing a weighted directed graph based on the risk factor embedding vectors, wherein nodes represent risk factors and edge weights represent coupling strengths, thereby obtaining a risk propagation network; Based on the self-organizing map model, cluster analysis and feature mapping are performed on the risk propagation network to obtain a risk feature mapping set.
5. The intelligent risk control method based on multi-source data fusion according to claim 4 is characterized in that: The risk propagation network is clustered and mapped based on the self-organizing map model to obtain a risk feature mapping set, including: Extracting features from the risk propagation network, converting node attributes and edge weights into feature vectors, and obtaining a risk network feature matrix; Inputting the risk network feature matrix into a self-organizing map model, performing feature mapping on each risk node, and obtaining an initial risk feature distribution on a two-dimensional plane; Analyzing the distance relationship between adjacent nodes based on the initial risk feature distribution to obtain risk clustering boundaries, and clustering the mapping results according to the risk clustering boundaries to identify risk clusters with similar features and obtain risk clustering results; Calculating the coordinates of the center point of each cluster in the risk cluster division result to obtain a risk cluster center set, and calculating the Euclidean distance between each risk cluster based on the risk cluster center set to construct a risk cluster topological structure; Identify high-order risk clusters on the risk cluster topological structure to obtain a multi-level risk cluster system, and calculate the intra-cluster variance, inter-cluster distance and silhouette coefficient of each level of risk clusters in the multi-level risk cluster system to obtain risk cluster characteristic indicators; Based on the risk cluster characteristic index, each risk cluster is ranked by importance to obtain a core risk cluster set, and the core risk cluster set is combined with the topological structure of the risk propagation network to construct a risk characteristic mapping set.
6. The intelligent risk control method based on multi-source data fusion according to claim 1 is characterized in that: The risk feature mapping set is input into a cost-sensitive deep neural network integrated model to predict the risk level, and a probability distribution of multiple risk levels is obtained, including: Performing feature combination and normalization processing on the risk feature mapping set to obtain a standardized risk feature input vector; The standardized risk feature input vector is respectively passed into multiple sub-network models based on a cost-sensitive deep neural network integration model for feature processing to obtain multiple groups of initial feature representations; the multiple sub-network models include a fully connected neural network, a convolutional neural network and a recurrent neural network; the fully connected neural network is processed by a multi-layer perceptron, and each layer uses a ReLU activation function and Dropout regularization to obtain a first group of initial feature representations; the convolutional neural network extracts local and global features through multi-layer convolution, pooling and batch normalization operations to obtain a second group of initial feature representations; the recurrent neural network obtains a third group of initial feature representations by capturing temporal dependencies; Each group of the initial feature representations is weighted by the cost-sensitive layer of the respective sub-network model, and different weights are assigned to samples of different categories according to a preset cost matrix to obtain multiple groups of cost-weighted features; Inputting each group of the cost-weighted features into the corresponding deep neural network of each sub-network model for processing to obtain multiple groups of deep feature representations; Performing attention mechanism analysis on the multiple groups of deep feature representations, calculating the importance weight of each sub-network model output, and obtaining an attention-weighted feature combination; Inputting the attention-weighted feature combination into the integration layer of the cost-sensitive deep neural network integration model for fusion to obtain a comprehensive feature representation, and performing risk level mapping on the comprehensive feature representation through the cost-sensitive fully connected layer of the cost-sensitive deep neural network integration model, and optimizing it using a cost-sensitive loss function to obtain an integrated prediction result; The ensemble prediction results are probability normalized by the Softmax function to obtain the probability distribution of multi-category risk levels.
7. The intelligent risk control method based on multi-source data fusion according to claim 1 is characterized in that: The warning threshold is set according to the probability distribution of the multi-category risk levels, and similarity matching and decision reasoning are performed in combination with the historical case database to obtain digital risk control decision suggestions, including: Analyze the probability distribution of the multi-category risk levels, calculate the optimal classification threshold based on the ROC curve and Youden index, and obtain a risk warning threshold set; Compare the risk warning threshold set with the current risk status, identify high-risk categories exceeding the threshold, and obtain a risk warning signal; Retrieving similar cases from a historical case database based on the risk warning signal, using cosine similarity to calculate the similarity between the current risk status and the historical cases, and obtaining a set of similar cases; Clustering the similar cases into groups to obtain case clustering results, extracting core features and decision elements of each cluster based on the case clustering results, and constructing an initial decision rule set; Constructing a decision tree model according to the initial decision rule set, and reasoning about the current risk status based on the decision tree model to generate a set of candidate decision solutions; Performing a multi-criteria decision analysis on the candidate decision solution set to obtain a decision solution scoring matrix, and calculating the weight of each evaluation indicator based on the hierarchical analysis method, and performing a comprehensive score in combination with the decision solution scoring matrix to obtain the optimal decision solution; Combine the optimal decision-making plan with the risk warning signal to generate a digital risk control decision recommendation that includes risk level, warning reason and response measures.
8. An intelligent risk control device for multi-source data fusion, characterized in that: For implementing the steps of the method according to any one of claims 1 to 7, the device comprises: An acquisition module is used to acquire multi-source heterogeneous data to be processed based on a big data platform, and to perform preprocessing and high-dimensional tensor conversion on the multi-source heterogeneous data to obtain a target high-dimensional tensor; A reconstruction module, used for reconstructing the target high-dimensional tensor to obtain a fused reconstructed tensor; A mapping module, used for performing multi-dimensional decomposition and feature extraction on the fused reconstructed tensor to obtain a risk propagation network, and performing cluster analysis and feature mapping on the risk propagation network to obtain a risk feature mapping set; A prediction module, used for inputting the risk feature mapping set into a cost-sensitive deep neural network integrated model to predict the risk level and obtain a probability distribution of multi-category risk levels; The decision-making module is used to set the warning threshold according to the probability distribution of the multi-category risk levels, and perform similarity matching and decision reasoning in combination with the historical case database to obtain digital risk control decision recommendations.
9. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Cited By
Intelligent substation secondary equipment defect diagnosis method
CN119337215A
Engineering project risk management method and system
CN119359049A
Data risk intelligent identification method and system based on collaborative analysis
CN119377825A
Intelligent prediction and response system for business risk control strategy
CN119379452A
Supply chain finance intelligent risk control system and method thereof
CN119624633A
