Scientific and technological financial risk identification method and system based on multi-modal data fusion

Abstract

The invention discloses a science and technology financial risk identification method and system based on multi-modal data fusion, and the method comprises the following steps: collecting multi-source heterogeneous data, and carrying out the preprocessing; performing modal space alignment and scale normalization on the data to generate a preliminary fusion representation vector; performing semantic association modeling and cross-modal feature collaborative enhancement by adopting a cross attention mechanism to generate a fusion representation vector; cross-modal semantic projection optimization and high-order semantic completion are executed through an improved AlignXpert algorithm, and final fusion representation is generated; based on the final fusion representation, constructing a time causal graph taking the risk identification object entity as the center; inputting the time causal graph into a graph neural network model, and outputting a risk level, an evolution trend and a key causal chain; and continuously updating by adopting a graph structure increment construction and model parameter fine adjustment mechanism. According to the method, the accuracy and interpretability of science and technology financial risk identification under multi-source data are improved, and the method has relatively high intelligence, expandability and practical application value.

Description

Technical Field

[0001] This invention relates to the field of intelligent financial risk assessment technology, and in particular to a method and system for identifying financial technology risks based on multimodal data fusion. Background Technology

[0002] With the rapid growth in risk identification needs in the fintech sector, intelligent fusion and deep modeling of multi-source heterogeneous data, including corporate financial information, governance structures, textual semantics, and external events, have become crucial for achieving accurate risk identification. Existing risk identification methods often employ machine learning models based on single-modality or weak fusion strategies for feature extraction and classification. However, the following key issues remain in practical applications:

[0003] First, existing methods typically employ independent modeling approaches for processing multimodal data such as corporate financial statements, governance structure information, and text corpora, lacking a unified mechanism for aligning embedding spaces and semantic dimensions. This leads to inconsistent cross-modal information expression, weak semantic representation capabilities after fusion, and impacts the accuracy of risk identification. Second, current mainstream multimodal fusion strategies largely rely on static weighting or simple concatenation, failing to fully consider the semantic complementarity and feature synergy between modalities, making it difficult to effectively model complex cross-modal interactive semantics. Third, regarding the temporal correlation between corporate behavior and risk events, most methods ignore the causal chain of events and the risk evolution path, relying solely on static classification based on planar features, making it difficult to achieve dynamic prediction and interpretable output of risk trends.

[0004] Therefore, how to provide a method and system for identifying financial risks based on multimodal data fusion is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a method and system for identifying technology finance risks based on multimodal data fusion. This invention fully integrates heterogeneous information from multiple sources, such as financial statements, governance structures, textual semantics, and external events. It constructs a unified fusion representation using mechanisms such as modal space alignment, cross-attention enhancement, and high-order semantic completion. It combines time-causal graphs and graph neural network models to achieve risk level prediction, evolution trend analysis, and identification of key causal chains. It has the advantages of high data fusion accuracy, strong risk identification capability, and good interpretability of evolution paths.

[0006] The technology finance risk identification method based on multimodal data fusion according to an embodiment of the present invention includes the following steps: Collect heterogeneous data from multiple sources, perform missing data completion, format standardization, and modality identification processing to obtain a multimodal dataset; Feature extraction is performed on the multimodal dataset to obtain financial vectors, structural vectors, and semantic vectors. These vectors are then initially fused through modal space alignment and scale normalization to generate a preliminary fused representation vector. A cross-attention mechanism guided by modal feature correlation is adopted to perform semantic association modeling and cross-modal feature collaborative enhancement on the preliminary fused representation vector to generate a fused representation vector; An improved AlignXpert algorithm is used to perform cross-modal semantic projection optimization and high-order semantic completion on the fused representation vector to generate the final fused representation; Based on the final fusion representation, a time-causal graph centered on the risk identification object entity is constructed; Input the time-cause-effect graph into the time-cause-effect graph neural network model, and output the risk level prediction results, risk evolution trend and key causal chain of the risk identification object; Based on continuous updates of multimodal data, an incremental graph structure is used to expand the temporal causal graph, and model parameter fine-tuning and adaptive structural optimization are performed to update the risk identification capability of the temporal causal graph neural network model.

[0007] Optionally, the multi-source heterogeneous data includes financial statement data, governance structure information, enterprise-related text corpora, and records of external risk events.

[0008] Optionally, the step of extracting features from the multimodal dataset to obtain financial vectors, structural vectors, and semantic vectors, and then performing preliminary fusion through modal space alignment and scale normalization, generates a preliminary fused representation vector, specifically as follows: Numerical indicators from financial data are input into a multilayer perceptron neural network. Through layer-by-layer nonlinear transformation and weight training mechanism, they are mapped into low-dimensional semantic embedding representations to generate financial vectors. Based on the node types, node attributes, and their connection relationships contained in the structured data, an enterprise structure graph is constructed, and a graph neural network is used to calculate the adjacency propagation of node states to generate structure vectors. Textual semantic data is input into the BERT model, which uses its bidirectional attention mechanism to encode the contextual content and combines it with the output representation of the CLS position to generate a semantic vector. By setting shared embedding dimensions and modal similarity constraints, modal embedding space alignment operations are performed on financial vectors, structural vectors, and semantic vectors to generate a set of aligned vectors with unified semantic dimensions. Max-min normalization is performed on each modal vector in the alignment vector set to map their values ​​to a unified range. Then, they are concatenated in a set order to form a fused feature representation vector. The mean-variance standardization is applied to the fused feature representation vector as a whole to generate a preliminary fused representation vector.

[0009] Optionally, the step of performing semantic association modeling and cross-modal feature collaborative enhancement on the preliminary fused representation vector specifically involves: We construct cross-attention computation pairs that combine the three modal features in the preliminary fusion representation vectors, including financial vector and structural vector, financial vector and semantic vector, and structural vector and semantic vector; Calculate the feature relevance weights between each cross-attention pair, use weighted cosine similarity as a metric, and determine the attention mapping direction based on the weight results; In each cross-attention computation pair, a cross-attention mapping is constructed using the lower relevance modality as the query vector and the higher relevance modality as the key and value vectors. Cross-modal semantic enhancement operations are then performed to obtain three sets of cross-attention enhancement vectors. The three sets of cross-attention enhancement vectors are concatenated and combined, and then fused using a modality weighting mechanism based on learnable weights to generate a unified set of semantic enhancement vectors. The semantically enhanced vector set is processed by feature compression and dimension preservation to output a fused representation vector.

[0010] Optionally, the improved AlignXpert algorithm is used to perform cross-modal semantic projection optimization and higher-order semantic completion on the fused representation vector, specifically as follows: Multi-head attention computation is performed on the financial modality subspace, structural modality subspace, and semantic modality subspace retained in the fused representation vector to generate cross-modal attention coefficients between the three types of modality subspaces. Based on these coefficients, a weighted combination of the three types of modality subvectors is performed to generate an attention weighted vector. The fused representation vector and the attention weighted vector are respectively input into a one-dimensional convolutional network, and sliding convolution is performed to obtain the residual information vector. The residual information vector is then input into a non-linear activation function to enhance the difference in feature expression, forming a residual completion vector. The attention weighted vector and the residual completion vector are concatenated along the feature dimension to form a joint representation vector. The joint representation vector is then input into multiple semantic mapping networks, each of which contains a fully connected layer and a non-linear activation function. For the semantic subspace vectors output by each semantic mapping network, calculate the KL divergence between the semantic subspace vectors, construct a semantic consistency supervision loss function, and optimize the parameters of each semantic mapping network through the backpropagation mechanism; Each semantic subspace vector is input into the compression mapping structure, and feature dimension normalization, linear transformation and semantic fusion operations are performed in sequence. Finally, the high-order feature information retained in the residual completion vector is combined to generate the final fused representation.

[0011] Optionally, the construction of the time-cause-effect graph centered on the risk identification object entity specifically involves: Based on the final fusion representation, the historical behavior trajectory and related event content of each risk identification object entity are identified, and a node set is constructed. The node types include historical state nodes and risk event nodes. Based on the time sequence and logical triggering relationship of historical state nodes and risk event nodes, a set of directed edges is constructed, and each edge is assigned a timestamp. Based on the high-order semantic clues and historical association information retained in the final fused representation, causal weight parameters are calculated for each edge; An initial temporal causal graph is constructed based on the set of nodes and the set of directed edges. Then, time window constraints, edge weight normalization, and loop path elimination operations are performed in sequence to normalize the graph structure. Using the risk-identified object entity as the anchor node of the time causal graph, the corresponding final fusion representation index information is marked to form a time causal graph centered on the risk-identified object entity; Optionally, the step of inputting the time-causal graph into the time-causal graph neural network model and outputting the risk level prediction results, risk evolution trend, and key causal chains of the risk identification object specifically includes: The temporal causal graph is input into the temporal causal graph neural network model. Combining the final fused representation index information and timestamp information of each node, an initial state representation vector is generated for each node through the node feature embedding initialization method. In the temporal causal graph, a time decay factor is constructed based on the timestamp difference marked on the edge, and this factor is multiplied by the causal weight parameter as the edge weight value for information propagation. This guides the state representation vector of the historical state node to perform weighted propagation to the risk event node, updating the temporal context dependency structure between nodes. At each risk event node, a multi-head attention mechanism is used to normalize the propagation feature representations from different historical paths, and a dynamic gating strategy is introduced to aggregate the path features with weights. The influence strength of the path is calculated based on the product of the aggregate weight and the node prediction probability. The dominant causal path is selected and a causal chain representation is formed. At the same time, the state representation vector of the risk event node is updated. The final node state representation vector of each risk identification object entity after propagation and aggregation in the time causal graph is concatenated with the corresponding causal chain representation, and input into the risk level classification prediction branch and the trend regression prediction branch respectively, and the risk level label and risk evolution trend curve are output. By matching the path sequence in the causal chain representation with the predicted risk level of each node, the critical path with the greatest risk impact and the risk evolution stage are identified, generating structured key causal chains and risk path explanation information.

[0012] Optionally, the continuous updating based on multimodal data employs an incremental graph structure construction method to expand the temporal causal graph, and performs model parameter fine-tuning and adaptive structural optimization to update the risk identification capability of the temporal causal graph neural network model, specifically as follows: Based on the newly accessed multimodal data samples, corresponding financial vectors, structural vectors, and semantic vectors are constructed, and preliminary fusion, cross-modal collaborative enhancement, and projection optimization operations are performed in sequence to generate a new final fusion representation; The new final fusion representation is mapped to the corresponding risk identification object entity in the time causal graph. Based on its timestamp information and semantic content features, the corresponding historical state nodes and risk event nodes are incrementally added to the graph, and causal edge connections with the relevant nodes in the original graph are constructed to form the updated time causal graph. Normalization is performed on newly added nodes and edges, their state representation vectors and causal edge weight parameters are calculated, and a local propagation mechanism is adopted to perform information propagation and embedding updates only within the neighborhood associated with the newly added structure. Using newly added subgraphs in the temporal causal graph as the source of supervision signals, local parameter fine-tuning and structural adaptive optimization are performed on the temporal causal graph neural network model. A joint loss function based on gradient smoothing and structural consistency is adopted to improve the model's ability to adapt to new semantic and behavioral patterns. The updated temporal causal graph neural network model, together with the expanded temporal causal graph, forms the basis of a new risk identification system, supporting continuous output of risk level, evolution trend and key causal chain, and forming a closed-loop feedback path.

[0013] The technology finance risk identification system based on multimodal data fusion according to an embodiment of the present invention includes the following modules: The data processing module is used to collect multi-source heterogeneous data, perform missing data completion, format standardization and modality identification processing, and construct a multimodal dataset; The feature fusion module is used to generate financial vectors, structural vectors, and semantic vectors from financial data, structured data, and text semantic data in the multimodal dataset, respectively, and then performs modal space alignment, scale normalization, cross-modal semantic association modeling, and collaborative enhancement in sequence to generate a fused representation vector. The semantic projection optimization module is used to perform cross-modal semantic projection optimization and high-order semantic completion on the fused representation vector to generate the final fused representation. The causal graph construction module is used to construct a time-based causal graph centered on the risk identification object entity based on the final fusion representation. The graph includes historical state nodes and risk event nodes, and timestamp information and causal weight parameters are set for each edge. The risk identification module is used to input the time causal graph into the time causal graph neural network model and output the risk level prediction results, risk evolution trend and key causal chain of the risk identification object; The model update module is used for continuous updates based on multimodal data. It expands the time causal graph by incrementally constructing a graph structure and performs fine-tuning of model parameters and adaptive optimization of structure to update the risk identification capability of the time causal graph neural network model.

[0014] The beneficial effects of this invention are: (1) Improve the accuracy and consistency of multimodal information fusion: By constructing financial vectors, structural vectors and semantic vectors, and performing operations such as modal embedding space alignment, scale normalization, cross attention enhancement and high-order semantic completion in sequence, the inconsistency of multi-source heterogeneous data in terms of semantic granularity, dimensional structure and information redundancy is effectively solved, and the semantic expression integrity and intermodal synergy of the fusion representation are significantly improved.

[0015] (2) Enhance the ability to model risk evolution paths and explain causal chains: Introduce a time-causal graph modeling method centered on the risk identification object entity, combine timestamps, causal weights and high-order semantic cues to construct a graph structure that reflects the law of state evolution, and integrate time decay mechanism and dynamic gating strategy in graph neural network, which can accurately identify the dominant propagation path and key causal nodes of risk events, and enhance the interpretability and traceability of the model.

[0016] (3) Support the dynamic evolution of the system and the continuous updating of risk identification capabilities: Based on the continuous access and structural changes of multimodal data, the graph structure incremental expansion and model local fine-tuning mechanism can be adopted to achieve adaptive updates of the risk identification model without destroying the original global structure, thereby improving the system's ability to perceive new risk semantics and behavioral patterns and ensuring the accuracy of risk identification and the efficiency of system response. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of the technology finance risk identification method based on multimodal data fusion proposed in this invention; Figure 2 This is a schematic diagram of the technology finance risk identification system based on multimodal data fusion proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1 A technology finance risk identification method based on multimodal data fusion includes the following steps: Step 1: Collect multi-source heterogeneous data, perform missing data completion, format standardization, and modality identification processing to obtain a multimodal dataset; Step 2: Extract features from the multimodal dataset to obtain financial vectors, structural vectors, and semantic vectors. Then, perform preliminary fusion through modal space alignment and scale normalization to generate a preliminary fused representation vector. Step 3: Employ a cross-attention mechanism guided by modal feature relevance to perform semantic association modeling and cross-modal feature collaborative enhancement on the preliminary fused representation vector, generating a fused representation vector; Step 4: Using the improved AlignXpert algorithm, perform cross-modal semantic projection optimization and high-order semantic completion on the fused representation vector to generate the final fused representation; Step 5: Based on the final fused representation, construct a time-causal graph centered on the risk identification object entity; Step Six: Input the time-cause-effect graph into the time-cause-effect graph neural network model, and output the risk level prediction results, risk evolution trend, and key causal chains of the risk identification object; Step 7: Based on continuous updates of multimodal data, expand the temporal causal graph using an incremental graph structure construction method, and perform model parameter fine-tuning and adaptive structural optimization to update the risk identification capability of the temporal causal graph neural network model.

[0021] In this embodiment, the multi-source heterogeneous data includes financial statement data, governance structure information, enterprise-related text corpus, and external risk event records.

[0022] In this embodiment, feature extraction of the multimodal dataset yields financial vectors, structural vectors, and semantic vectors. Preliminary fusion is then performed through modal space alignment and scale normalization to generate a preliminary fused representation vector. Specifically: Numerical indicators from financial data are input into a multilayer perceptron neural network. Through layer-by-layer nonlinear transformation and weight training mechanism, they are mapped into low-dimensional semantic embedding representations to generate financial vectors. Based on the node types, node attributes, and their connection relationships contained in the structured data, an enterprise structure graph is constructed, and a graph neural network is used to calculate the adjacency propagation of node states to generate structure vectors. Textual semantic data is input into the BERT model, which uses its bidirectional attention mechanism to encode the contextual content and combines it with the output representation of the CLS position to generate a semantic vector. By setting shared embedding dimensions and modal similarity constraints, modal embedding space alignment operations are performed on financial vectors, structural vectors, and semantic vectors to generate a set of aligned vectors with unified semantic dimensions. Max-min normalization is performed on each modal vector in the alignment vector set to map their values ​​to a unified range. Then, they are concatenated in a set order to form a fused feature representation vector. The mean-variance standardization is applied to the fused feature representation vector as a whole to generate a preliminary fused representation vector.

[0023] This implementation significantly improves the semantic consistency and collaborative modeling capabilities among heterogeneous data by unifying the embedding dimension and performing modality space alignment and normalization fusion. It solves the problems of inconsistent expression scales and alignment difficulties among different modalities, and provides a unified and highly expressive fusion representation foundation for subsequent semantic enhancement and causal modeling.

[0024] In this embodiment, the semantic association modeling and cross-modal feature collaborative enhancement of the preliminary fused representation vector specifically includes: We construct cross-attention computation pairs that combine the three modal features in the preliminary fusion representation vectors, including financial vector and structural vector, financial vector and semantic vector, and structural vector and semantic vector; Calculate the feature relevance weights between each cross-attention pair, use weighted cosine similarity as a metric, and determine the attention mapping direction based on the weight results; In each cross-attention computation pair, a cross-attention mapping is constructed using the lower relevance modality as the query vector and the higher relevance modality as the key and value vectors. Cross-modal semantic enhancement operations are then performed to obtain three sets of cross-attention enhancement vectors. The three sets of cross-attention enhancement vectors are concatenated and combined, and then fused using a modality weighting mechanism based on learnable weights to generate a unified set of semantic enhancement vectors. The semantically enhanced vector set is processed by feature compression and dimension preservation to output a fused representation vector.

[0025] This implementation constructs cross-attention pairs, calculates relevance weights, and performs cross-modal semantic enhancement operations to generate three sets of attention enhancement vectors. Furthermore, it introduces a learnable weight mechanism to compress dimensions and generate a fusion representation vector after fusion. This effectively achieves intermodal information complementarity and feature collaborative expression, enhances the recognition ability of key semantics, overcomes the limitations of single-modal expression and the problem of semantic isolation between modalities, and improves the expression effect of fusion representation on risk features.

[0026] In this embodiment, the improved AlignXpert algorithm is used to perform cross-modal semantic projection optimization and higher-order semantic completion on the fused representation vector, specifically as follows: Multi-head attention computation is performed on the financial modality subspace, structural modality subspace, and semantic modality subspace retained in the fused representation vector to generate cross-modal attention coefficients between the three types of modality subspaces. Based on these coefficients, a weighted combination of the three types of modality subvectors is performed to generate an attention weighted vector. The fused representation vector and the attention weighted vector are respectively input into a one-dimensional convolutional network, and sliding convolution is performed to obtain the residual information vector. The residual information vector is then input into the ReLU function to enhance the difference in feature expression and form a residual completion vector. The attention weighted vector and the residual completion vector are concatenated along the feature dimension to form a joint representation vector. The joint representation vector is then input into multiple semantic mapping networks, each of which contains a fully connected layer and a ReLU function. For the semantic subspace vectors output by each semantic mapping network, the KL divergence between the semantic subspace vectors is calculated, a semantic consistency supervised loss function is constructed, and the parameters of each semantic mapping network are optimized through a backpropagation mechanism. Specifically, for any two semantic subspace vectors... and First, it is normalized using softmax and then denoted as... and The semantic consistency supervision loss function is defined as: ; By minimizing this loss function, consistency constraints are achieved in the subspace distribution of multiple semantic mapping networks, which encourages each modality to maintain cooperative expression in the semantic space, avoids semantic shifts in cross-modal representations, and improves the robustness and semantic completeness of the final fused representation. Each semantic subspace vector is input into the compression mapping structure, and feature dimension normalization, linear transformation and semantic fusion operations are performed in sequence. Finally, the high-order feature information retained in the residual completion vector is combined to generate the final fused representation.

[0027] This implementation introduces an improved AlignXpert algorithm, combining multi-head attention mechanisms and cross-modal weighted combination operations to fully capture the synergistic enhancement features among financial, structural, and semantic information. It generates residual completion vectors through one-dimensional convolution and nonlinear activation functions, effectively compensating for higher-order nonlinear relationships missed in the initial fusion. Furthermore, it utilizes multiple semantic mapping networks to construct a subspace projection structure and employs KL divergence to build a semantic consistency supervision loss function, thereby enhancing representation consistency and semantic robustness from multiple semantic perspectives.

[0028] In this embodiment, the construction of the time-cause-effect graph centered on the risk identification object entity specifically refers to: Based on the final fusion representation, the historical behavior trajectory and related event content of each risk identification object entity are identified, and a node set is constructed. The node types include historical state nodes and risk event nodes. Based on the time sequence and logical triggering relationship of historical state nodes and risk event nodes, a set of directed edges is constructed, and each edge is assigned a timestamp. Based on the high-order semantic cues and historical association indications retained in the final fusion representation, causal weight parameters are calculated for each edge. Specifically, high-order semantic cues are extracted from the final fusion representation, including modal residual information, cross-modal attention aggregation features, and semantic subspace fusion representation. Historical association indications between adjacent nodes are obtained, including behavior trigger labels, time intervals, event levels, and type similarity indicators. A causal weight scoring function is constructed. The high-order semantic cues and historical association indications are input into a multilayer perceptron neural network, and the initial causal weights of the edges are output. The causal weights of all edges are normalized, and weak causal connections below a preset threshold are removed. An initial temporal causal graph is constructed based on the set of nodes and the set of directed edges. Then, time window constraints, edge weight normalization, and loop path elimination operations are performed in sequence to normalize the graph structure. Using the risk-identified object entity as the anchor node of the time causal graph, the corresponding final fusion representation index information is marked to form a time causal graph centered on the risk-identified object entity; This implementation method parses high-order semantic cues and historical association indications in the final fusion representation to construct an input vector that integrates semantic differences and historical behavioral features. This enables the accurate identification of triggering features and transmission strength in key causal paths, improves the credibility and interpretability of graph structure causal modeling, effectively enhances the causal identification capability and temporal coherence between risk events, and provides a structural optimization basis for subsequent risk trend prediction.

[0029] In this embodiment, the step of inputting the time-causal graph into the time-causal graph neural network model and outputting the risk level prediction results, risk evolution trend, and key causal chains of the risk identification object specifically involves: The temporal causal graph is input into the temporal causal graph neural network model. Combining the final fused representation index information and timestamp information of each node, an initial state representation vector is generated for each node through the node feature embedding initialization method. In the temporal causal graph, a time decay factor is constructed based on the timestamp difference marked on the edge, and this factor is multiplied by the causal weight parameter as the edge weight value for information propagation. This guides the state representation vector of the historical state node to perform weighted propagation to the risk event node, updating the temporal context dependency structure between nodes. At each risk event node, a multi-head attention mechanism is used to normalize the propagation feature representations from different historical paths, and a dynamic gating strategy is introduced to aggregate the path features with weights. The influence strength of the path is calculated based on the product of the aggregate weight and the node prediction probability. The dominant causal path is selected and a causal chain representation is formed. At the same time, the state representation vector of the risk event node is updated. The final node state representation vector of each risk identification object entity after propagation and aggregation in the time causal graph is concatenated with the corresponding causal chain representation, and input into the risk level classification prediction branch and the trend regression prediction branch respectively, and the risk level label and risk evolution trend curve are output. By matching the path sequence in the causal chain representation with the predicted risk level of each node, the critical path with the greatest risk impact and the risk evolution stage are identified, generating structured key causal chains and risk path explanation information.

[0030] This implementation constructs an edge-weighting mechanism that combines timestamp differences and causal weights to guide the biased propagation of the state representation vector in the temporal causal graph. At risk event nodes, a multi-head attention mechanism and dynamic gating strategy are employed to aggregate path features and extract dominant causal chains. This method effectively enhances the model's ability to identify key causal paths, improves the accuracy of risk level and evolution trend prediction, and possesses good path interpretability and temporal reasoning capabilities, thus contributing to the comprehensive assessment and early warning response of fintech risks.

[0031] In this embodiment, the continuous updating based on multimodal data employs an incremental graph structure construction method to expand the temporal causal graph, and performs model parameter fine-tuning and adaptive structural optimization to update the risk identification capability of the temporal causal graph neural network model. Specifically: Based on the newly accessed multimodal data samples, corresponding financial vectors, structural vectors, and semantic vectors are constructed, and preliminary fusion, cross-modal collaborative enhancement, and projection optimization operations are performed in sequence to generate a new final fusion representation; The new final fusion representation is mapped to the corresponding risk identification object entity in the time causal graph. Based on its timestamp information and semantic content features, the corresponding historical state nodes and risk event nodes are incrementally added to the graph, and causal edge connections with the relevant nodes in the original graph are constructed to form the updated time causal graph. Normalization is performed on newly added nodes and edges, their state representation vectors and causal edge weight parameters are calculated, and a local propagation mechanism is adopted to perform information propagation and embedding updates only within the neighborhood associated with the newly added structure. Using newly added subgraphs in the temporal causal graph as the source of supervision signals, local parameter fine-tuning and structural adaptive optimization are performed on the temporal causal graph neural network model. A joint loss function based on gradient smoothing and structural consistency is adopted to improve the model's ability to adapt to new semantic and behavioral patterns. The updated temporal causal graph neural network model, together with the expanded temporal causal graph, forms the basis of a new risk identification system, supporting continuous output of risk level, evolution trend and key causal chain, and forming a closed-loop feedback path.

[0032] This implementation constructs a fusion representation of newly added multimodal data samples and expands the temporal causal graph using an incremental graph structure construction method. This method enables rapid adaptation to new risk patterns and efficient integration of new structures, significantly improving the robustness, response flexibility, and continuous optimization capability of the temporal causal graph neural network model in risk identification in dynamic environments, and effectively supporting risk evolution tracking and closed-loop feedback in fintech scenarios.

[0033] refer to Figure 2 The technology finance risk identification system based on multimodal data fusion includes the following modules: The data processing module is used to collect multi-source heterogeneous data, perform missing data completion, format standardization and modality identification processing, and construct a multimodal dataset; The feature fusion module is used to generate financial vectors, structural vectors, and semantic vectors from financial data, structured data, and text semantic data in the multimodal dataset, respectively, and then performs modal space alignment, scale normalization, cross-modal semantic association modeling, and collaborative enhancement in sequence to generate a fused representation vector. The semantic projection optimization module is used to perform cross-modal semantic projection optimization and high-order semantic completion on the fused representation vector to generate the final fused representation. The causal graph construction module is used to construct a time-based causal graph centered on the risk identification object entity based on the final fusion representation. The graph includes historical state nodes and risk event nodes, and timestamp information and causal weight parameters are set for each edge. The risk identification module is used to input the time causal graph into the time causal graph neural network model and output the risk level prediction results, risk evolution trend and key causal chain of the risk identification object; The model update module is used for continuous updates based on multimodal data. It expands the time causal graph by incrementally constructing a graph structure and performs fine-tuning of model parameters and adaptive optimization of structure to update the risk identification capability of the time causal graph neural network model.

[0034] Example 1: To verify the feasibility of this invention in practice, it was applied to a risk monitoring platform for technology finance enterprises in a certain region. 2,634 technology companies registered within the past three years in a specific province were selected as research subjects, covering key areas such as biotechnology, artificial intelligence, new energy materials, and high-end equipment manufacturing. The multi-source heterogeneous data for these companies included their annual financial statements (quarterly dimension), shareholder structure and board composition, business registration history, litigation and credit records, as well as policy and public opinion texts and announcements of sudden risk events. The platform used the method described in this invention to perform risk level identification and evolution trend modeling, focusing on evaluating its dynamic adaptability and risk identification accuracy under different time windows.

[0035] First, the data processing module connects to the aforementioned multi-source data, uniformly employing a hybrid strategy of mean imputation and temporal interpolation for missing value imputation, unified time format, indicator naming, and modal identifier encoding. Subsequently, the feature fusion module inputs financial indicators into a three-layer MLP structure, outputting a 128-dimensional financial vector; corporate governance structure data undergoes graph construction and GCN encoding to generate a 128-dimensional structure vector; and text corpora are processed by BERT-Base to extract CLS bit embeddings, serving as 768-dimensional semantic vectors, which are then compressed to 128 dimensions via linear mapping. Finally, modal space alignment, min-max normalization, and concatenation standardization are performed on the three types of modal vectors, ultimately forming a preliminary fusion representation vector of a uniform length of 384 dimensions.

[0036] The fused representation vectors then enter the semantic enhancement stage, employing a cross-attention mechanism to perform trimodal collaborative enhancement. The system uses weighted cosine similarity to calculate intermodal attention directions and introduces a modality weighting factor. To control the influence of the dominant modality, a multi-layer Transformer structure is used to perform mapping and feature reorganization, generating a fused semantically enhanced representation vector. In the improved AlignXpert semantic optimization stage, a three-head attention and a one-dimensional sliding convolutional network are introduced to extract the residual signal, and a semantic consistency loss function based on KL divergence is constructed. Five rounds of reverse updates are performed on the semantic mapping network to finally obtain a high-order semantic fusion representation.

[0037] When constructing the time-causal graph, the system extracts state nodes and event nodes based on each enterprise's annual behavioral sequence and risk event reports, constructing a directed edge structure with timestamps and causal weights. The generated time-causal graph consists of 3031 state nodes and 1592 risk event nodes, with a total of 8475 edges. The graph structure is input into the time-causal graph neural network model for risk level prediction and causal path output. To evaluate the recognition effect of this invention, comparative experiments were conducted with the traditional logistic regression model, the standard LSTM model, and the Graph Attention Network (GAT). The experimental results are shown in the table below:

[0038] Table 1. Performance Comparison of the Invention Method and Existing Models

[0039] Experimental results show that, in terms of AUC, the method of this invention achieves 0.887, significantly higher than traditional logistic regression (0.713) and LSTM (0.776), and also superior to the current mainstream graph attention network method (0.821). This metric indicates that the method of this invention has a stronger ability to distinguish between different risk levels and can more accurately identify risky and non-risky enterprises. Secondly, in terms of F1-score, the method of this invention achieves 0.759, significantly better than the other three methods, indicating a leading advantage in comprehensive performance that balances recall and precision, especially in handling the problem of imbalanced positive and negative samples. In terms of prediction lead time, the method of this invention achieves 2.6 months, a significant improvement compared to logistic regression (0.5 months) and LSTM (1.1 months), indicating that the system can capture potential risk signs earlier, providing valuable risk intervention time for financial regulatory agencies or investment and financing platforms. In addition, in terms of the accuracy of key causal chains, the method of this invention achieves 81.3%, significantly exceeding the 64.8% of graph attention networks. This indicator is derived from manual annotation and verification, comparing the correspondence between high-weight causal paths identified by the system and actual risk events. The results show that this invention has higher reliability and explanatory power in identifying causal mechanisms.

[0040] This embodiment verifies the applicability and significant effects of the present invention, particularly in that: the construction of the fusion representation fully preserves the multimodal high-order semantics, effectively improving the accuracy of risk identification; the causal graph neural network can provide interpretable causal chain information, realizing the transparency of prediction results; the model has extremely strong incremental adaptability, supports dynamic updates and continuous identification, and meets the dual requirements of timeliness and accuracy in regulatory practice.

[0041] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for identifying technology finance risks based on multimodal data fusion, characterized in that, Includes the following steps: Collect heterogeneous data from multiple sources, perform missing data completion, format standardization, and modality identification processing to obtain a multimodal dataset; Feature extraction is performed on the multimodal dataset to obtain financial vectors, structural vectors, and semantic vectors. These vectors are then initially fused through modal space alignment and scale normalization to generate a preliminary fused representation vector. A cross-attention mechanism guided by modal feature correlation is adopted to perform semantic association modeling and cross-modal feature collaborative enhancement on the preliminary fused representation vector to generate a fused representation vector; An improved AlignXpert algorithm is used to perform cross-modal semantic projection optimization and high-order semantic completion on the fused representation vector to generate the final fused representation; Based on the final fusion representation, a time-causal graph centered on the risk identification object entity is constructed; Input the time-cause-effect graph into the time-cause-effect graph neural network model, and output the risk level prediction results, risk evolution trend and key causal chain of the risk identification object; Based on continuous updates of multimodal data, an incremental graph structure is used to expand the temporal causal graph, and model parameter fine-tuning and adaptive structural optimization are performed to update the risk identification capability of the temporal causal graph neural network model.

2. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, The multi-source heterogeneous data includes financial statement data, governance structure information, enterprise-related text corpora, and records of external risk events.

3. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, The process involves feature extraction from the multimodal dataset to obtain financial vectors, structural vectors, and semantic vectors. These are then preliminarily fused through modal space alignment and scale normalization to generate a preliminary fused representation vector. Specifically: Numerical indicators from financial data are input into a multilayer perceptron neural network. Through layer-by-layer nonlinear transformation and weight training mechanism, they are mapped into low-dimensional semantic embedding representations to generate financial vectors. Based on the node types, node attributes, and their connection relationships contained in the structured data, an enterprise structure graph is constructed, and a graph neural network is used to calculate the adjacency propagation of node states to generate structure vectors. Textual semantic data is input into the BERT model, which uses its bidirectional attention mechanism to encode the contextual content and combines it with the output representation of the CLS position to generate a semantic vector. By setting shared embedding dimensions and modal similarity constraints, modal embedding space alignment operations are performed on financial vectors, structural vectors, and semantic vectors to generate a set of aligned vectors with unified semantic dimensions. Max-min normalization is performed on each modal vector in the alignment vector set to map their values ​​to a unified range. Then, they are concatenated in a set order to form a fused feature representation vector. The mean-variance standardization is applied to the fused feature representation vector as a whole to generate a preliminary fused representation vector.

4. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, The semantic association modeling and cross-modal feature collaborative enhancement of the preliminary fused representation vector are specifically as follows: We construct cross-attention computation pairs that combine the three modal features in the preliminary fusion representation vectors, including financial vector and structural vector, financial vector and semantic vector, and structural vector and semantic vector; Calculate the feature relevance weights between each cross-attention pair, use weighted cosine similarity as a metric, and determine the attention mapping direction based on the weight results; In each cross-attention computation pair, a cross-attention mapping is constructed using the lower relevance modality as the query vector and the higher relevance modality as the key and value vectors. Cross-modal semantic enhancement operations are then performed to obtain three sets of cross-attention enhancement vectors. The three sets of cross-attention enhancement vectors are concatenated and combined, and then fused using a modality weighting mechanism based on learnable weights to generate a unified set of semantic enhancement vectors. The semantically enhanced vector set is processed by feature compression and dimension preservation to output a fused representation vector.

5. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, The improved AlignXpert algorithm is used to perform cross-modal semantic projection optimization and higher-order semantic completion on the fused representation vector, specifically as follows: Multi-head attention computation is performed on the financial modality subspace, structural modality subspace, and semantic modality subspace retained in the fused representation vector to generate cross-modal attention coefficients between the three types of modality subspaces. Based on these coefficients, a weighted combination of the three types of modality subvectors is performed to generate an attention weighted vector. The fused representation vector and the attention weighted vector are respectively input into a one-dimensional convolutional network, and sliding convolution is performed to obtain the residual information vector. The residual information vector is then input into a non-linear activation function to enhance the difference in feature expression, forming a residual completion vector. The attention weighted vector and the residual completion vector are concatenated along the feature dimension to form a joint representation vector. The joint representation vector is then input into multiple semantic mapping networks, each of which contains a fully connected layer and a non-linear activation function. For the semantic subspace vectors output by each semantic mapping network, calculate the KL divergence between the semantic subspace vectors, construct a semantic consistency supervision loss function, and optimize the parameters of each semantic mapping network through the backpropagation mechanism; Each semantic subspace vector is input into the compression mapping structure, and feature dimension normalization, linear transformation and semantic fusion operations are performed in sequence. Finally, the fused representation is generated by combining the high-order feature information retained in the residual completion vector.

6. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, The construction of the time-cause-effect graph centered on the risk identification object entity is specifically as follows: Based on the final fusion representation, the historical behavior trajectory and related event content of each risk identification object entity are identified, and a node set is constructed. The node types include historical state nodes and risk event nodes. Based on the time sequence and logical triggering relationship of historical state nodes and risk event nodes, a set of directed edges is constructed, and each edge is assigned a timestamp. Based on the high-order semantic clues and historical association information retained in the final fused representation, causal weight parameters are calculated for each edge; An initial temporal causal graph is constructed based on the set of nodes and the set of directed edges. Then, time window constraints, edge weight normalization, and loop path elimination operations are performed in sequence to normalize the graph structure. Using the risk-identified entity as the anchor node of the time-cause graph, the corresponding final fusion representation index information is labeled to form a time-cause graph centered on the risk-identified entity.

7. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, S6 specifically includes: The temporal causal graph is input into the temporal causal graph neural network model. Combining the final fused representation index information and timestamp information of each node, an initial state representation vector is generated for each node through the node feature embedding initialization method. In the temporal causal graph, a time decay factor is constructed based on the timestamp difference marked on the edge, and this factor is multiplied by the causal weight parameter as the edge weight value for information propagation. This guides the state representation vector of the historical state node to perform weighted propagation to the risk event node, updating the temporal context dependency structure between nodes. At each risk event node, a multi-head attention mechanism is used to normalize the propagation feature representations from different historical paths, and a dynamic gating strategy is introduced to aggregate the path features with weights. The influence strength of the path is calculated based on the product of the aggregate weight and the node prediction probability. The dominant causal path is selected and a causal chain representation is formed. At the same time, the state representation vector of the risk event node is updated. The final node state representation vector of each risk identification object entity after propagation and aggregation in the time causal graph is concatenated with the corresponding causal chain representation, and input into the risk level classification prediction branch and the trend regression prediction branch respectively, and the risk level label and risk evolution trend curve are output. By matching the path sequence in the causal chain representation with the predicted risk level of each node, the critical path with the greatest risk impact and the risk evolution stage are identified, generating structured key causal chains and risk path explanation information.

8. The method for identifying technology finance risks based on multimodal data fusion according to claim 1, characterized in that, The continuous updating based on multimodal data employs an incremental graph structure construction method to expand the temporal causal graph, and performs model parameter fine-tuning and adaptive structural optimization to update the risk identification capability of the temporal causal graph neural network model. Specifically: Based on the newly accessed multimodal data samples, corresponding financial vectors, structural vectors, and semantic vectors are constructed, and preliminary fusion, cross-modal collaborative enhancement, and projection optimization operations are performed in sequence to generate a new final fusion representation; The new final fusion representation is mapped to the corresponding risk identification object entity in the time causal graph. Based on its timestamp information and semantic content features, the corresponding historical state nodes and risk event nodes are incrementally added to the graph, and causal edge connections with the relevant nodes in the original graph are constructed to form the updated time causal graph. Normalization is performed on newly added nodes and edges, their state representation vectors and causal edge weight parameters are calculated, and a local propagation mechanism is adopted to perform information propagation and embedding updates only within the neighborhood associated with the newly added structure. Using newly added subgraphs in the temporal causal graph as the source of supervision signals, local parameter fine-tuning and structural adaptive optimization are performed on the temporal causal graph neural network model. A joint loss function based on gradient smoothing and structural consistency is adopted to improve the model's ability to adapt to new semantic and behavioral patterns. The updated temporal causal graph neural network model, together with the expanded temporal causal graph, forms the basis of a new risk identification system, supporting continuous output of risk level, evolution trend and key causal chain, and forming a closed-loop feedback path.

9. A technology finance risk identification system based on multimodal data fusion, comprising the technology finance risk identification method based on multimodal data fusion as described in any one of claims 1 to 8, characterized in that, Includes the following modules: The data processing module is used to collect multi-source heterogeneous data, perform missing data completion, format standardization and modality identification processing, and construct a multimodal dataset; The feature fusion module is used to generate financial vectors, structural vectors, and semantic vectors from financial data, structured data, and text semantic data in the multimodal dataset, respectively, and then performs modal space alignment, scale normalization, cross-modal semantic association modeling, and collaborative enhancement in sequence to generate a fused representation vector. The semantic projection optimization module is used to perform cross-modal semantic projection optimization and high-order semantic completion on the fused representation vector to generate the final fused representation. The causal graph construction module is used to construct a time-based causal graph centered on the risk identification object entity based on the final fusion representation. The graph includes historical state nodes and risk event nodes, and timestamp information and causal weight parameters are set for each edge. The risk identification module is used to input the time causal graph into the time causal graph neural network model and output the risk level prediction results, risk evolution trend and key causal chain of the risk identification object; The model update module is used for continuous updates based on multimodal data. It expands the time causal graph by incrementally constructing a graph structure and performs fine-tuning of model parameters and adaptive optimization of structure to update the risk identification capability of the time causal graph neural network model.

Images