A method for intelligent multimodal fusion and uncertainty-enhancing reasoning based on knowledge dissemination in large-scale model domain graphs.

Abstract

This application discloses an intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation, belonging to the field of CAE simulation technology. It solves the problem of existing technologies lacking intelligent fusion and reasoning of CAE simulation knowledge graphs with interpretable and measurable uncertainty. The method includes: performing heterogeneous data mapping and defining cross-modal relationships for heterogeneous data of different modalities in the CAE simulation domain to construct a multimodal fusion large-scale model domain graph; constructing a multimodal kernel function oriented towards knowledge propagation based on the large-scale model domain graph; dynamically adjusting the relation weights in the large-scale model domain graph by optimizing the kernel weight vector of the multimodal kernel function; and enhancing uncertainty reasoning for the knowledge graph completion task of the large-scale model domain graph.

Description

Technical Field

[0001] This application relates to the field of CAE simulation technology, and in particular to an intelligent multimodal fusion and uncertainty reasoning enhancement method for large model domain graphs based on knowledge propagation. Background Technology

[0002] Computer-aided engineering (CAE) simulation technology, as a core tool for modern industrial design and optimization, has been widely applied in high-end equipment manufacturing fields such as aerospace, automotive manufacturing, and energy equipment. With the development of Industry 4.0 and digital twin technology, CAE simulation is evolving from traditional single-physics analysis to complex system simulation involving multi-physics coupling and multi-scale collaboration. In this evolution, how to effectively integrate multi-source heterogeneous data and construct an intelligent knowledge system has become a current research hotspot.

[0003] In the construction of industrial knowledge graphs, deep learning-based natural language processing technologies have made significant progress in recent years. Large language models, exemplified by the Transformer architecture, have demonstrated powerful capabilities in areas such as technical document parsing and patent document mining, automatically extracting technical terms and their potential relationships from unstructured text. However, the application of these general-purpose models in specialized engineering fields still faces challenges: on the one hand, engineering terminology possesses strict domain-specific semantics, making it difficult for general semantic representations to accurately capture; on the other hand, engineering knowledge has a rigorous logical system, requiring deep integration with domain ontology.

[0004] Multimodal data fusion technology, as another important research direction, has gained widespread attention in the field of intelligent manufacturing. Existing methods mainly employ feature-level fusion or decision-level fusion strategies to correlate and analyze CAD model data, sensor monitoring data, and textual description information. Among these, neural network-based methods have shown advantages in representing heterogeneous data relationships, but they still have limitations in handling complex scenarios such as cross-modal semantic alignment and dynamic relationship optimization. Particularly in the field of engineering simulation, how to establish a dynamic mapping relationship between simulation parameters, experimental data, and domain knowledge remains a problem that needs to be solved.

[0005] In the area of ​​uncertainty quantification, Bayesian inference and probabilistic graphical models provide the theoretical foundation for knowledge reasoning. These methods describe the uncertainty of a system by introducing probability distributions, but they face challenges such as high computational complexity and poor real-time performance when processing large-scale industrial data. In recent years, deep learning-based approximate reasoning methods have made some progress, but improving computational efficiency while maintaining inference accuracy is still under continuous development and optimization.

[0006] The industry is currently actively exploring the deep integration of knowledge graphs and CAE simulation. Related research has begun to explore applying domain knowledge graphs to scenarios such as simulation parameter optimization and fault diagnosis. However, existing systems still have significant shortcomings in knowledge representation, relation modeling, and reasoning mechanisms, making it difficult to meet the comprehensive requirements of accuracy and interpretability in complex industrial scenarios.

[0007] Currently, the urgent problems to be solved include the following aspects:

[0008] (1) Current technology systems generally adopt an independent analysis paradigm when processing multimodal information such as text, 3D models, and sensor time-series data. This fragmented analysis method makes it difficult to effectively capture and utilize the inherent correlation between different modal data, and fails to construct a complete cross-modal knowledge representation system. Due to the lack of a unified data modeling framework, it is difficult for different modal data to form a synergistic effect, which restricts the performance capability of knowledge graphs in complex application scenarios.

[0009] (2) Existing methods rely mainly on manual experience or simple rule-matching mechanisms to define the interaction relationships between different modal data, lacking a systematic quantitative calculation model. This approach cannot accurately depict the complex relationships between text descriptions and 3D component features, sensor data and structural changes, resulting in a lack of adaptability and scalability in cross-modal relationship modeling. In dynamically changing industrial environments, this vague relationship definition method is difficult to meet the needs of precise analysis.

[0010] (3) Existing knowledge graph systems, after integrating multimodal data, have failed to establish an effective contribution evaluation mechanism, resulting in unclear weights of the influence of different modalities on the reasoning results. Furthermore, the systems lack the ability to quantitatively assess the uncertainty of prediction results, making it difficult to guarantee the credibility of the decision support process. This lack of interpretability restricts the application value of knowledge graphs in key decision-making scenarios. Summary of the Invention

[0011] Based on the above analysis, the embodiments of this application aim to provide an intelligent multimodal fusion and uncertainty reasoning enhancement method for large model domain graphs based on knowledge propagation, in order to solve the problem that the existing technology lacks intelligent fusion and reasoning of CAE simulation knowledge graphs with interpretable and measurable uncertainty.

[0012] This application discloses an intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation, the method comprising: Heterogeneous data mapping and cross-modal relationship definition are performed on heterogeneous data of different modalities in the CAE simulation field to construct a large model domain map of multimodal fusion; Based on the large model domain graph, a multimodal kernel function for knowledge dissemination is constructed; and the relation weights in the large model domain graph are dynamically adjusted by optimizing the kernel weight vector of the multimodal kernel function. Enhanced uncertainty reasoning is implemented for knowledge graph completion tasks in large-scale domain graphs.

[0013] Based on the above solution, this application also makes the following improvements: Furthermore, the construction of the large model domain graph through multimodal fusion is performed by: Based on the node sets obtained by mapping heterogeneous data of various modes in the CAE simulation field, a graph-structured node set is constructed. Construct edge sets of graph structures based on the relationships between heterogeneous data for each modality; The interaction between heterogeneous data of different modalities is quantized using three-dimensional relation tensors to construct cross-modal relations in graph structure; Based on the constructed graph structure, a corresponding multimodal fusion large model domain graph is generated.

[0014] Furthermore, node set It consists of three parts: (1) Represents a set of text term nodes. Represents a set of three-dimensional component nodes. This represents the set of sensor time-series data nodes.

[0015] Furthermore, the set of edges It consists of three parts: (2) in, Indicates the semantic relationships between text terms. This indicates the connection relationships between three-dimensional components. This indicates the temporal correlation between sensor time-series data.

[0016] Furthermore, cross-modal relationships among heterogeneous data of different modalities include descriptive relationships, inclusion relationships, adjacency relationships, synchronization relationships, causal relationships, and equivalence relationships.

[0017] Furthermore, the dynamic adjustment of relation weights in the large model's domain graph is performed as follows: Based on the aforementioned large model domain graph, a multimodal kernel function oriented towards knowledge dissemination is constructed; A conditional probability-based kernel weight parameter optimization method is used to optimize the kernel weight vector of the multimodal kernel function; Based on the kernel weight vector of the optimized multimodal kernel function, the relation weights in the domain graph of the large model are dynamically adjusted.

[0018] Furthermore, the multimodal kernel function for knowledge dissemination is a linear combination of three sub-kernels, expressed as: (3) in, Represents an individual and Overall similarity Represents an individual and In the Similarity across individual kernel dimensions; Indicates the first Weight parameters for each sub-kernel dimension.

[0019] Furthermore, the multimodal kernel function includes three sub-kernels: the semantic similarity kernel, the geometric matching kernel, and the temporal correlation kernel.

[0020] Furthermore, by maximizing the logarithmic function For the kernel weight vector Optimize: (4) in, For observation labels, Represents sample data; It is composed of multimodal kernel functions The constructed Gram matrix, Indicates the noise variance. Represents the identity matrix. For constant terms; This is a transpose.

[0021] The uncertainty reasoning enhancement is performed by: Based on the knowledge graph completion task of the large model domain graph, obtain the corresponding test data and training data, and calculate the corresponding covariance matrix; For the relationship to be evaluated, confidence is calculated based on the covariance matrix, and uncertainty reasoning is enhanced based on the confidence calculation results.

[0022] Compared with the prior art, this application can achieve at least one of the following beneficial effects: The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation proposed in this application has the following beneficial effects: (1) A heterogeneous data fusion framework based on a unified graph structure is proposed, which innovatively maps textual semantic features, three-dimensional geometric features, and temporal dynamic features into a node system in the same graph structure. By designing a multi-dimensional edge relationship definition mechanism, a systematic network of connections between different modal data is established. This method breaks through the limitations of traditional single-modal analysis and realizes the organic unification and collaborative expression of cross-modal data.

[0023] (2) A multimodal kernel function calculation system is constructed, organically integrating three core features: semantic similarity, geometric matching degree, and temporal correlation. By introducing the optimization objective of maximizing marginal likelihood, adaptive weight allocation of each modal kernel function is achieved. This optimization process simultaneously considers data fitting accuracy and model complexity control, ensuring the interpretability and generalization ability of the knowledge dissemination process.

[0024] (3) A confidence assessment system based on posterior covariance analysis was proposed, and a complete mechanism for quantifying prediction uncertainty was established. Through determinant analysis of the covariance matrix, the reliability of the inference results was assessed, and a standardized confidence scoring standard was established. This uncertainty quantification capability provides a reference for the reliability verification of knowledge graphs.

[0025] In this application, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this application will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing this application. The objectives and other advantages of this application can be realized and obtained from the specific points highlighted in the description and accompanying drawings. Attached Figure Description

[0026] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Throughout the drawings, the same reference numerals denote the same parts. Figure 1 A flowchart of the intelligent multimodal fusion and uncertainty reasoning enhancement method for large model domain graphs based on knowledge propagation provided in the embodiments of this application. Detailed Implementation

[0027] The preferred embodiments of this application are described in detail below with reference to the accompanying drawings, which constitute a part of this application and are used together with the embodiments of this application to illustrate the principles of this application, but are not intended to limit the scope of this application.

[0028] One specific embodiment of this application discloses an intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation. The flowchart of this method is as follows: Figure 1 As shown.

[0029] Step S1: Perform heterogeneous data mapping and cross-modal relationship definition for heterogeneous data of different modalities in the CAE simulation field, and construct a large model domain map of multimodal fusion.

[0030] Step S11: Construct a graph-structured node set based on the node set obtained by mapping heterogeneous data of various modes in the CAE simulation field.

[0031] In this embodiment, the mapping process for heterogeneous data first requires establishing a unified graph structure. .

[0032] Node set of graph structure It consists of three parts: (1) This represents a set of text term nodes. These nodes can be encoded using a specific model, based on a large model architecture, through a multi-layered self-attention mechanism to transform each term into a dense vector representation with several dimensions. This encoding method not only preserves the semantic information of the terms but also captures context-dependent syntactic features.

[0033] This represents a set of 3D component nodes. 3D component nodes can extract various types of key features, including geometric features (such as parameters such as volume and curvature) and topological features (such as graph theory indices such as connectivity and number of rings). These indices can effectively describe the assembly relationships between components.

[0034] The sensor time series data node set can be processed using time-frequency analysis methods. For example, FFT coefficients can be used to reflect the periodicity of the signal, or wavelet energy can be used to capture transient changes. The combination of these features serves as the basis for subsequent time series relationship analysis.

[0035] Step S12: Construct the edge set of the graph structure based on the relationship between the heterogeneous data of each modality.

[0036] set of edges It consists of three parts: (2) in, Indicates the semantic relationships between text terms. This indicates the connection relationships between three-dimensional components (or structural components). This indicates the temporal correlation between sensor time-series data.

[0037] In this embodiment, this multi-dimensional edge definition method enables a complete modeling of the relationships between data from different modalities. In real-world industrial scenarios, this unified representation method can organically integrate fault descriptions from maintenance documents, component structures from CAD drawings, and operational status data collected by sensors into a single graph structure, facilitating subsequent analysis and processing.

[0038] Step S13: Use three-dimensional relation tensors to quantize the interaction between heterogeneous data of different modalities and construct cross-modal relations of graph structure.

[0039] In this embodiment, a three-dimensional relation tensor is constructed. To define cross-modal relationships between heterogeneous data of different modalities, heterogeneous data of different modalities Cross-modal relationships Represented as: (3) The above six components correspond to different types of cross-modal relationships.

[0040] This method describes the correspondence between text terms and 3D components. Its calculation combines cosine similarity and cross-modal attention mechanisms. Specifically, it first calculates the cosine similarity between the text vector and the CAD feature vector as a basic metric. Then, it weights important features using a query-key attention mechanism. Finally, it uses a gated neural network to synthesize this information and generate the final relationship weights. This multi-layered calculation method ensures the accuracy of the relationship description.

[0041] This method represents inclusion relationships, used to describe the hierarchical assembly relationships between 3D components. Its calculation can be based on bounding box detection and hierarchical structure analysis. Specifically, the intersection-union ratio (CUI) of the component bounding boxes can be calculated first as an initial estimate, then corrected by combining predefined part_of relationships, and finally iteratively optimized using a graph neural network. This combined approach integrates objective geometric calculations with domain knowledge, making the determination of inclusion relationships more reliable.

[0042] This approach represents adjacency relationships and quantifies the spatial proximity between three-dimensional components. Its calculation is based on a minimum distance metric and contact area analysis. Specifically, the minimum Euclidean distance between points on the component surfaces is first calculated, and then converted into a similarity value using an exponential function. For components with actual contact, an additional weighting coefficient is added to reflect the importance of this physical connection. This approach can identify component pairs that may experience contact wear in mechanical fault analysis.

[0043] This indicates synchronization relationships and is used for temporal correlation analysis between sensor time-series data. Specifically, a generalized cross-correlation method can be used to calculate the time delay and similarity between signals, while combining phase lock values ​​to evaluate signal synchronicity. This method can detect synchronization phenomena and phase coupling relationships. For example, in the vibration analysis of rotating machinery, this method can be used to identify sensor groups with synchronous vibration characteristics, providing a basis for fault diagnosis.

[0044] Determining causal relationships requires a comprehensive application of statistical methods. Specifically, causality tests can be used first to assess the predictive power between time-series signals, then transition entropy can be calculated to quantify information flow, and finally, Bayesian network learning can be used to determine the direction of causality. This approach can distinguish between true causal relationships and simple time-series correlations, playing a role in tracing equipment failures.

[0045] This represents equivalence relations and is used to identify nodes representing the same entity in different modalities of data. Specifically, it can be determined using methods such as ontology reasoning, name similarity calculation, and functional equivalence verification. Functional equivalence verification can employ FMEA (Failure Mode and Effects Analysis) to confirm functional consistency by comparing the performance of nodes under different failure scenarios. This method can be applied to the construction of equipment maintenance knowledge graphs, effectively eliminating duplicate nodes and improving graph quality.

[0046] Step S14: Generate the corresponding multimodal fusion large model domain graph based on the constructed graph structure.

[0047] In the specific implementation process, the nodes in the large model domain graph are determined based on the node set of the graph structure, and the relationships between the nodes in the large model domain graph are determined based on the edge set and cross-modal relationships of the graph structure, thereby generating the corresponding multimodal fusion large model domain graph.

[0048] Step S2: Based on the large model domain graph, construct a multimodal kernel function for knowledge dissemination; and dynamically adjust the relation weights in the large model domain graph by optimizing the kernel weight vector of the multimodal kernel function.

[0049] Step S21: Based on the large model domain graph, construct a multimodal kernel function for knowledge dissemination.

[0050] First, by constructing a multimodal kernel function oriented towards knowledge propagation, the similarity between nodes in the domain graph of the large model is defined. The multimodal kernel function oriented towards knowledge propagation is a linear combination of three sub-kernels, expressed as: (4) in, Represents an individual and Overall similarity Represents an individual and In the Similarity across individual kernel dimensions; Indicates the first The weight parameters for each sub-kernel dimension are used to control the contribution of the corresponding sub-kernel and are obtained through optimization learning.

[0051] Below, we present the specific forms of the three sub-kernel functions.

[0052] (1) Semantic similarity kernel The semantic similarity kernel is based on the cosine similarity calculation of text features and is suitable for semantic matching of concepts or attributes in an ontology.

[0053] when hour, Its definition is: (5) in, This kernel represents a vectorized representation of text (such as word embeddings or ontology concept embeddings). It can capture the semantic relevance of text attributes such as names and descriptions.

[0054] (2) Geometric matching kernel

[0055] If individuals are associated with geometric data (such as 3D models), the kernel measures shape similarity by the inverse of the Hausdorff distance.

[0056] when hour, Its definition is: (6) in, Represents an individual , Corresponding point set , The Hausdorff distance reflects the differences in geometry. Its reciprocal transformation ensures that similarity monotonically increases as the distance decreases.

[0057] (3) Time-domain correlation kernel

[0058] For time-series data (such as sensor readings), the negative exponential form of the Dynamic Time Warping (DTW) distance is used.

[0059] when hour, Its definition is: (6) in, Individuals , Time data, Indicates unequal-length time series data alignment ; The scaling parameter controls the similarity decay rate. This kernel is suitable for predicting time-dependent attributes.

[0060] Step S22: Optimize the kernel weight vector of the multimodal kernel function using a kernel weight parameter optimization method based on conditional probability.

[0061] In this embodiment, by maximizing the logarithmic function For the kernel weight vector Optimize: (7) in, For observation labels, Represents sample data; It is composed of multimodal kernel functions The constructed Gram matrix, This represents the noise variance (used to handle annotation uncertainty). Represents the identity matrix. For constant terms; This is the transpose. The first term in formula (7) is used to measure the data fit, and the second term is used to penalize model complexity (to avoid overfitting). Specifically, the parameters are explained below.

[0062] Kernel weight vector This represents the combined weights of different kernel functions, used to adjust the weights of each kernel in a multimodal kernel function. The contribution of the three sub-cores is considered. In the simulation, different cores may correspond to different physical relationships (such as linear, nonlinear, or periodic dependence), and optimization is required. The model's ability to fit different relationships can be adaptively adjusted through constraints. The gradient ascent method is used for optimization to ensure the physical interpretability of the kernel combination, while avoiding model instability caused by negative weights.

[0063] Observation tags For simulation or experimental observation data (such as stress, displacement, temperature, etc.), it represents the true physical response to be fitted. In simulation software, The data may originate from finite element analysis results or sensor measurements, and its accuracy directly affects the optimization direction of the kernel weights. If the data contains noise (such as measurement errors or numerical discretization errors), it is necessary to analyze the noise variance. Perform robust modeling.

[0064] matrix It is composed of multimodal kernel functions The elements of the constructed symmetric positive definite matrix are... Used to measure samples With sample Similarity. Kernel function The construction of the kernel function must meet certain conditions to ensure its validity in the Hilbert space, thereby supporting subsequent matrix inversion operations.

[0065] noise variance This represents the noise variance in the observed data, used to model annotation uncertainties (such as experimental errors or numerical noise). In simulations, noise variance... The introduction of [aspect name] can alleviate overfitting problems, for example, when the finite element mesh is coarse, the discretization error may lead to [problem]. Deviation from the true solution. This can be addressed by adjusting the noise variance. The model can balance the goodness of fit and generalization ability to the data, but has a large noise variance. It will reduce sensitivity to noisy data.

[0066] identity matrix As a regularization term, it ensures that the matrix Reversibility avoids numerical instability issues. In simulation optimization, the identity matrix... The dimension is consistent with the number of samples, and the same noise hypothesis is assigned to each data point.

[0067] Therefore, the meanings represented by the three terms are explained as follows.

[0068] Data fitting term Used to measure the model's responsiveness to observed data. The degree of fit is denoted by , with a smaller value indicating a lower fitting error. Its calculation relies on the inverse operation of the kernel matrix, reflecting the multimodal kernel function's ability to model the data distribution. In simulation scenarios, optimizing this term can enable the model to more accurately predict key physical quantities (such as critical stress or thermal deformation).

[0069] Complexity penalty This approach penalizes model complexity through matrix determinant calculations, preventing overfitting of kernel combinations to noise or local features. The determinant value varies with the kernel weight vector. As the kernel size increases, the optimization process spontaneously suppresses the contribution of redundant kernels. In multiphysics coupling analysis, this term prevents the model from over-relying on a single kernel function (such as fitting all relationships with only a Gaussian kernel), thus improving generalization.

[0070] constant term It is a constant independent of the parameters and can be ignored during the optimization process, without affecting the direction of weight update.

[0071] The optimized solution process is as follows: Initialize kernel weight vector : Set initial values ​​based on prior knowledge (such as material constitutive relations).

[0072] Calculate the Gram matrix Based on the current situation Sample data Construct the multimodal kernel function and its corresponding Gram matrix.

[0073] Gradient ascent optimization: for about The derivative is calculated, and the weights are iteratively updated using an adaptive learning rate until the marginal likelihood converges.

[0074] Step S23: Dynamically adjust the relation weights in the large model neighborhood graph based on the kernel weight vector of the optimized multimodal kernel function.

[0075] In the specific implementation process, the kernel weight vector of the optimized multimodal kernel function is embedded into the simulation solver, and the simulation solver uses the optimized kernel weight vector... Used to dynamically adjust the weights of knowledge graph relationships to guide subsequent simulation calculations (such as mesh adaptation or parameter calibration).

[0076] Step S3: For the knowledge graph completion task of the large model domain graph, perform uncertainty reasoning enhancement.

[0077] In the specific implementation process, the function of step S3 is implemented based on the uncertainty reasoning engine, as described in detail below.

[0078] Step S31: Based on the knowledge graph completion task of the large model domain graph, obtain the corresponding test data and training data, and calculate the corresponding covariance matrix.

[0079] First, calculate the kernel matrix based on the training data. And test-train covariance matrix Then, the matrix inversion formula is used to calculate... This step should employ appropriate methods to improve numerical stability. Finally, the posterior covariance is obtained through matrix multiplication. The diagonal elements of this matrix represent the independent prediction variances of each test point, while the off-diagonal elements reflect the correlation between predictions of different test points.

[0080] Step S32: For the relationship to be evaluated, the confidence level is calculated based on the covariance matrix, and uncertainty reasoning is enhanced based on the confidence level calculation results.

[0081] For each relationship to be evaluated Extract the corresponding submatrix (usually a) The matrix involves the predicted covariance of two entities. Then the determinant is calculated. The larger this value, the more uncertain the joint prediction of the two entities is (e.g., possibly due to a lack of similar relation instances in the training data). Finally, the confidence score is standardized by global normalization (divided by the largest determinant value) to ensure comparability between different relations.

[0082] This method, applied to knowledge graph completion tasks, prioritizes high-confidence predictions while triggering manual review or further data collection for low-confidence predictions. This enhances uncertainty reasoning based on confidence calculations. During dynamic knowledge base updates, changes in confidence levels can be monitored in real time; for example, the confidence level can be recalculated after new data is added. We then observe whether the original low-confidence predictions have improved. Using the methods described above, the uncertainty inference engine can provide prediction results along with a reliability assessment, making the knowledge completion process more transparent and controllable.

[0083] Below, the specific details of the two steps are given respectively.

[0084] (1) Analysis of covariance matrix In an uncertainty inference engine, the covariance matrix Used to quantify the confidence level of prediction results. This matrix is ​​calculated using the following model: (8) This represents the prior covariance matrix between test data points, with dimension 1. ( (where is the number of test samples), representing the inherent similarity between test points when there is no observation data. It is the covariance matrix between the test data and the training data, with dimensions of . ( (This is the number of training samples), used to measure the relationship between test points and known data. It is the covariance matrix between the training data, with dimension 1. It uses a certain type of kernel function to calculate and reflect the similarity structure of the training samples. It is the variance of the observed noise, used to adjust the robustness of the model to noise in the training data. The larger the value, the higher the tolerance of the model to noise. It is the identity matrix, used to ensure the matrix It is reversible, thus avoiding numerical calculation problems.

[0085] In the above method, It measures the uncertainty of predicting test data given the training data. Specifically, This represents the initial uncertainty of the test data, while This indicates the reduction in uncertainty due to the introduction of training data. Therefore, The smaller the value, the higher the reliability of the prediction.

[0086] (2) Calculation of confidence index To transform the uncertainty of the covariance matrix into an interpretable confidence index, this embodiment provides the following settings: (9) in, It is the covariance matrix submatrix The determinant of a matrix is ​​used to measure the uncertainty of a local forecast. The larger the determinant, the less reliable the forecast for that region. It is the maximum value of the determinant of all possible submatrices, used to normalize the confidence index so that it falls within the range of 1 / 2. Within the range. It is the final confidence index, and the closer its value is to... This indicates that the pre- The higher the credibility, the less credible the result; conversely, the closer the result is to the truth. This indicates that the greater the uncertainty in the prediction.

[0087] This confidence metric can increase the reliability of quantitative predictions because, in knowledge graph completion tasks, the prediction of certain relationships may be unreliable due to data sparsity. This can be reflected intuitively. In addition, it can guide active learning by marking low-confidence predictions as targets that require manual review or additional data collection, thereby improving the efficiency of model iterative optimization.

[0088] To help those skilled in the art better understand the implementation process of this application, specific implementation examples of the above method are also provided in Specific Embodiment 2 of this application, as detailed in the following description.

[0089] First, a multimodal graph is constructed. This embodiment proposes a method for heterogeneous data mapping and cross-modal relationship definition based on a unified graph structure. A specific implementation process is as follows.

[0090] First, in the heterogeneous data mapping stage, a unified graph structure needs to be constructed. Node set From text term nodes 3D component nodes and sensor timing data nodes Composition. For text terminology nodes, pre-trained language models based on the Transformer architecture (such as BERT or GPT) can be used for encoding, transforming terms from maintenance documents and technical manuals into high-dimensional dense vectors while preserving semantic and contextual features. For 3D component nodes, key features are extracted from the geometric model of the simulation software, including geometric parameters (such as volume, surface area, curvature) and topological indices (such as connectivity, number of rings) to describe the structural characteristics of the component and its relationship within the assembly. For sensor time-series data nodes, time-frequency analysis methods (such as FFT, wavelet transform) are used to extract features, such as frequency domain energy distribution or transient response features, to characterize the equipment's operating status. A set of edges. Including textual relationships Structural Relationships and time sequence relationship They respectively modeled the semantic relationships between text terms, the assembly connections between 3D components, and the temporal correlations between sensor time series data.

[0091] In the cross-modal relation definition phase, using the three-dimensional relation tensor Quantify the interactions between different modalities of data. Describe the relationships. This is used to associate textual terms with 3D components, such as mapping the description of "bearing wear" in a maintenance document to a bearing component in a CAD model. During calculation, the text vector and CAD feature vector are first compared using cosine similarity. Then, a cross-modal attention mechanism is introduced to weight key features, and finally, a gated neural network outputs the relation weights. (Inclusion relationship) Based on bounding box detection and hierarchical analysis, for example, determining that a bolt belongs to a sub-assembly of a flange component can be corrected by combining the intersection-over-union (IoU) ratio of bounding boxes with domain knowledge (such as the Bill of Materials). Adjacency relationships... By calculating the minimum Euclidean distance and contact area between three-dimensional components, component pairs that may experience interference or wear, such as the mating surfaces of gears and shafts, can be identified. Synchronization relationships. Used to analyze the temporal correlation between sensor time-series data, for example, in multibody dynamics simulations, it identifies synchronous vibration signals through generalized cross-correlation and phase-locked values, aiding in fault location. Causality By combining causality tests and transfer entropy, we can analyze fault propagation paths, such as how bearing failure leads to increased shaft vibration. Equivalence relations. By employing ontology reasoning and functional consistency verification, the same entity across different data sources can be identified. For example, "Node 12" in a simulation model and "Measurement Point A" in sensor data may point to the same physical location. In practical implementation, a multi-level relation weight calculation model can be developed, integrating attention mechanisms with domain knowledge reasoning to achieve deep correlation calculation between textual semantics and 3D geometric features. This allows for the establishment of a fault propagation model based on statistical analysis and causal reasoning, and the proposal of an entity matching method for functional consistency verification. This dynamic quantification mechanism can adaptively adjust relation strength to meet the precise analysis needs of different application scenarios.

[0092] In simulation software, the method of constructing multimodal fusion graph structures can be integrated into the simulation post-processing module. For example, in structural strength analysis, the system automatically correlates simulation results (such as stress contour maps), CAD components (such as critical load-bearing members), and monitoring data (such as strain gauge readings) to locate high-risk areas through cross-modal relationships. In fault diagnosis scenarios, users input fault descriptions (such as "abnormal noise"), and the system recommends possible failed components (such as broken gear teeth) and associated abnormal sensor signals based on descriptive and causal relationships, assisting engineers in making quick decisions.

[0093] Next, the construction and parameter optimization of multimodal kernel functions for knowledge propagation are performed. This method for constructing and optimizing multimodal kernel functions for knowledge propagation is particularly suitable for multiphysics coupling analysis and knowledge graph construction in simulation software. A specific implementation method is given below.

[0094] (1) Construction and initialization of multimodal kernel function In a simulation environment, the first step is to construct a multimodal data framework based on the characteristics of the simulation object. Taking structure-thermal coupling analysis as an example, the textual attributes of the simulation object (such as material names and boundary condition descriptions) are converted into vector representations using a pre-trained word embedding model (such as Word2Vec or BERT) to form a semantic similarity kernel. The input is a geometric model (such as a CAD file) that is discretized to generate point cloud data, which is used to calculate the Hausdorff distance and construct the geometric matching kernel. Sensor time-series data (such as temperature field monitoring results) are aligned using Dynamic Time Warping (DTW) before being input into the time-domain correlation kernel. Initial kernel weight vector It can be set according to domain experience, for example, giving higher weight to the geometric kernel in structural analysis, while focusing on the time-domain kernel in heat conduction problems.

[0095] (2) Kernel matrix calculation and simulation data integration In simulation software, simulation results (such as stress-strain fields and temperature distributions) are automatically extracted as observation labels through plug-in modules. And construct a multimodal kernel function matrix. For example, in automobile chassis fatigue analysis, The nodal stress values ​​calculated by finite element method Each element Samples are calculated using a multimodal kernel function. Multimodal similarity. Noise variance. Based on the joint calibration of grid discretization error and experimental measurement error, it is usually set as follows: variance To balance fitting accuracy and numerical stability.

[0096] (3) Gradient Ascent Optimization and Parameter Adaptation Optimize the kernel weight vector using gradient ascent based on automatic differentiation (such as PyTorch or TensorFlow backends). In each iteration, the marginal likelihood is calculated. For kernel weight vector The partial derivatives of the model are used, where the data fitting term drives the model to match key physical quantities in the simulation results (such as maximum von Mises stress), while the complexity penalty term suppresses redundant kernels (such as over-reliance on geometric kernels leading to thermal analysis bias). The optimization process embeds the simulation software's API interface to verify the prediction effect after weight updates in real time. For example, in battery thermal management simulation, when... When the weight is increased, the system automatically enhances its sensitivity to temperature time-series data and optimizes the design parameters of the cooling channel.

[0097] (4) Knowledge graph embedding and simulation decision support The optimized kernel weights are fed back to the CAE knowledge graph system to dynamically adjust the strength of entity relationships. Taking aero-engine blade simulation as an example, high weights in the semantic kernel trigger semantic retrieval from the material knowledge base, while a dominant geometric kernel automatically associates similar cases from the CAD model library. The optimization results of the temporal kernel are then used to correct the boundary conditions for transient analysis. Simultaneously, through... Noise Adaptation in Monitoring: Reducing Noise Variance When the Simulation Mesh is Refined To improve accuracy, increase the noise variance when processing experimental data. To enhance robustness, the final output multimodal kernel function model can be integrated into the parametric design module of the simulation software to guide subsequent mesh generation strategies or material constitutive model selection.

[0098] In the simulation field, the specific implementation method for constructing an uncertainty inference engine can be tailored to the uncertainty quantification requirements in engineering simulations. Firstly, during the training phase, a covariance matrix needs to be constructed based on historical simulation data (such as finite element analysis results, material parameter experimental data, or multiphysics coupling simulation outputs). For example, in structural mechanics simulations, the training data may include stress-strain relationships, modal frequencies, or fatigue life prediction results under different working conditions, and the kernel matrix can be calculated using a Gaussian kernel function. This reflects the similarity between different simulation cases. The test data corresponds to newly designed simulation tasks, such as performance prediction of a novel composite material structure, and its test-training covariance matrix... The correlation between the new operating conditions and historical data is measured using the same kernel function.

[0099] To improve numerical stability, when calculating the inverse matrix... In such cases, Cholesky decomposition combined with regularization techniques can be used to avoid matrix ill-conditioned problems caused by highly linearly correlated parameters in the simulation data (such as similar mesh partitions or boundary conditions). Noise variance The value of can be adjusted based on the statistical characteristics of simulation errors, for example, by cross-validation, to balance the model's sensitivity to numerical noise (such as mesh discretization errors or convergence residuals). Posterior covariance matrix The calculation results will directly reflect the credibility of the new simulation prediction. For example, in fluid simulation, if the predicted covariance of the velocity field in a certain region is large, it may indicate that the mesh resolution of that region is insufficient or the applicability of the turbulence model is questionable.

[0100] During the confidence level calculation phase, for key relationships that need to be evaluated in the simulation software (such as the relationship between stress concentration at a node and deformation of adjacent elements), from... Extract the corresponding 2×2 submatrix Calculate its determinant For example, in thermo-mechanical coupling analysis, if the determinant value of the interface parameters of a certain group of materials is significantly higher than that of other regions, it indicates that the joint uncertainty of the thermal conductivity coefficient and stress prediction at that interface is high (possibly due to a lack of experimental data for that material combination). The confidence level can be increased through global normalization (e.g., dividing by the maximum determinant value of all material combinations). Transform into Standardized indicators.

[0101] In practical applications, simulation software can achieve the following functions based on confidence level indicators: (1) Result confidence labeling: Automatically mark low confidence areas in the simulation report (e.g. The peak stress value indicates that engineers should focus on checking the mesh generation or boundary conditions. (2) Adaptive optimization: When the confidence level is too low, trigger local mesh refinement or suggest supplementing experimental data for specific working conditions (such as linking co-simulation and physical testing). (3) Dynamic knowledge base update: Real-time recalculation after new experimental data is imported. It also monitors changes in existing low-confidence predictions. For example, if the confidence level of a certain alloy fatigue model increases with the addition of new high-temperature test data, the manual review flag can be automatically removed.

[0102] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0103] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A method for intelligent multimodal fusion and uncertainty reasoning enhancement based on knowledge dissemination in a large-scale model domain graph, characterized in that: The method includes: Heterogeneous data mapping and cross-modal relationship definition are performed on heterogeneous data of different modalities in the CAE simulation field to construct a large model domain map of multimodal fusion; Based on the large model domain graph, a multimodal kernel function for knowledge dissemination is constructed; and the relation weights in the large model domain graph are dynamically adjusted by optimizing the kernel weight vector of the multimodal kernel function. Enhanced uncertainty reasoning is implemented for knowledge graph completion tasks in large-scale domain graphs.

2. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation as described in claim 1, characterized in that, The construction of a large model domain graph through multimodal fusion is performed as follows: Based on the node sets obtained by mapping heterogeneous data of various modes in the CAE simulation field, a graph-structured node set is constructed. Construct edge sets of graph structures based on the relationships between heterogeneous data for each modality; The interaction between heterogeneous data of different modalities is quantized using three-dimensional relation tensors to construct cross-modal relations in graph structure; Based on the constructed graph structure, a corresponding multimodal fusion large model domain graph is generated.

3. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation as described in claim 2, is characterized in that... Node set It consists of three parts: （1） Represents a set of text term nodes. Represents a set of three-dimensional component nodes. This represents the set of sensor time-series data nodes.

4. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation as described in claim 3, is characterized in that... set of edges It consists of three parts: （2） in, Indicates the semantic relationships between text terms. This indicates the connection relationships between three-dimensional components. This indicates the temporal correlation between sensor time-series data.

5. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation according to claim 4, characterized in that, Cross-modal relationships between heterogeneous data of different modalities include descriptive relationships, inclusion relationships, adjacency relationships, synchronization relationships, causal relationships, and equivalence relationships.

6. The method for intelligent multimodal fusion and uncertainty reasoning enhancement of large model domain graph based on knowledge propagation according to any one of claims 1-5, characterized in that, The dynamic adjustment of relation weights in the large model's domain graph is performed by: Based on the aforementioned large model domain graph, a multimodal kernel function oriented towards knowledge dissemination is constructed; A conditional probability-based kernel weight parameter optimization method is used to optimize the kernel weight vector of the multimodal kernel function; Based on the kernel weight vector of the optimized multimodal kernel function, the relation weights in the domain graph of the large model are dynamically adjusted.

7. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation as described in claim 6, characterized in that, The multimodal kernel function for knowledge dissemination is a linear combination of three sub-kernels, expressed as: （3） in, Represents an individual and Overall similarity Represents an individual and In the Similarity across individual kernel dimensions; Indicates the first Weight parameters for each sub-kernel dimension.

8. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation according to claim 7, characterized in that, The multimodal kernel function includes three sub-kernels: semantic similarity kernel, geometric matching kernel, and temporal correlation kernel.

9. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation as described in claim 8, characterized in that, By maximizing the logarithmic function For the kernel weight vector Optimize: （4） in, For observation labels, Represents sample data; It is composed of multimodal kernel functions The constructed Gram matrix, Indicates the noise variance. Represents the identity matrix. For constant terms; This is a transpose.

10. The intelligent multimodal fusion and uncertainty reasoning enhancement method for large-scale model domain graphs based on knowledge propagation according to claim 9, characterized in that, The uncertainty reasoning enhancement is performed by: Based on the knowledge graph completion task of the large model domain graph, obtain the corresponding test data and training data, and calculate the corresponding covariance matrix; For the relationship to be evaluated, confidence is calculated based on the covariance matrix, and uncertainty reasoning is enhanced based on the confidence calculation results.

Images