A method for constructing enterprise-level simulation knowledge graph based on multimodal data integration

Abstract

The present invention relates to a method for constructing an enterprise-level simulation knowledge graph based on multimodal data integration, and belongs to the field of knowledge graph technology. The method comprises: integrating structured data, semi-structured data, and unstructured data through a multimodal data warehouse; extracting knowledge from semi-structured data and unstructured data to obtain entities and relationships, and performing knowledge fusion; storing the fused entities and relationships using a graph database to construct a simulation knowledge graph; embedding a vector database in combination with the simulation knowledge graph, and implementing semantic extension search through multimodal joint search, searching for similar cases through simulation result graphs, and automatically generating a simulation scheme comparison matrix. The method realizes the effective integration of multimodal data, improves the comprehensiveness and accuracy of knowledge graph construction, provides more powerful, efficient and intelligent support for the enterprise's simulation analysis, and can assist enterprises in making scientific decisions quickly in complex and changing business scenarios.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graph technology, and specifically relates to an enterprise-level simulation knowledge graph construction method based on multimodal data integration. Background Art

[0002] In modern manufacturing, CAE simulation technology has become a core component of product development. However, simulation data exhibits multimodal characteristics (structured parameters, semi-structured logs, and unstructured images / documents), comes from dispersed sources, and has heterogeneous formats. For example, while systems such as SLM and Ansys Minerva support standardized simulation processes, they lack the ability to dynamically integrate and semantically correlate multimodal data, leading to prominent data silos.

[0003] Enterprises need to manage massive amounts of data such as simulation models, experimental results, and optimization parameters at the same time. Traditional SPDM systems rely on hierarchical storage and static knowledge bases, making it difficult to achieve real-time cross-modal knowledge reasoning.

[0004] Multimodal data exhibits significant differences in physical meaning, format accuracy, and time series (for example, it's difficult to align the temporal features of 3D scan point clouds with those of equipment logs), making it difficult to ensure consistency after fusion. While existing data warehouse technologies can adapt to different data types, they lack a unified semantic alignment mechanism, resulting in inefficient cross-modal retrieval. For example, the semantic association between text descriptions and simulated images in industrial big data relies on manual annotation, which is costly and lacks scalability.

[0005] Traditional knowledge graph construction relies on single-modal data, and entity disambiguation and attribute fusion require manual intervention in rules. This makes it difficult to adapt to the dynamic updating needs of multi-source heterogeneous data in enterprise-level simulation scenarios. Incremental update algorithms are inefficient when processing large-scale dynamic data and cannot effectively handle complex semantic associations between cross-modal entities.

[0006] Existing search technologies lack deep integration with business processes and are unable to incorporate semantic extensions based on scene tags or device metadata, resulting in poor alignment between search results and business needs. Cross-modal correlations in simulation result graphs rely on manual experience, making it difficult to automatically generate a solution comparison matrix, hindering decision-making efficiency.

[0007] Currently, the enterprise-level simulation knowledge graph based on multimodal data integration still has the following areas for improvement:

[0008] 1. Semantic alignment of heterogeneous data sources (such as CAD models, sensor logs, and lab reports) relies on manual rules and lacks automated semantic annotation and cross-modal mapping mechanisms. For example, the "material fatigue strength" parameter in an unstructured document cannot be automatically associated with the "stress threshold" field in a structured database.

[0009] 2. Differences in the reference to the same entity in multi-source data (e.g., inconsistent naming of "motor model A" across different systems) lead to redundancy and errors in the knowledge graph. Existing disambiguation methods (rule-based or shallow machine learning) struggle with scenarios where short texts lack contextual features. Entity alignment requires a combination of domain synonym lists and dynamic context analysis, but predefined rules struggle to cover complex business scenarios, and attribute priority conflicts require manual intervention.

[0010] 3. Existing hybrid search solutions suffer from data synchronization delays and fragmented query scope, resulting in low recall rates for cross-modal correlation cases. The locality flaws of graph databases and the alignment flaws of vector databases jointly limit the performance of complex queries, especially in scenarios requiring joint metadata filtering and semantic expansion.

[0011] 4. Online updates to the knowledge graph rely on full reconstruction, making it impossible to achieve real-time integration of high-frequency incremental data. Existing incremental algorithms experience exponential computational complexity when processing hyperedge relationships, and lack lightweight models to support edge computing scenarios.

[0012] 5. The intent analysis of natural language queries relies on general NLP models and lacks domain knowledge enhancement (such as implicit constraints on device models), causing semantic expansion to deviate from business logic. In cross-modal retrieval, the mapping of text queries to simulated images relies on shallow feature matching, making it difficult to capture physical connections in business scenarios. Summary of the Invention

[0013] In order to solve the above problems existing in the prior art, the present invention provides an enterprise-level simulation knowledge graph construction method based on multimodal data integration;

[0014] The purpose of the present invention can be achieved through the following technical solutions:

[0015] S1: Integrate structured data, semi-structured data, and unstructured data through a multimodal data warehouse, standardize the structured data through parameterized templates, semantically annotate and associate the unstructured data through metadata models, and unify the semi-structured data into a standardized format.

[0016] S2: performing knowledge extraction on the semi-structured data and the unstructured data to obtain entities and relationships, and performing knowledge fusion on the entities and relationships in the structured data, the semi-structured data, and the unstructured data;

[0017] S3: Use the graph database to store the fused entities and relationships and build a simulation knowledge graph;

[0018] S4: Combine the simulation knowledge graph with the embedded vector database, implement semantic extension search through multimodal joint search, find similar cases through the simulation result graph, and automatically generate a simulation scheme comparison matrix.

[0019] Specifically, the multimodal data warehouse is used to achieve semantic alignment, efficient retrieval and cross-modal association of multi-source heterogeneous data; adapt to different data types through a hierarchical storage engine; define field constraints through parameterized templates in the structured data layer, and implement dynamic mapping in combination with data warehouse tools; convert log format data into columnar time series tables in the semi-structured data layer; convert image and document data into feature vectors in the unstructured data layer, and perform semantic annotation and association based on the metadata model.

[0020] Specifically, the metadata model extracts metadata of preset fields from the unstructured data to generate a label set; and trains the model through the label set of historical simulation data to identify metadata in the unstructured data.

[0021] Specifically, the metadata model automatically converts multi-source data into nodes and edges of a graph database by matching the semantic types of entities and relationships of the metadata dictionary with those of the metadata model.

[0022] Specifically, the metadata model and the vector database collaborate in the following ways: vector embedding and metadata are jointly stored, vectors of simulation result graphs are associated with scene labels and device model metadata, and hybrid retrieval is supported to narrow the search scope through metadata filtering.

[0023] Specifically, the knowledge extraction methods include: semantic alignment, entity relationship extraction, and cross-modal knowledge fusion;

[0024] The semantic alignment includes explicit alignment, implicit alignment, and semantic space mapping; the explicit alignment defines semantic type constraints through a metadata dictionary and establishes mapping rules between structured data fields and unstructured data labels; the implicit alignment adopts the cross-modal attention mechanism of the Transformer architecture to dynamically learn the association between multimodal data sub-components during training; the semantic space mapping maps text descriptions, time series data, and simulated images to a unified vector space.

[0025] The entity relationship extraction uses named entity recognition technology to extract entities from text and determines the relationship between entities with the help of dependency syntax analysis;

[0026] The cross-modal knowledge fusion associates entities and relationships in different modal data to generate a unified knowledge representation.

[0027] Specifically, the knowledge fusion includes: entity disambiguation, attribute fusion, and relationship fusion; the entity disambiguation is used to resolve the reference differences of the same entity in multi-source data, and to force the alignment of entity names in structured and unstructured data through a predefined domain synonym table; the attribute fusion is used to integrate multi-source attribute conflicts of the same entity, and to overwrite conflicting attribute values ​​according to priority by defining attribute priorities; the relationship fusion is used to unify the association relationships in multimodal data, and to use a graph embedding model to map semantically similar relationships to a low-dimensional space.

[0028] Specifically, the graph database uses an attribute graph model to store the fused entities and relationships, including:

[0029] Define node types and relationship types based on the metadata model, and support dynamic addition of new entity types; establish combined indexes for high-frequency query attributes, and accelerate complex traversal queries through label-attribute joint indexes.

[0030] Specifically, the method for constructing the simulation knowledge graph is:

[0031] When new simulation data flows in, the knowledge graph nodes, relationships, and attributes are updated in real time through the online entity alignment model and incremental graph embedding algorithm. Knowledge graph version snapshots are generated regularly to record the entity relationship change history and support timeline backtracking or difference comparison.

[0032] Cross-modal entities are constructed as hyperedge relationships to represent complex semantic associations, multimodal feature vectors are attached to graph nodes, and a joint index is established with the vector database; a dual storage engine is used to save the basic graph and incremental graph respectively.

[0033] Specifically, the integration of semantic extended search and enterprise business processes includes:

[0034] Define semantic extension trigger rules based on business processes. When a user enters a query, natural language processing is used to parse the query intent and automatically associate the scene tags and device model metadata in the knowledge graph to generate a multimodal joint query vector.

[0035] A cross-modal joint index is established in the vector database, and hierarchical semantic retrieval is performed using scene labels as filtering conditions. Simulation cases with fully aligned metadata are prioritized, and then expanded to cross-modal related cases through vector similarity.

[0036] Dynamically integrate the search results with the historical decision parameters in the business process, use graph neural networks to build a simulation scheme comparison matrix, and automatically mark the performance difference indicators and applicable scenario constraints between schemes.

[0037] The beneficial effects of the present invention are:

[0038] Through parameterized templates, structured data standardization is enforced to resolve conflicts in multi-source heterogeneous data formats and reduce cross-system data integration costs. The metadata model semantically annotates unstructured data, breaking down the semantic barriers between text, images, and structured data, and improving the efficiency of multimodal data association. Through a layered storage engine (dynamic mapping of structured data, conversion of semi-structured time series tables, and quantization of unstructured features), the system adapts to the characteristics of different modal data and reduces redundant storage space.

[0039] Regarding automated and high-precision knowledge extraction and fusion, a cross-modal attention mechanism (Transformer architecture) dynamically learns implicit associations between text descriptions, time series data, and images, resolving entity ambiguity caused by insufficient context in short text. Predefined domain synonym tables and attribute priority rules reduce manual intervention and ensure knowledge graph consistency. Semantic space mapping embeds multimodal data into a unified vector space, supporting cross-modal entity association.

[0040] Through core technologies such as multimodal semantic alignment, dynamic knowledge fusion, and graph-vector collaborative retrieval, the integration efficiency of enterprise-level simulation data, knowledge reasoning capabilities, and the level of intelligent business decision-making have been significantly improved, solving the bottlenecks of traditional solutions in real-time, scalability, and accuracy, and providing a feasible knowledge-driven solution for complex product development. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] To facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0042] Figure 1 A flowchart of a method for constructing an enterprise-level simulation knowledge graph based on multimodal data integration according to the present invention;

[0043] Figure 2 This is a hierarchical filtering flow chart for the hybrid search in the present invention;

[0044] Figure 3 This is a technical architecture diagram of a method for constructing an enterprise-level simulation knowledge graph based on multimodal data integration in the present invention. DETAILED DESCRIPTION

[0045] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0046] See also Figure 1-3 , a method for constructing an enterprise-level simulation knowledge graph based on multimodal data integration, including:

[0047] S1: Integrate structured data, semi-structured data, and unstructured data through a multimodal data warehouse, standardize the structured data through parameterized templates, semantically annotate and associate the unstructured data through metadata models, and unify the semi-structured data into a standardized format.

[0048] S2: performing knowledge extraction on the semi-structured data and the unstructured data to obtain entities and relationships, and performing knowledge fusion on the entities and relationships in the structured data, the semi-structured data, and the unstructured data;

[0049] S3: Use the graph database to store the fused entities and relationships and build a simulation knowledge graph;

[0050] S4: Combine the simulation knowledge graph with the embedded vector database, implement semantic extension search through multimodal joint search, find similar cases through the simulation result graph, and automatically generate a simulation scheme comparison matrix.

[0051] Specifically, the multimodal data warehouse is used to achieve semantic alignment, efficient retrieval and cross-modal association of multi-source heterogeneous data; adapt to different data types through a hierarchical storage engine; define field constraints through parameterized templates in the structured data layer, and implement dynamic mapping in combination with data warehouse tools; convert log format data into columnar time series tables in the semi-structured data layer; convert image and document data into feature vectors in the unstructured data layer, and perform semantic annotation and association based on the metadata model.

[0052] In this embodiment, taking aircraft engine design simulation as an example, the company needs to integrate multimodal data such as experimental reports (text), sensor logs (semi-structured JSON), material parameter tables (structured CSV), stress cloud maps (images), etc., to build a simulation knowledge graph, support rapid retrieval of historical cases, and generate a design scheme comparison matrix.

[0053] Among them, structured data is a material parameter table (CSV format): containing fields such as material ID, density, melting point, and tensile strength; constraints (such as "density range") are defined using parameterized templates and dynamically mapped to the data warehouse through Apache Atlas; semi-structured data is sensor logs (JSON format): recording time series data such as timestamp, device ID, temperature, and vibration amplitude, converted into a columnar time series table (Parquet format), stored in shards by device ID, and a timestamp index is established; unstructured data includes experimental reports (PDF) describing material fatigue test results and stress cloud maps (PNG) showing the stress distribution of engine blades under high temperature. Labels ("Material ID = Ti-6Al-4V", "Test temperature = 800°C") are extracted through a metadata model, and the cloud maps are converted into ResNet-50 feature vectors.

[0054] Building a multimodal data warehouse based on S1:

[0055] Structured layer:

[0056] Use parameterized templates to parse the material parameter table, standardize field names (such as "Tensile Strength → Tensile_Strength"), and map them to the Hive table.

[0057] Semi-structured layer:

[0058] Aggregate JSON logs into time series tables by time window (every minute), store them in ClickHouse, and create a (device ID, timestamp) joint index.

[0059] Unstructured layer:

[0060] Text data: Use the trained SciBERT model to extract entities (such as "fatigue life = 10^5 cycles") from the experimental report and associate them with the material ID;

[0061] Image data: Use OpenCV to extract the stress cloud map ROI area, generate a 768-dimensional feature vector, and store it in the Milvus vector library.

[0062] The knowledge extraction and fusion based on S2 is reflected in:

[0063] Extract entities (device ID, temperature threshold) from sensor logs and relationships ("The fatigue life of material Ti-6Al-4V at 800°C is 10^5 cycles") from experimental reports. Use a synonym table to align "Ti-6Al-4V" with "Titanium alloy TA6V." The "tensile strength = 950 MPa" parameter in the experimental data takes precedence over the simulation parameter "900 MPa" and is automatically overwritten. Combine "cause" and "trigger" into a unified relationship type.

[0064] S3-based graph database storage: Define node types in Neo4j: material, equipment, and working condition; relationship types are applied to and cause failure; establish a combined index for high-frequency query attributes (material ID, temperature);

[0065] Multimodal joint search and solution generation based on S4:

[0066] The user enters "Search for failure cases of titanium alloys at high temperatures" and the system automatically expands the query to:

[0067] Use the knowledge graph to associate "high temperature → 800°C" and "failure → fatigue life <10^6 cycles"; retrieve similar stress cloud maps in Milvus (eigenvector distance <0.2).

[0068] Specifically, the metadata model extracts metadata of preset fields from the unstructured data to generate a label set; and trains the model through the label set of historical simulation data to identify metadata in the unstructured data.

[0069] Specifically, the metadata model automatically converts multi-source data into nodes and edges of a graph database by matching the semantic types of entities and relationships of the metadata dictionary with those of the metadata model.

[0070] Specifically, the metadata model and the vector database collaborate in the following ways: vector embedding and metadata are jointly stored, vectors of simulation result graphs are associated with scene labels and device model metadata, and hybrid retrieval is supported to narrow the search scope through metadata filtering.

[0071] In this embodiment, taking power equipment fault analysis as an example, enterprises need to extract metadata from equipment fault reports (unstructured text), sensor waveforms (images), and equipment parameter tables (structured data), build a knowledge graph, and collaborate with the vector database to support hybrid retrieval and root cause analysis.

[0072] The input data is unstructured text: fault report (PDF), containing descriptive paragraphs such as "September 1, 2023, Equipment ID: Transformer_001, Fault phenomenon: Winding temperature exceeds the limit (120°C), it is recommended to replace the insulation material model to Class-H"; and unstructured image: current waveform (PNG) collected by the sensor, showing harmonic distortion characteristics.

[0073] Definition of metadata model:

[0074] Preset metadata fields: device ID, fault type, temperature threshold, material model.

[0075] Tag collection example: {"Device ID":"Transformer_001","Fault Type":"Overheating","Temperature Threshold":120,"Material Model":"Class-H"}.

[0076] For unstructured text processing, a domain-tuned BERT model is used to extract entities from fault reports. For unstructured image processing, ResNet-50 is used to extract waveform feature vectors and associate labels using a metadata model. The NER model is fine-tuned based on a collection of historical fault report labels to improve the recognition accuracy of fields such as "temperature threshold" and "material model."

[0077] Metadata drives automatic conversion of the graph database. The fields Device ID = Transformer_001, Rated Temperature = 100°C in the device parameter table (CSV) are mapped to nodes through the metadata dictionary. Metadata extracted from the fault report, Device ID = Transformer_001, Fault Type = Overheating, is converted to nodes and relationships. Furthermore, data in the sensor log (JSON) such as {"timestamp":"2023-09-01T14:00:00","device ID":"Transformer_001","current harmonic distortion rate":"15%"}, is converted to a time series relationship.

[0078] Metamodel and vector database collaborative implementation:

[0079] The feature vector of the fault waveform and metadata (device ID, fault type) are stored in Milvus;

[0080] Filter candidate vectors with "fault type = harmonic distortion" and "device model = Transformer_series";

[0081] Retrieve the top 10 cases that are most similar to the current waveform in the candidate set;

[0082] Returns the similarity score between the associated graph nodes (such as fault causes and maintenance records) and the vector;

[0083] In actual business, engineers input the current fault waveform, and the system automatically generates a root cause analysis report, including:

[0084] Failure types and maintenance plans of similar historical cases;

[0085] Temperature resistance performance comparison matrix of related material models;

[0086] Graph reasoning results (e.g., “harmonic distortion → insulation aging → recommended material replacement”).

[0087] Specifically, the knowledge extraction methods include: semantic alignment, entity relationship extraction, and cross-modal knowledge fusion;

[0088] The semantic alignment includes explicit alignment, implicit alignment, and semantic space mapping; the explicit alignment defines semantic type constraints through a metadata dictionary and establishes mapping rules between structured data fields and unstructured data labels; the implicit alignment adopts the cross-modal attention mechanism of the Transformer architecture to dynamically learn the association between multimodal data sub-components during training; the semantic space mapping maps text descriptions, time series data, and simulated images to a unified vector space.

[0089] The entity relationship extraction uses named entity recognition technology to extract entities from text and determines the relationship between entities with the help of dependency syntax analysis;

[0090] The cross-modal knowledge fusion associates entities and relationships in different modal data to generate a unified knowledge representation.

[0091] In this embodiment, the display alignment is based on metadata dictionary driven rule mapping. The algorithm uses the regularization engine and OpenCV region detection to generate a mapping table:

[0092] Text parsing example:

[0093] {

[0094] def extract_metadata(text):

[0095] pattern = r"FaultCode: (d{4})" #Explicit rule definition#

[0096] fault_code = re.search(pattern, text).group(1)

[0097] return {"fault code": fault_code}

[0098] }

[0099] Image processing examples:

[0100] {

[0101] def img2metadata(img):

[0102] sensor_roi = detect_roi(img) #Hot spot detection based on YOLOv5#

[0103] sensor_id = query_sensor_db(sensor_roi.coords) #Coordinate conversion to device ID#

[0104] threshold = get_threshold(sensor_id) #Query the database threshold#

[0105] return {"temperature threshold": threshold}

[0106] }

[0107] The implicitly aligned cross-modal attention mechanism uses a dual-tower Transformer network. The text tower uses BERT-base, the image tower uses ViT-B / 16, and the time tower uses LSTM. The cross-modal attention layer calculates the token association between modalities:

[0108] ,

[0109] Among them, Q is the query matrix, which comes from the text modality, K is the key matrix, which comes from the image or time series modality, V is the value matrix, which comes from the image or time series modality, d k The dimension of the key is used to scale the dot product to stabilize the gradient. The softmax function normalizes the scaled dot product result to obtain an attention weight matrix. The attention weight matrix is ​​multiplied by the value matrix to obtain the final cross-modal attention output.

[0110] The training strategy uses contrastive learning loss. The positive samples are time series combinations of images and texts of the same fault event, and the negative samples are randomly replaced with other modes.

[0111] ,

[0112] in, Represents the value of the contrastive learning loss function, and the goal is to minimize this loss value; Represents sample pairs The similarity score of x i and y i Represents a pair of positive samples (e.g., an image and its corresponding text description); Used to control the distribution of similarity scores; N is the number of negative samples, Indicates that the similarity score of the positive sample is obtained through the parameter Adjusted exponential form.

[0113] A multimodal encoder that unifies vector encoding in the semantic mapping space uses a CLIP variant to extend temporal processing;

[0114] Based on the Stanford Dependency parsing results, define the relationship extraction template:

[0115] Rule 1: nsubj(fault, component) ∧ dobj(fault, cause) → Relationship: component-cause-cause;

[0116] Rule 2: amod(temperature, abnormality) ∧ nmod(exceed, threshold) → Relationship: temperature-exceed threshold;

[0117] The input example "Bearing temperature abnormality exceeds the threshold and causes F002 fault" is parsed as:

[0118] [bearing]-nsubj→[exceed]←dobj-[threshold]

[0119] [Exceed]-nmod→[Temperature]→Generate the relationship: (bearing, exceed threshold, temperature), (temperature, trigger, F002).

[0120] Specifically, the knowledge fusion includes: entity disambiguation, attribute fusion, and relationship fusion; the entity disambiguation is used to resolve the reference differences of the same entity in multi-source data, and to force the alignment of entity names in structured and unstructured data through a predefined domain synonym table; the attribute fusion is used to integrate multi-source attribute conflicts of the same entity, and to overwrite conflicting attribute values ​​according to priority by defining attribute priorities; the relationship fusion is used to unify the association relationships in multimodal data, and to use a graph embedding model to map semantically similar relationships to a low-dimensional space.

[0121] Specifically, the graph database uses an attribute graph model to store the fused entities and relationships, including:

[0122] Define node types and relationship types based on the metadata model, and support dynamic addition of new entity types; establish combined indexes for high-frequency query attributes, and accelerate complex traversal queries through label-attribute joint indexes.

[0123] Specifically, the method for constructing the simulation knowledge graph is:

[0124] When new simulation data flows in, the knowledge graph nodes, relationships, and attributes are updated in real time through the online entity alignment model and incremental graph embedding algorithm. Knowledge graph version snapshots are generated regularly to record the entity relationship change history and support timeline backtracking or difference comparison.

[0125] Cross-modal entities are constructed as hyperedge relationships to represent complex semantic associations, multimodal feature vectors are attached to graph nodes, and a joint index is established with the vector database; a dual storage engine is used to save the basic graph and incremental graph respectively.

[0126] Specifically, the integration of semantic extended search and enterprise business processes includes:

[0127] Define semantic extension trigger rules based on business processes. When a user enters a query, natural language processing is used to parse the query intent and automatically associate the scene tags and device model metadata in the knowledge graph to generate a multimodal joint query vector.

[0128] A cross-modal joint index is established in the vector database, and hierarchical semantic retrieval is performed using scene labels as filtering conditions. Simulation cases with fully aligned metadata are prioritized, and then expanded to cross-modal related cases through vector similarity.

[0129] Dynamically integrate the search results with the historical decision parameters in the business process, use graph neural networks to build a simulation scheme comparison matrix, and automatically mark the performance difference indicators and applicable scenario constraints between schemes.

[0130] In this embodiment, the multimodal joint query of semantic extended search and business integration uses Semantic Parse technology to convert natural language into Cypher query, and the hybrid search adopts a hierarchical filtering process, such as Figure 2 As shown in the following figure, a hierarchical index is built in Milvus.

[0131] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0132] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0133] The program code included in the computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF or the like, or any suitable combination thereof. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, and the programming language includes an object-oriented programming language such as Java, Smalltalk, C++, and also includes a conventional procedural programming language such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, utilizing an Internet service provider to connect through the Internet).

[0134] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for constructing an enterprise-level simulation knowledge graph based on multimodal data integration, characterized in that: include: S1: Integrate structured data, semi-structured data, and unstructured data through a multimodal data warehouse, standardize the structured data through parameterized templates, semantically annotate and associate the unstructured data through metadata models, and unify the semi-structured data into a standardized format. S2: performing knowledge extraction on the semi-structured data and the unstructured data to obtain entities and relationships, and performing knowledge fusion on the entities and relationships in the structured data, the semi-structured data, and the unstructured data; S3: Use a graph database to store the fused entities and relationships and construct a simulation knowledge graph. The method for constructing the simulation knowledge graph is as follows: When new simulation data flows in, the knowledge graph nodes, relationships, and attributes are updated in real time through the online entity alignment model and incremental graph embedding algorithm. Knowledge graph version snapshots are generated regularly to record the entity relationship change history and support timeline backtracking or difference comparison. Cross-modal entities are constructed as hyperedge relationships to represent complex semantic associations. Multimodal feature vectors are added to graph nodes, and a joint index is established with the vector database. A dual storage engine is used to store the basic graph and incremental graph respectively. S4: combining the simulation knowledge graph with the embedded vector database, and implementing semantic extension search through multimodal joint search, finding similar cases through the simulation result graph, and automatically generating a simulation scheme comparison matrix; The integration of semantic extended search and enterprise business processes includes: Define semantic extension trigger rules based on business processes. When a user enters a query, natural language processing is used to parse the query intent and automatically associate the scene tags and device model metadata in the knowledge graph to generate a multimodal joint query vector. A cross-modal joint index is established in the vector database, and hierarchical semantic retrieval is performed using scene labels as filtering conditions. Simulation cases with fully aligned metadata are prioritized, and then expanded to cross-modal related cases through vector similarity. Dynamically integrate the search results with the historical decision parameters in the business process, use graph neural networks to build a simulation scheme comparison matrix, and automatically mark the performance difference indicators and applicable scenario constraints between schemes.

2. The method according to claim 1, characterized in that The multimodal data warehouse is used to achieve semantic alignment, efficient retrieval and cross-modal association of multi-source heterogeneous data; adapt to different data types through a layered storage engine; define field constraints through parameterized templates in the structured data layer, and implement dynamic mapping in combination with data warehouse tools; convert log format data into columnar time series tables in the semi-structured data layer; convert image and document data into feature vectors in the unstructured data layer, and perform semantic annotation and association based on the metadata model.

3. The method according to claim 1, characterized in that The metadata model extracts metadata of preset fields from the unstructured data to generate a label set; and the label set training model of historical simulation data is used to identify metadata in the unstructured data.

4. The method according to claim 1, wherein The metadata model automatically converts multi-source data into nodes and edges of a graph database by matching the semantic types of entities and relationships in the metadata dictionary with those of the metadata model.

5. The method according to claim 1, wherein The synergy between the metadata model and the vector database is as follows: vector embedding and metadata are jointly stored, the vectors of the simulation result graph are associated with scene labels and device model metadata, and hybrid retrieval is supported to narrow the search scope through metadata filtering.

6. The method according to claim 1, characterized in that The knowledge extraction method includes: semantic alignment, entity relationship extraction, and cross-modal knowledge fusion; The semantic alignment includes explicit alignment, implicit alignment, and semantic space mapping. The explicit alignment defines semantic type constraints through a metadata dictionary and establishes mapping rules between structured data fields and unstructured data labels. The implicit alignment uses the cross-modal attention mechanism of the Transformer architecture to dynamically learn the association between multimodal data subcomponents during training. The semantic space mapping maps text descriptions, time series data, and simulated images to a unified vector space. The entity relationship extraction uses named entity recognition technology to extract entities from text and determines the relationship between entities with the help of dependency syntax analysis; The cross-modal knowledge fusion associates entities and relationships in different modal data to generate a unified knowledge representation.

7. The method according to claim 1, characterized in that The knowledge fusion includes: entity disambiguation, attribute fusion, and relationship fusion; the entity disambiguation is used to resolve the reference differences of the same entity in multi-source data, and enforce the alignment of entity names in structured and unstructured data through a predefined domain synonym table; the attribute fusion is used to integrate multi-source attribute conflicts of the same entity, and overwrite conflicting attribute values ​​according to priority by defining attribute priorities; the relationship fusion is used to unify the association relationships in multimodal data, and use a graph embedding model to map semantically similar relationships to a low-dimensional space.

8. The method according to claim 1, characterized in that The graph database uses an attribute graph model to store the fused entities and relationships, specifically including: Define node types and relationship types based on the metadata model, and support dynamic addition of new entity types; establish combined indexes for high-frequency query attributes, and accelerate complex traversal queries through label-attribute joint indexes.

Images