Enterprise-level simulation knowledge graph construction method based on multi-modal data integration

Abstract

The invention relates to an enterprise-level simulation knowledge graph construction method based on multi-modal data integration, and belongs to the technical field of knowledge graphs. The method comprises the following steps: integrating structured data, semi-structured data and unstructured data through a multi-modal data warehouse; performing knowledge extraction on the semi-structured data and the non-structured data to obtain entities and relationships, and performing knowledge fusion; storing the fused entities and relationships by using a graph database, and constructing a simulation knowledge graph; a vector database is embedded in combination with a simulation knowledge graph, semantic extension search is realized through multi-modal joint search, similar cases are searched through a simulation result graph, and a simulation scheme comparison matrix is automatically generated. The multi-modal data is effectively integrated, the comprehensiveness and accuracy of knowledge graph construction are improved, more powerful, efficient and intelligent support is provided for simulation analysis of enterprises, and the enterprises can be assisted in rapidly making scientific decisions in complex and changeable business scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge graphs, and particularly relates to a method for constructing an enterprise-level simulation knowledge graph based on multi-modal data integration. Background Art

[0002] In modern manufacturing, CAE simulation technology has become a core link in product R & D. However, simulation data presents multi-modal characteristics (structured parameters, semi-structured logs, unstructured images / documents), and the sources are scattered and the formats are heterogeneous. For example, although systems such as SLM and Ansys Minerva support the standardization of the simulation process, they have insufficient capabilities for the dynamic fusion and semantic association of multi-modal data, resulting in prominent data island problems. Enterprises need to manage a large amount of data such as simulation models, experimental results, and optimization parameters at the same time. Traditional SPDM systems rely on hierarchical storage and static knowledge bases and are difficult to achieve cross-modal real-time knowledge reasoning.

[0003] There are significant differences in physical meaning, format accuracy, and time series among multi-modal data (for example, it is difficult to align the time series characteristics of 3D scan point clouds and device logs), resulting in difficulty in ensuring the consistency after fusion. Although existing data warehouse technologies can adapt to different data types, they lack a unified semantic alignment mechanism and have low cross-modal retrieval efficiency. For example, the semantic association between text descriptions and simulation images in industrial big data relies on manual annotation, which is costly and has poor scalability.

[0004] Traditional knowledge graph construction relies on single-modal data, and entity disambiguation and attribute fusion require manual intervention rules, which are difficult to meet the dynamic update requirements of multi-source heterogeneous data in enterprise-level simulation scenarios. Incremental update algorithms are inefficient in processing large-scale dynamic data and cannot effectively process the complex semantic associations between cross-modal entities.

[0005] Existing retrieval technologies lack deep integration with business processes and cannot perform semantic expansion by combining scenario tags or device metadata, resulting in a low matching degree between search results and business requirements. The cross-modal association of simulation result graphs relies on manual experience, and it is difficult to automatically generate a solution comparison matrix, affecting decision-making efficiency.

[0006] Currently, there are still the following areas to be improved in the enterprise-level simulation knowledge graph based on multi-modal data integration: 1. The semantic alignment of heterogeneous data sources (such as CAD models, sensor logs, experimental reports) relies on manual rules, lacking an automated semantic annotation and cross-modal mapping mechanism. For example, the parameter of "material fatigue strength" in unstructured documents cannot be automatically associated with the "stress threshold" field in structured databases. 2. Differences in the reference of the same entity in multi-source data (such as the inconsistent naming of "motor model A" in different systems) lead to redundancy and errors in the knowledge graph. Existing disambiguation methods (rule-based or shallow machine learning) are difficult to handle scenarios with insufficient short text context features; entity alignment requires combining domain synonym tables and dynamic context analysis, but predefined rules are difficult to cover complex business scenarios, and attribute priority conflicts require manual intervention; 3. Existing hybrid retrieval schemes have data synchronization delays and fragmented query scopes, resulting in a low recall rate of cross-modal association cases; the locality defects of graph databases and the alignment defects of vector databases jointly limit the performance of complex queries, especially in scenarios that require joint metadata filtering and semantic expansion; 4. The online update of the knowledge graph depends on full-scale reconstruction and cannot achieve real-time fusion of high-frequency incremental data; existing incremental algorithms have an exponential growth in computational complexity when dealing with hyperedge relationships and lack lightweight models to support edge computing scenarios; 5. The intent parsing of natural language queries depends on general NLP models and lacks domain knowledge enhancement (such as implicit constraints on device models), resulting in semantic expansion deviating from business logic; in cross-modal retrieval, the mapping between text queries and simulation images depends on shallow feature matching and is difficult to capture physical associations in business scenarios. Summary of the Invention

[0007] To solve the above problems existing in the prior art, the present invention provides an enterprise-level simulation knowledge graph construction method based on multi-modal data integration; The object of the present invention can be achieved through the following technical solutions: S1: Integrate structured data, semi-structured data, and unstructured data through a multi-modal data warehouse, perform standardized processing on the structured data through a parameterized template, perform semantic annotation and association on the unstructured data through a metadata model, and unify the schema of the semi-structured data through a standard format; S2: Extract knowledge from the semi-structured data and the unstructured data to obtain entities and relationships, and fuse the entities and relationships in the structured data, the semi-structured data, and the unstructured data; S3: Store the fused entities and relationships using a graph database to construct a simulation knowledge graph; S4: Combine the simulation knowledge graph with an embedded vector database, and achieve semantic expansion search through multi-modal joint search, find similar cases through a simulation result graph, and automatically generate a simulation scheme comparison matrix.

[0008] Specifically, the multi-modal data warehouse is used to achieve semantic alignment, efficient retrieval, and cross-modal association of multi-source heterogeneous data; adapt different data types through a hierarchical storage engine; define field constraints through parameterized templates in the structured data layer and implement dynamic mapping in combination with data warehouse tools; convert log-format data into columnar time series tables in the semi-structured data layer; convert image and document data into feature vectors in the unstructured data layer, and perform semantic annotation and association based on the metadata model.

[0009] Specifically, the metadata model extracts metadata of preset fields from the unstructured data to generate a label set; trains a model through the label set of historical simulation data to identify metadata in the unstructured data.

[0010] Specifically, the metadata model matches the semantic types of entities and relationships in the metadata dictionary and the metadata model, and automatically converts multi-source data into nodes and edges of a graph database.

[0011] Specifically, the collaboration between the metadata model and the vector database is manifested as: joint storage of vector embedding and metadata, associating the vectors of simulation result graphs with scene labels and device model metadata, and supporting hybrid retrieval, narrowing the search scope through metadata filtering.

[0012] Specifically, the knowledge extraction method includes: semantic alignment, entity relationship extraction, and cross-modal knowledge fusion; The semantic alignment includes explicit alignment, implicit alignment, and semantic space mapping; the explicit alignment defines semantic type constraints through a metadata dictionary and establishes mapping rules between structured data fields and unstructured data labels; the implicit alignment uses the cross-modal attention mechanism of the Transformer architecture to dynamically learn the associations of multi-modal data sub-components during training; the semantic space mapping maps text descriptions, time series data, and simulation images to a unified vector space.

[0013] The entity relationship extraction uses named entity recognition technology to extract entities from text and determines the relationships between entities with the help of dependency syntactic analysis. The cross-modal knowledge fusion associates entities and relationships in different modal data to generate a unified knowledge representation.

[0014] Specifically, the knowledge fusion includes: entity disambiguation, attribute fusion, and relationship fusion; the entity disambiguation is used to solve the reference differences of the same entity in multi-source data, and the entity names in structured and unstructured data are forced to be aligned through a predefined domain synonym table; the attribute fusion is used to integrate the multi-source attribute conflicts of the same entity, and the conflicting attribute values are overwritten according to the defined attribute priorities; the relationship fusion is used to unify the association relationships in multi-modal data, and a graph embedding model is used to map semantically similar relationships to a low-dimensional space.

[0015] Specifically, the graph database stores the fused entities and relationships using an attribute graph model, which specifically includes: Define node types and relationship types according to the metadata model, and support dynamically adding new entity types; establish a composite index for frequently queried attributes, and accelerate complex traversal queries through a label-attribute joint index.

[0016] Specifically, the construction method of the simulation knowledge graph is as follows: When new simulation data flows in, the knowledge graph nodes, relationships, and attributes are updated in real time through an online entity alignment model and an incremental graph embedding algorithm; knowledge graph version snapshots are generated regularly to record the entity relationship change history, and support backtracking or comparing differences along the timeline; Cross-modal entities are constructed into hyperedge relationships to represent composite semantic associations, multi-modal feature vectors are attached to the graph nodes, and a joint index is established with the vector database; a dual storage engine is used to save the basic graph and the incremental graph respectively.

[0017] Specifically, the integration of the semantic extended search and the enterprise business process includes: Based on the business process, define semantic extension trigger rules. When the user inputs query conditions, the query intent is parsed through natural language processing and automatically associated with the scenario labels and device model metadata in the knowledge graph to generate a multi-modal joint query vector; Establish a cross-modal joint index in the vector database, perform hierarchical semantic retrieval with the scenario label as the filtering condition, first match the simulation cases with completely aligned metadata, and then expand to cross-modal associated cases through vector similarity; Dynamically fuse the retrieval results with the historical decision parameters in the business process, use a graph neural network to construct a simulation scheme comparison matrix, and automatically label the performance difference indicators and applicable scenario constraint conditions between the schemes.

[0018] The beneficial effects of the present invention are: Standardize structured data through parameterized templates, resolve multi-source heterogeneous data format conflicts, and reduce the cost of cross-system data integration; semantic annotation of unstructured data by the metadata model breaks the semantic barriers between text, images, and structured data, improving the efficiency of multi-modal data association; adapt to the characteristics of different modal data through a hierarchical storage engine (dynamic mapping of structured data, conversion of semi-structured time-series tables, and vectorization of unstructured features), reducing redundant storage space occupancy.

[0019] Regarding the automation and high precision of knowledge extraction and fusion, dynamically learn the implicit associations between text descriptions, time-series data, and images through a cross-modal attention mechanism (Transformer architecture) to resolve the problem of entity reference ambiguity caused by insufficient short-text context. Pre-define domain synonym tables and attribute priority rules to reduce manual intervention and ensure the consistency of the knowledge graph. Semantic space mapping embeds multi-modal data into a unified vector space to support cross-modal entity association.

[0020] Through core technologies such as multi-modal semantic alignment, dynamic knowledge fusion, and graph-vector collaborative retrieval, the integration efficiency of enterprise-level simulation data, knowledge reasoning ability, and business decision-making intelligence level have been significantly improved, resolving the bottlenecks of traditional solutions in terms of real-time performance, scalability, and accuracy, and providing a practical knowledge-driven solution for complex product R & D. Brief Description of the Drawings

[0021] For the convenience of those skilled in the art to understand, the present invention will be further described below in conjunction with the accompanying drawings.

[0022] Figure 1 It is a schematic flowchart of a method for constructing an enterprise-level simulation knowledge graph based on multi-modal data integration according to the present invention; Figure 2 It is a hierarchical filtering flowchart of hybrid retrieval in the present invention; Figure 3 It is a schematic diagram of the technical architecture of a method for constructing an enterprise-level simulation knowledge graph based on multi-modal data integration according to the present invention. Detailed Embodiments

[0023] To further elaborate on the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following will, in conjunction with the accompanying drawings and preferred embodiments, describe in detail the specific embodiments, structures, features, and effects of the present invention.

[0024] Please refer to Figures 1-3 , a method for constructing an enterprise-level simulation knowledge graph based on multi-modal data integration, including: S1: Integrate structured data, semi-structured data, and unstructured data through a multimodal data warehouse. Standardize the structured data through a parameterized template, perform semantic annotation and association on the unstructured data through a metadata model, and unify the schema of the semi-structured data through a standard format. S2: Extract entities and relationships from the semi-structured data and the unstructured data, and fuse the entities and relationships in the structured data, the semi-structured data, and the unstructured data. S3: Store the fused entities and relationships using a graph database to construct a simulation knowledge graph. S4: Incorporate the simulation knowledge graph into a vector database, and achieve semantic extended search through multimodal joint search. Find similar cases through the simulation result graph, and automatically generate a simulation scheme comparison matrix.

[0025] Specifically, the multimodal data warehouse is used to achieve semantic alignment, efficient retrieval, and cross-modal association of multi-source heterogeneous data; adapt different data types through a hierarchical storage engine; define field constraints through a parameterized template in the structured data layer, and achieve dynamic mapping in combination with a data warehouse tool; convert log-format data into a columnar time-series table in the semi-structured data layer; convert image and document data into feature vectors in the unstructured data layer, and perform semantic annotation and association based on a metadata model.

[0026] In this embodiment, taking the design simulation of an aero-engine as an example, an enterprise needs to integrate multimodal data such as experimental reports (text), sensor logs (semi-structured JSON), material parameter tables (structured CSV), and stress nephograms (images), construct a simulation knowledge graph, support rapid retrieval of historical cases, and generate a design scheme comparison matrix.

[0027] Among them, the structured data is a material parameter table (CSV format): including fields such as material ID, density, melting point, and tensile strength; define constraints using a parameterized template (such as "density range"), and dynamically map it to the data warehouse through Apache Atlas; the semi-structured data is sensor logs (JSON format): record time series data such as timestamps, device IDs, temperatures, and vibration amplitudes, convert it into a columnar time-series table (Parquet format), store it in slices according to the device ID, and establish a timestamp index; the unstructured data is an experimental report (PDF) describing the results of material fatigue tests and a stress nephogram (PNG) showing the stress distribution of engine blades at high temperatures. Extract tags ("material ID = Ti-6Al-4V", "test temperature = 800°C") through a metadata model, and convert the nephogram into a ResNet-50 feature vector.

[0028] Building a multi-modal data warehouse based on S1: Structured layer: Use a parameterized template to parse the material parameter table, standardize field names (e.g., "Tensile Strength → Tensile_Strength"), and map them to a Hive table.

[0029] Semi-structured layer: Aggregate JSON logs into a time series table by time window (per minute), store them in ClickHouse, and establish a composite index of (device ID, timestamp).

[0030] Unstructured layer: Text data: Extract entities (e.g., "Fatigue life = 10^5 cycles") from the experimental report through the trained SciBERT model and associate them with the material ID; Image data: Use OpenCV to extract the ROI region of the stress nephogram, generate 768-dimensional feature vectors, and store them in the Milvus vector library.

[0031] Knowledge extraction and fusion based on S2 are reflected in: Extract entities (device ID, temperature threshold) from the sensor log, and extract relationships ("The fatigue life of material Ti-6Al-4V at 800°C is 10^5 cycles") from the experimental report; Align "Ti-6Al-4V" with "titanium alloy TA6V" through a synonym table; The "Tensile strength = 950MPa" in the experimental data has a higher priority than the simulation parameter "900MPa" and automatically overrides it; Merge "cause" and "trigger" into a unified relationship type.

[0032] Graph database storage based on S3: Define node types in Neo4j: materials, devices, working conditions; The relationship type is applied to, causes failure; Establish a composite index for frequently queried attributes (material ID, temperature); Multi-modal joint search and solution generation based on S4: When the user enters "Find failure cases of titanium alloy at high temperature", the system automatically expands the query: Associate "high temperature → 800°C" and "failure → fatigue life < 10^6 cycles" through the knowledge graph; Retrieve similar stress nephograms in Milvus (feature vector distance < 0.2).

[0033] Specifically, the metadata model extracts metadata of preset fields from the unstructured data to generate a label set; Train a model through the label set of historical simulation data to identify metadata in unstructured data.

[0034] Specifically, the metadata model automatically converts multi-source data into nodes and edges of a graph database by matching the semantic types of entities and relationships in the metadata dictionary with the metadata model.

[0035] Specifically, the collaboration between the metadata model and the vector database is manifested as follows: vector embeddings and metadata are jointly stored, the vectors of the simulation result graph are associated with scenario labels and device model metadata, and hybrid retrieval is supported to narrow the search scope through metadata filtering.

[0036] In this embodiment, taking the fault analysis of power equipment as an example, an enterprise needs to extract metadata from equipment fault reports (unstructured texts), sensor waveform diagrams (images), and equipment parameter tables (structured data), construct a knowledge graph and collaborate with the vector database to support hybrid retrieval and root cause analysis.

[0037] The input data is unstructured text: a fault report (PDF), containing descriptive paragraphs such as "On September 1, 2023, equipment ID: Transformer_001, fault phenomenon: winding temperature exceeding the limit (120°C), and it is recommended to replace the insulation material model with Class-H"; and unstructured images: current waveform diagrams (PNG) collected by sensors, showing harmonic distortion characteristics.

[0038] Definition of the metadata model: Preset metadata fields: equipment ID, fault type, temperature threshold, material model.

[0039] Example of the label set: {"equipment ID": "Transformer_001", "fault type": "overheat", "temperature threshold": 120, "material model": "Class-H"}.

[0040] For unstructured text processing, a domain-fine-tuned BERT model is used to extract entities from the fault report. For unstructured image processing, ResNet-50 is used to extract waveform diagram feature vectors and associate labels through the metadata model. The NER model is fine-tuned based on the historical fault report label set to improve the recognition accuracy of fields such as "temperature threshold" and "material model". The metadata-driven graph database is automatically converted. The fields in the equipment parameter table (CSV) with equipment ID = Transformer_001 and rated temperature = 100°C are mapped to nodes through the metadata dictionary. The metadata equipment ID = Transformer_001 and fault type = overheat extracted from the fault report are converted into nodes and relationships. {"Timestamp": "2023-09-01T14:00:00", "equipment ID": "Transformer_001", "current harmonic distortion rate": "15%"} in the sensor log (JSON) is converted into a time series relationship. Implementation of the meta - model in collaboration with the vector database: Jointly store the feature vectors of the fault waveform diagram and the metadata (equipment ID, fault type) into Milvus; Filter candidate vectors with "fault type = harmonic distortion" and "equipment model = Transformer_ series"; Retrieve the top - 10 cases in the candidate set that are most similar to the current waveform diagram; Return the associated graph nodes (such as fault causes, maintenance records) and the vector similarity scores; In actual business, when an engineer inputs the current fault waveform diagram, the system automatically generates a root cause analysis report, including: Fault types and maintenance solutions of similar historical cases; The temperature - resistance performance comparison matrix of associated material models; Graph inference results (such as "harmonic distortion → insulation aging → it is recommended to replace the material").

[0041] Specifically, the knowledge extraction method includes: semantic alignment, entity - relationship extraction, and cross - modal knowledge fusion; The semantic alignment includes explicit alignment, implicit alignment, and semantic space mapping; the explicit alignment defines semantic type constraints through a metadata dictionary and establishes mapping rules between structured data fields and unstructured data labels; the implicit alignment uses the cross - modal attention mechanism of the Transformer architecture to dynamically learn the associations of multi - modal data sub - components during training; the semantic space mapping maps text descriptions, time - series data, and simulation images into a unified vector space.

[0042] The entity - relationship extraction uses named - entity recognition technology to extract entities from text and determines the relationships between entities with the help of dependency syntax analysis; The cross - modal knowledge fusion associates entities and relationships in different - modal data to generate a unified knowledge representation.

[0043] In this embodiment, the display alignment is based on rule mapping driven by a metadata dictionary, and the algorithm uses a regular expression engine and OpenCV region detection to generate a mapping table: Text parsing example: { def extract_metadata(text): pattern = r"FaultCode: (d{4})" # Explicit rule definition# fault_code = re.search(pattern, text).group(1) return {"fault code": fault_code} } Example of image processing: { def img2metadata(img): sensor_roi = detect_roi(img) # Hot zone detection based on YOLOv5# sensor_id = query_sensor_db(sensor_roi.coords) # Coordinate to device ID conversion# threshold = get_threshold(sensor_id) # Query database threshold# return {"temperature threshold": threshold} } The implicit alignment cross-modal attention mechanism uses a two-tower Transformer network. The text tower uses BERT-base, the image tower uses ViT-B / 16, and the temporal tower is LSTM; the cross-modal attention layer calculates the token associations between modalities: , where Q is the query matrix from the text modality, K is the key matrix from the image or temporal modality, V is the value matrix from the image or temporal modality, and d k is the dimension of the key, used to scale the dot product to stabilize the gradient. The softmax function normalizes the scaled dot product result to obtain an attention weight matrix, and multiplies the attention weight matrix by the value matrix to obtain the final cross-modal attention output.

[0044] The training strategy uses contrastive learning loss. The positive samples are the text-image-temporal combinations of the same fault event, and the negative samples randomly replace the modalities: , where represents the value of the contrastive learning loss function, and the goal is to minimize this loss value; represents the similarity score of the sample pair where x i and y i respectively represent a pair of positive samples (e.g., an image and the corresponding text description); is used to control the distribution of the similarity scores; N is the number of negative samples, represents the exponential form of the similarity score of the positive sample after being adjusted by the parameter .

[0045] A multi-modal encoder with unified vector encoding in the semantic mapping space uses a CLIP variant to expand temporal processing; Based on the Stanford Dependency parsing results, define relation extraction templates: Rule 1: nsubj(fault, component) ∧ dobj(fault, cause) → relation: component - causes - cause; Rule 2: amod(temperature, abnormal) ∧ nmod(exceed, threshold) → relation: temperature - exceeds threshold; The input example "The abnormal temperature of the bearing exceeds the threshold and causes F002 fault" is parsed as: [bearing] - nsubj → [exceed] ← dobj - [threshold] [exceed] - nmod → [temperature] → generate relations: (bearing, exceeds threshold, temperature), (temperature, causes, F002).

[0046] Specifically, the knowledge fusion includes: entity disambiguation, attribute fusion, and relation fusion; the entity disambiguation is used to solve the reference differences of the same entity in multi-source data, and force-align the entity names in structured and unstructured data through a predefined domain synonym table; the attribute fusion is used to integrate multi-source attribute conflicts of the same entity, and overwrite the conflicting attribute values according to the defined attribute priorities; the relation fusion is used to unify the association relations in multi-modal data, and map semantically similar relations to a low-dimensional space using a graph embedding model.

[0047] Specifically, the graph database stores the fused entities and relations using an attribute graph model, specifically including: Define node types and relation types according to the metadata model, and support dynamically adding new entity types; establish composite indexes for frequently queried attributes, and accelerate complex traversal queries through label-attribute joint indexes.

[0048] Specifically, the construction method of the simulation knowledge graph is: When new simulation data flows in, real-time update the nodes, relations, and attributes of the knowledge graph through an online entity alignment model and an incremental graph embedding algorithm; regularly generate knowledge graph version snapshots, record the entity relation change history, and support backtracking or comparing differences along the time axis; Construct cross-modal entities as hyperedge relations to represent composite semantic associations, attach multi-modal feature vectors to the graph nodes, and establish a joint index with the vector database; use a dual storage engine to save the basic graph and the incremental graph respectively.

[0049] Specifically, the integration of the semantic extended search and the enterprise business process includes: Based on the business process to define semantic extension trigger rules. When the user inputs query conditions, the query intention is parsed through natural language processing and the scenario tags and device model metadata in the knowledge graph are automatically associated to generate a multi-modal joint query vector. A cross-modal joint index is established in the vector database. Hierarchical semantic retrieval is performed with the scenario tags as the filtering conditions. First, simulation cases with completely aligned metadata are preferentially matched, and then cross-modal associated cases are extended through vector similarity. The retrieval results are dynamically fused with the historical decision parameters in the business process, and a simulation scheme comparison matrix is constructed using a graph neural network to automatically label the performance difference indicators and applicable scenario constraint conditions between the schemes.

[0050] In this embodiment, the multi-modal joint query of semantic extension search and business integration uses the Semantic Parse technology to convert natural language into Cypher queries, and the hybrid retrieval adopts a hierarchical filtering process. As Figure 2 shown, a hierarchical index is constructed in Milvus.

[0051] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device.

[0052] The computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0053] The program code contained on a computer-readable medium can be transmitted with any appropriate medium, including but not limited to wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0054] As described above, it is only the preferred embodiment of the present invention, and there is no any form of limitation to the present invention. Although the present invention has been disclosed as above with the preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to be equivalent embodiments of equivalent changes within the scope of the technical solution of the present invention. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention still belong to the scope of the technical solution of the present invention.

Claims

1. A method for constructing an enterprise-level simulation knowledge graph based on multi-modal data integration, characterized in that, Including: S1: Integrate structured data, semi-structured data, and unstructured data through a multi-modal data warehouse, standardize the structured data through a parameterized template, perform semantic annotation and association on the unstructured data through a metadata model, and unify the schema of the semi-structured data through a standard format; S2: Extract entities and relationships from the semi-structured data and the unstructured data, and fuse the entities and relationships in the structured data, the semi-structured data, and the unstructured data; S3: Store the fused entities and relationships using a graph database to construct a simulation knowledge graph; S4: Incorporate the simulation knowledge graph into a vector database, and achieve semantic extended search through multi-modal joint search, find similar cases through a simulation result graph, and automatically generate a simulation scheme comparison matrix.

2. The method according to claim 1, wherein The multi-modal data warehouse is used to achieve semantic alignment, efficient retrieval, and cross-modal association of multi-source heterogeneous data; adapt to different data types through a hierarchical storage engine; define field constraints through a parameterized template in the structured data layer, and achieve dynamic mapping in combination with a data warehouse tool; convert log-format data into a columnar-format time series table in the semi-structured data layer; convert image and document data into feature vectors in the unstructured data layer, and perform semantic annotation and association based on a metadata model.

3. The method according to claim 1, wherein The metadata model extracts metadata of preset fields from the unstructured data to generate a label set; trains a model through the label set of historical simulation data to identify metadata in unstructured data.

4. The method according to claim 1, wherein The metadata model matches the semantic types of entities and relationships of the metadata model through a metadata dictionary, and automatically converts multi-source data into nodes and edges of a graph database.

5. The method according to claim 1, characterized in that, The collaboration between the metadata model and the vector database is manifested as: joint storage of vector embedding and metadata, associating the vectors of the simulation result graph with scenario labels and device model metadata, and supporting hybrid retrieval, narrowing the search scope through metadata filtering.

6. The method according to claim 1, characterized in that, The knowledge extraction method includes: semantic alignment, entity relationship extraction, and cross-modal knowledge fusion; The semantic alignment includes explicit alignment, implicit alignment, and semantic space mapping; the explicit alignment defines semantic type constraints through a metadata dictionary to establish mapping rules between structured data fields and unstructured data labels; the implicit alignment adopts a cross-modal attention mechanism of the Transformer architecture to dynamically learn the associations of multi-modal data sub-components during training; the semantic space mapping maps text descriptions, time series data, and simulation images to a unified vector space; The entity relationship extraction uses named entity recognition technology to extract entities from text, and determines the relationships between entities with the help of dependency syntactic analysis; The cross-modal knowledge fusion associates the entities and relationships in different modal data to generate a unified knowledge representation.

7. The method according to claim 1, wherein The knowledge fusion includes: entity disambiguation, attribute fusion, and relationship fusion; the entity disambiguation is used to solve the reference differences of the same entity in multi-source data, and force-align the entity names in structured and unstructured data through a predefined domain synonym table; the attribute fusion is used to integrate the multi-source attribute conflicts of the same entity, and override the conflicting attribute values according to the defined attribute priorities; the relationship fusion is used to unify the association relationships in multi-modal data, and map semantically similar relationships to a low-dimensional space using a graph embedding model.

8. The method according to claim 1, wherein The graph database stores the fused entities and relationships using an attribute graph model, specifically including: Defining node types and relationship types according to the metadata model, and supporting dynamic addition of new entity types; establishing composite indexes for frequently queried attributes, and accelerating complex traversal queries through label-attribute joint indexes.

9. The method according to claim 1, characterized in that, The construction method of the simulation knowledge graph is: When new simulation data flows in, the knowledge graph nodes, relationships, and attributes are updated in real time through an online entity alignment model and an incremental graph embedding algorithm; knowledge graph version snapshots are generated regularly to record the entity relationship change history, and support backtracking or comparing differences along the time axis; Constructing cross-modal entities into hyperedge relationships to represent composite semantic associations, attaching multi-modal feature vectors to the graph nodes, and establishing a joint index with the vector database; using a dual storage engine to save the basic graph and the incremental graph respectively.

10. The method according to claim 1, wherein The integration of the semantic extended search and the enterprise business process includes: Defining semantic extension trigger rules based on the business process. When the user inputs a query condition, the query intent is parsed through natural language processing and automatically associated with the scenario labels and device model metadata in the knowledge graph to generate a multi-modal joint query vector; Establishing a cross-modal joint index in the vector database, performing hierarchical semantic retrieval with the scenario label as the filtering condition, preferentially matching the simulation cases with completely aligned metadata, and then extending to cross-modal associated cases through vector similarity; Dynamically fusing the retrieval results with the historical decision parameters in the business process, and using a graph neural network to construct a simulation plan comparison matrix to automatically label the performance difference indicators and applicable scenario constraint conditions between the plans.

Images