Assembling process knowledge graph establishing and updating method based on graph attention network mapping

Abstract

The invention discloses an assembly process knowledge graph construction and updating method based on graph attention network mapping, and the method mainly comprises the following steps: standardization system design construction, data collection, data classification, multi-modal data preprocessing, multi-source information extraction and graph node feature initialization. Semantic modeling and vector space mapping driven by a graph attention network, knowledge storage, construction of an assembly process knowledge graph by using stored knowledge data, and semantic complementation and dynamic updating of a process information model. Through the unique self-attention mechanism of the GAT, the weight can be dynamically distributed for the relationship among different information nodes, so that knowledge reasoning and expression are more accurately carried out, and the problems of knowledge fragmentation, management rigidity, semantic understanding deficiency, low knowledge utilization efficiency and the like in the prior art are systematically solved.

Description

Technical Field

[0001] This invention relates to the field of technical information processing in complex equipment manufacturing, and in particular to a method for establishing and updating assembly process knowledge graphs based on graph attention network mapping. Background Technology

[0002] The main disadvantages of existing technologies are as follows:

[0003] 1. Information "Black Box" and Knowledge Fragmentation: Although existing manufacturing information systems (such as MES / ERP) can store and manage production data, the data is often isolated. For example, the design requirements of a part, its corresponding assembly process, the tools used, and historical failure records are scattered across different systems, forming knowledge fragments. The complex logical relationships within these fragments cannot be explicitly expressed and utilized, making the entire process flow appear as an "information black box" to the user.

[0004] 2. Knowledge management is static and rigid, making it difficult to adapt to flexible production: Traditional process management methods are mostly based on static documents or database tables. When faced with frequent product changes or process optimization, knowledge updates and maintenance heavily rely on manual operations, which are cumbersome, time-consuming, and prone to errors. This rigid management model cannot meet the requirements of modern manufacturing for rapid response and high flexibility.

[0005] 3. Lack of deep semantic understanding limits intelligent applications: Traditional technologies primarily process data at the data level, lacking an understanding of the deeper semantics behind the data. The system doesn't "understand" the process and cannot perform knowledge-based reasoning and decision-making. For example, when a new problem arises, the system cannot analyze its root cause like an expert, by relating it to historical cases, equipment principles, and process flows. This significantly restricts the development and implementation of advanced intelligent applications such as fault diagnosis and process optimization.

[0006] 4. Low knowledge utilization efficiency and heavy reliance on expert experience: Due to the difficulty in acquiring knowledge and its lack of a systematic approach, new employee training cycles are lengthy, and on-site problem-solving heavily relies on the experience of a few senior experts. Knowledge cannot be effectively accumulated, inherited, and reused, making it difficult to improve the overall knowledge management level and operational efficiency of the enterprise.

[0007] Therefore, existing technologies need further improvement and refinement. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for establishing and updating assembly process knowledge graphs based on graph attention network mapping.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] The method for constructing and updating assembly process knowledge graphs based on graph attention network mapping mainly includes the following specific steps:

[0011] Multimodal data fusion under human-machine collaborative assembly line production changeover

[0012] Step S1: Construction of Standardization System Design: In order to solve the problems of diverse data sources and inconsistent standards, we first conduct a target and demand practicality analysis of the target assembly field, and on this basis, we carry out standard system structure design and standard compilation.

[0013] Step S2: Data Acquisition: Extract information from three major data sources—text documents, business systems, and expert experience—using relevant interfaces and parsers according to the standards of Step S1.

[0014] Step S3: Data Classification: The data extracted in the above steps is classified into static data or dynamic data types; static data refers to relatively fixed data that serves as a benchmark or historical record; dynamic data refers to data that changes in real time and reflects the current state.

[0015] Step S4: Multimodal data preprocessing: Clean, mine, and analyze the classified data to obtain corresponding rules, mechanisms, and cases, and use these as a high-quality knowledge source for subsequent steps;

[0016] Process Relationship Modeling and Semantic Mapping Based on Graph Attention

[0017] Step S5: Multi-source information extraction and graph node feature initialization: Different extraction techniques are used for different data modalities to extract key information from the data of different modalities, and then alignment and mapping are performed. Finally, the mapped data is used as the initial feature vector of the corresponding graph node.

[0018] Step S6: Semantic Modeling and Vector Space Mapping Driven by Graph Attention Network: This step is the core of "modeling and mapping". The initial graph structure and node features generated in step S5 are used for end-to-end deep learning through a graph attention network to generate a high-dimensional semantic vector representation. This representation adopts a two-layer representation architecture, including symbol layer representation and vector layer representation.

[0019] Step S7: Knowledge Storage: Store the knowledge modeled by the graph attention network in a unified manner; use a graph database to store the knowledge triples of the symbolic layer and the entity / relation embeddings of the vector layer, and provide users with an efficient query interface through the database;

[0020] Step S8: Construct an assembly process knowledge graph using the knowledge data stored in Step S7, including intelligent process Q&A, predictive maintenance, fault cause analysis, and production process traceability;

[0021] Step S9: Semantic completion and dynamic update of the process information model: By using semantic completion based on vector representation and dynamic knowledge update, the process information model constructed in step S6 can continuously improve and grow through reasoning completion and event-driven processes, providing truly dynamic and reliable knowledge support for upper-level applications.

[0022] As a preferred embodiment of the present invention, the semantic completion based on vector representation in step S9 includes:

[0023] Link prediction: Using the knowledge vector representation generated in step S6, a link prediction model is trained; the model learns the pattern of existing knowledge triples and calculates the confidence score of forming a new link between any two entities; when the score exceeds a preset threshold, the link will be added to the graph as inferred new knowledge, thereby enriching the knowledge base.

[0024] Entity alignment: At the same time, entity alignment technology is used to automatically discover and merge different entities that refer to the same physical object by calculating the similarity between entity vectors, thus ensuring the uniqueness of knowledge.

[0025] As a preferred embodiment of the present invention, the knowledge dynamic update method in step S9 includes:

[0026] Incremental update mechanism: Deploy an update module based on a temporal graph network; this module listens to the real-time data stream, and when a new event is detected, it triggers a transaction, adding the event and its related information as new nodes or edges incrementally to the graph database; at the same time, the temporal graph network model updates the vector representations of the local nodes affected by this event, rather than retraining the entire system.

[0027] Knowledge Verification and Calibration: While knowledge is being incrementally updated, a verification engine based on SHACL (Shapes Constraint Language) or SPARQL CONSTRUCT runs simultaneously. This engine performs consistency checks on newly added or inferred knowledge according to predefined ontology constraints and business rules. Conflicting or inconsistent knowledge will be marked or automatically corrected according to preset strategies, thereby ensuring the logical consistency and high quality of the knowledge graph.

[0028] As a preferred embodiment of the present invention, the construction of the standardization system design in step S1 includes the construction of a standardized terminology dictionary, the construction of a standardized ontology design, the construction of a standardized service technology system, and the construction of standards for typical application scenarios.

[0029] The construction of the standardized terminology dictionary refers to the analysis and construction of an industry knowledge standardized terminology dictionary for the field of complex equipment assembly, defining core entities, synonyms, and abbreviations.

[0030] The construction of the standardized ontology refers to designing and constructing a standardized ontology that describes the concepts, attributes and their interrelationships in the assembly domain, and studying the fusion method of multi-source heterogeneous ontology to provide a unified semantic framework for the structured representation of knowledge.

[0031] The construction of the standardized service technology system refers to the establishment of technical standards and specifications for data collection, cleaning, storage, management, and external service provision. This system forms a closed-loop management system through promotion, feedback, maintenance, and improvement.

[0032] The construction of typical application scenario standards refers to the process of pre-defining and standardizing the input, output, interaction logic, and evaluation criteria of the core applications planned to be supported after the knowledge graph is built. This step aims to ensure that the knowledge graph can accurately and efficiently serve the final business needs, providing a clear and standardized framework and interfaces for the development and integration of upper-layer applications.

[0033] As a preferred embodiment of the present invention, the data cleaning in step S4 refers to the automated cleaning of the collected static data, including removing irrelevant information, unifying the data format, filling in missing values, and correcting errors based on predefined rules.

[0034] The data mining and analysis refers to the preliminary offline or online analysis of dynamic data; including using association rule mining algorithms to analyze the potential correlations between process parameters, and the strong association rules mined can be regarded as the direct basis for constructing "candidate edges" in the graph; using clustering algorithms to classify equipment operating conditions, and the classification results can be used as dynamic state characteristics of equipment nodes, greatly enriching the information content of the nodes; and using regression analysis to establish a preliminary mathematical model between parameters and quality indicators.

[0035] As a preferred embodiment of the present invention, step S5 further includes:

[0036] Step S51: Structured extraction of text data:

[0037] Entity recognition: Deploy a named entity recognition model that integrates a pre-trained language model and is fine-tuned with assembly domain corpus. This model can automatically and efficiently identify core process entities from unstructured text such as maintenance logs and operation manuals.

[0038] Relationship modeling: The entities identified by the named entity recognition model are used as input and fed into a relation extraction model based on a graph attention network for modeling, thereby forming the initial skeleton of the graph.

[0039] Step S52: Feature vectorization for non-textual multimodal data:

[0040] To integrate non-textual information from images and time-series signals, this method employs a multimodal collaborative learning framework, transforming them into numerical feature vectors usable by graph attention networks. These vectors include: image data, which extracts visual features through convolutional neural networks to generate feature vectors representing image content; and time-series signals, which extract time-series features through recurrent neural networks or their variants to capture dynamic patterns of device operation.

[0041] Step S53: Perform cross-modal feature alignment. Through a joint embedding space, the feature vectors extracted from the non-text data are aligned and mapped with the entities extracted from the text. Finally, these vectors will serve as the initial feature vectors of the corresponding graph nodes, greatly enriching the information dimension of the nodes and providing a data foundation for graph attention networks to perform deeper reasoning.

[0042] As a preferred embodiment of the present invention, the two-layer representation architecture in step S6 includes:

[0043] Symbolic layer representation: Around the production factors of people-machine-material-method-environment, all extracted knowledge triples are mapped to the domain ontology defined in step S1, following the RDF standard; this step ensures that all knowledge has clear semantics that conform to industry standards.

[0044] Vector layer representation: Modeling is performed using a graph attention network. The process takes the initial graph structure containing nodes and edges constructed in step S1, as well as the initial feature vectors of each node that incorporate multimodal information, as input.

[0045] As a preferred embodiment of the present invention, the vector layer representation in step S6 further includes the following processing steps:

[0046] Step S61: Feature Transformation and Attention Coefficient Calculation:

[0047] For each node i and its neighbor node j in the graph, feature transformation is first performed; assuming the initial feature vectors of node i and node j are h and h respectively. i and h j And map these features to a higher-dimensional space:

[0048] h′ i =Wh i , h′ j =Wh j Formula 1

[0049] In the formula, W is a learnable weight matrix.

[0050] Next, the feature vectors of node i and node j are concatenated and fed into a single-layer fully connected neural network to calculate the unnormalized attention coefficient e. ij :

[0051] e ij =LeakyReLU(a T [h′ i ||h′ j ]) Formula 2

[0052] In the formula, a is a learnable attention weight vector, || represents the vector concatenation operation, and LeakyReLU represents the activation function.

[0053] Step S62: Normalization of attention weights:

[0054] To ensure the comparability of attention coefficients between nodes, the Softmax function is used to normalize the unnormalized attention coefficients for all neighboring nodes j of node i:

[0055]

[0056] In the formula, N(i) represents the set of neighbors of node i, and a ij It is the attention weight of the relationship between node j and node i; the masking mechanism in this step ensures that only the coefficients of neighboring nodes are involved in the calculation.

[0057] Step S63: Weighted aggregation of neighbor features:

[0058] After calculating the attention weights of all neighboring nodes, the neighbor features are aggregated; for each neighbor j, its feature vector h is used. j Perform a linear transformation and multiply by the corresponding attention weight 'a'. ij Then sum all the weighted features:

[0059]

[0060] In the formula, h′ i It is the aggregated feature vector of node i in the current layer.

[0061] Step S64: Multi-head attention mechanism:

[0062] To capture the relationship information between nodes from different perspectives, a multi-head attention mechanism is adopted; for each head k, a different weight matrix W is used. k and attention weight vector a k To calculate the attention coefficient and feature aggregation:

[0063]

[0064] The aggregated features of each head are:

[0065]

[0066] Finally, the output features of all heads are concatenated or averaged to obtain the final node feature representation:

[0067]

[0068] Step S65: Stacking multiple layers of graph attention networks:

[0069] To capture more complex long-range dependencies, multiple GAT layers need to be stacked. Each GAT layer is calculated based on the output of the previous layer, thereby gradually aggregating information from more distant neighbors. Through multi-layer stacking, nodes can gradually integrate more layers of semantic information.

[0070] Step S66: Finally, the graph attention network model transforms the feature representation of each node into a high-dimensional semantic vector h. i This vector represents the semantic information carried by node i in the graph and serves as input for subsequent steps.

[0071] The working process and principle of this invention are as follows: The technical solution of this invention aims to transform the original multimodal assembly data into a dynamic and intelligent process information model through a systematic process, the core of which is to use Graph Attention Network (GAT) for deep modeling and intelligent mapping. Through GAT's unique self-attention mechanism, this invention can dynamically assign weights to the relationships between different information nodes, thereby enabling more accurate knowledge reasoning and representation, and systematically solving the problems of knowledge fragmentation, rigid management, lack of semantic understanding, and inefficient knowledge utilization in existing technologies.

[0072] Compared with the prior art, the present invention also has the following advantages:

[0073] (1) The assembly process knowledge graph construction and updating method based on graph attention network mapping provided by the present invention can overcome knowledge fragmentation. Through multimodal data fusion and knowledge extraction, isolated process data can be integrated into a unified, comprehensive and deeply related assembly process knowledge graph.

[0074] (2) The assembly process knowledge graph construction and updating method based on graph attention network mapping provided by the present invention can overcome management rigidity. By establishing a dynamic updating and self-evolution mechanism for the knowledge graph, the knowledge system can change synchronously with the physical production line, realizing flexible and adaptive management of knowledge.

[0075] (3) The assembly process knowledge graph construction and updating method based on graph attention network mapping provided by the present invention can overcome the lack of semantic understanding. By introducing graph attention network, it can automatically focus on key process links and parameters when analyzing problems, just like a domain expert, thereby overcoming the limitation of traditional technology lacking deep semantic understanding.

[0076] (4) The assembly process knowledge graph construction and updating method based on graph attention network mapping provided by the present invention can overcome the inefficiency of knowledge utilization. It can accurately model knowledge relationships through graph attention network, transform implicit and scattered expert experience into explicit models that can be understood and reasoned by machines, and build an enterprise-level knowledge base that is easy to query, reason and reuse, significantly improving the efficiency of knowledge inheritance and utilization. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of the structure and process of the assembly process knowledge graph construction and updating method based on graph attention network mapping provided by the present invention.

[0078] Figure 2 This is a schematic diagram of the knowledge extraction process of the assembly process knowledge graph construction and updating method based on graph attention network mapping provided by the present invention.

[0079] Figure 3 This is a schematic diagram of the knowledge dynamic update process of the assembly process knowledge graph construction and update method based on graph attention network mapping provided by the present invention. Detailed Implementation

[0080] To make the objectives, technical solutions, and advantages of the present invention clearer and more explicit, the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0081] Terminology Explanation:

[0082] 1. Graph Attention Network (GAT): A novel neural network architecture based on graph-structured data, which utilizes hidden self-attention layers to address the shortcomings of previous graph convolutional or approximation-based methods. Through stacked layers, nodes can participate in the features of their neighbors, and can (implicitly) assign different weights to different nodes in the neighborhood.

[0083] 2. Multimodal data: refers to data sets from different modalities. In the manufacturing assembly scenario, it usually includes structured business system data, semi-structured process cards, unstructured technical manuals and maintenance logs (text), as well as sensor monitoring data (time series signals), equipment operating condition images, etc.

[0084] 3. Link Prediction: Based on the existing connections in the graph, a graph learning model is used to predict the probability of an unknown link (relationship) between two nodes. In this invention, this is used to complete missing steps in the process flow or to discover potential process relationships.

[0085] Example 1:

[0086] like Figures 1 to 3As shown, this embodiment aims to solve the technical problem of how to effectively integrate, model, and map massive, heterogeneous, and multimodal assembly process information in modern complex equipment manufacturing. These technical problems mainly include:

[0087] 1. How to standardize and integrate structured, semi-structured and unstructured process data that are scattered across different business systems (such as MES, PLM), technical documents, operation manuals and sensor terminals, break down information silos and build a unified and interconnected knowledge view.

[0088] 2. How to automatically and accurately extract knowledge elements with clear semantics (such as entities like equipment, materials, processes, parameters, and faults, and the complex relationships between them) from the fused massive data, and transform the tacit knowledge that exists in a large amount of text and expert experience into explicit knowledge that is machine-readable and understandable.

[0089] 3. How to utilize the dynamic perception capability of graph attention networks for node relationships to construct a dynamically evolving knowledge model that can respond in real time to changes in the production site (such as product changeover, process adjustment, and equipment upgrade), and automatically fill in missing knowledge and verify the accuracy of knowledge through reasoning and learning mechanisms, thereby maintaining the integrity and timeliness of the knowledge base.

[0090] 4. How to provide a solid, semantic knowledge foundation for upper-level intelligent applications (such as intelligent process question answering, root cause analysis of failures, predictive maintenance, production process traceability, etc.) based on a knowledge model accurately constructed by graph attention network, thereby improving the intelligent decision-making and execution capabilities of the assembly line.

[0091] This embodiment comprehensively utilizes cutting-edge artificial intelligence technologies such as ontology, natural language processing, graph attention networks, and multimodal learning. It aims to systematically solve the challenges of knowledge acquisition, representation, fusion, reasoning, and dynamic management in the assembly field, thereby constructing a comprehensive, accurate, and dynamically evolving assembly process knowledge brain, providing core support for achieving flexible and intelligent manufacturing. The specific technical steps are as follows:

[0092] 1. Multimodal data fusion in human-machine collaborative assembly line production changeover

[0093] This module is the cornerstone of the entire technical solution, and its core task is to provide high-quality, structured input for subsequent Graph Attention Network (GAT) modeling. This process not only involves collecting and standardizing raw data from diverse sources and in various forms, but more importantly, it involves initially constructing an initial graph structure containing nodes, edges, and rich features.

[0094] 1.1 Standardization System Construction and Data Collection

[0095] Standardization System Design: To address the issues of diverse data sources and inconsistent standards, the first step is to conduct a practicality analysis of the target assembly domain, including objectives and requirements. Based on this analysis, a standard system architecture design and standard development are carried out. This process aims to establish a complete set of knowledge management specifications, with core outputs including a standardized terminology dictionary, a standardized ontology design (defining concepts, attributes, and relationships within the assembly domain), a standardized service technology system, and standards for typical application scenarios.

[0096] Among them, the construction of a standardized terminology dictionary is aimed at the field of complex equipment assembly. It involves analyzing and building a standardized terminology dictionary for industry knowledge, defining core entities, synonyms, abbreviations, etc., to solve the problem of inconsistent terminology. This step is crucial because it directly defines the unique identifier and semantic label of all nodes in the subsequent graph model, laying the foundation for GAT to accurately identify and process different process entities (such as equipment, materials, and processes).

[0097] Standardized ontology construction and fusion involves designing and building a standardized ontology that describes concepts, attributes, and their interrelationships in the assembly domain, and researching methods for fusing multi-source heterogeneous ontology to provide a unified semantic framework for the structured representation of knowledge. This ontology clarifies the possible relationship types between entities (such as 'containment', 'cause', and 'use'), which is equivalent to predefining the types and patterns of edges in a graph model. The standardized management and service technology system establishes technical standards and specifications for data acquisition, cleaning, storage, management, and external service provision. This system forms a closed-loop management system through promotion, feedback, maintenance, and improvement.

[0098] Data Acquisition and Classification: All collected data adheres to the aforementioned standardized system. This solution primarily targets three sources for data acquisition: text documents, business systems, and expert experience. The system extracts information from these sources through targeted interfaces and parsers, classifying it into the following two main data types:

[0099] Static data refers to relatively fixed data that serves as a benchmark or historical record. In this solution, static data specifically includes CNC programs extracted from technical documents, welding procedures extracted from business manuals, and various process records and maintenance logs.

[0100] Dynamic data refers to data that changes in real time and reflects the current state. In this solution, dynamic data specifically covers process parameters during production (such as spinning and temperature), monitoring data from sensors (such as real-time load), and working conditions and machine tool utilization rates that reflect the on-site situation.

[0101] 1.2 Multimodal Data Preprocessing

[0102] Data cleaning: Automated cleaning of collected static data (such as technical documents, business manuals, process records, etc.), including removing irrelevant information (such as HTML tags), standardizing data format, filling in missing values, and correcting errors based on predefined rules.

[0103] Data Mining and Analysis: This step aims to proactively discover potential graph structure information and enrich node features from raw data, providing more valuable input for GAT. Preliminary offline or online analysis is performed on dynamic data (such as process parameters, monitoring data, and operating environment). Association rule mining algorithms (such as Apriori) are used to analyze the potential correlations between process parameters. Strongly correlated rules (e.g., the simultaneous occurrence of 'parameter exceeding limits' and 'fault') can be considered as direct evidence for constructing "candidate edges" in the graph. Clustering algorithms (such as K-Means) are used to classify equipment operating conditions. The classification results (e.g., 'high load state', 'idle state') can serve as dynamic state features of equipment nodes, greatly enriching the information content of the nodes. Regression analysis is used to establish a preliminary mathematical model between parameters and quality indicators. The rules, mechanisms, and cases derived from the analysis will serve as high-quality knowledge sources and be input into the next module.

[0104] 2. Process Relationship Modeling and Semantic Mapping Based on Graph Attention

[0105] This module is the core of the entire technical solution. It is responsible for transforming the multimodal data fused and preprocessed in the above steps into an intelligent information model that contains deep process logic and is machine-understandable. Its core is to utilize the powerful context awareness and relational reasoning capabilities of graph attention networks to achieve accurate modeling and high-dimensional semantic mapping of complex assembly process relationships.

[0106] 2.1 Multi-source information extraction and graph node feature initialization

[0107] In order to construct an information-rich graph that can be processed by GAT, this step first extracts key information from data of different modalities and uses it as the initial features of graph nodes. Appropriate extraction techniques are adopted for different data modalities.

[0108] (1) Structured extraction of text data

[0109] Entity Recognition: Deploy a Named Entity Recognition (NER) model that integrates a pre-trained language model (such as BERT) and is fine-tuned with assembly domain corpora. This model can automatically and efficiently identify core process entities from unstructured text such as maintenance logs and operation manuals, such as equipment name (machine tool A), component type (spindle, bearing B), fault phenomenon (abnormal noise), operation type (replacement), etc.

[0110] Relationship Modeling: Entities identified by NER are used as input and fed into a relation extraction model based on Graph Attention Network (GAT). Unlike traditional methods, GAT can simulate the analytical logic of domain experts. Through its self-attention mechanism, it dynamically assigns different importance weights to different contextual words in a sentence. For example, when analyzing "The spindle of machine tool A made an abnormal noise, which was confirmed to be caused by the wear of bearing B," GAT can focus on key verbs and prepositions such as "caused" and "occurred at," thereby accurately extracting deep logical relation triples such as (spindle, located at, machine tool A), (abnormal noise, the cause is, wear), and (wear, occurred at, bearing B), forming the initial skeleton of the graph.

[0111] (2) Feature vectorization of other multimodal data besides text data

[0112] To integrate non-textual information such as images and time-series signals, this solution employs a multimodal collaborative learning framework to transform it into numerical feature vectors usable by GAT.

[0113] Image data: Visual features are extracted using a convolutional neural network (CNN) to generate feature vectors representing the image content.

[0114] Time series signals: Time series features are extracted using recurrent neural networks (RNNs) or their variants (such as LSTMs) to capture dynamic patterns in the operating state of equipment.

[0115] Then, cross-modal feature alignment is performed. Through a joint embedding space, these feature vectors extracted from non-textual data are aligned and mapped with entities extracted from text. Ultimately, these vectors serve as the initial feature vectors for the corresponding graph nodes, greatly enriching the information dimension of the nodes and providing a data foundation for GAT to perform deeper reasoning.

[0116] 2.2 GAT-Driven Semantic Modeling and Vector Space Mapping

[0117] This step is the core of "modeling and mapping," taking the initial graph structure and node features generated in 2.1 and using GAT for end-to-end deep learning to ultimately generate a high-dimensional semantic vector representation. A two-layer representation architecture is employed:

[0118] Symbolic layer representation: Around the production factors of people, machines, materials, methods, and environment, all extracted knowledge triples are mapped onto the domain ontology defined in Module 1, following the RDF standard. This step ensures that all knowledge has clear, industry-standard semantics.

[0119] Vector layer representation: This invention utilizes GAT for modeling. The process takes the initial graph structure containing nodes and edges constructed in Module 1, as well as the initial feature vectors of each node incorporating multimodal information, as input. The specific steps are as follows:

[0120] (1) Feature transformation and attention coefficient calculation:

[0121] For each node i and its neighbor node j in the graph, feature transformation is performed first. Assume the initial feature vectors of node i and node j are h and h, respectively. i and h j And map these features to a higher-dimensional space:

[0122] h′ i =Wh i , h′ j =Wh j

[0123] Here, W is a learnable weight matrix.

[0124] Next, the feature vectors of node i and node j are concatenated and fed into a single-layer fully connected neural network to calculate the unnormalized attention coefficient e. ij :

[0125] e ij =LeakyReLU(a T [h′ i ||h′ j ])

[0126] Where a is a learnable attention weight vector, || denotes the vector concatenation operation, and LeakyReLU is the activation function.

[0127] (2) Normalization of attention weights:

[0128] To ensure the comparability of attention coefficients among nodes, the Softmax function is used to normalize the unnormalized attention coefficients. For all neighboring nodes j of node i:

[0129]

[0130] Where N(i) represents the set of neighbors of node i, a ij This represents the attention weight of the relationship between node j and node i. The masking mechanism in this step ensures that only the coefficients of neighboring nodes are included in the calculation.

[0131] (3) Weighted aggregation of neighbor features:

[0132] After calculating the attention weights of all neighboring nodes, the neighbor features are aggregated. For each neighbor j, its feature vector h is used.j Perform a linear transformation and multiply by the corresponding attention weight 'a'. ij Then sum all the weighted features:

[0133]

[0134] Where, h′ i It is the aggregated feature vector of node i in the current layer.

[0135] (4) Multi-head attention mechanism:

[0136] To capture the relationship information between nodes from different perspectives, a multi-head attention mechanism is employed. For each head k, a different weight matrix W is used. k and attention weight vector a k To calculate the attention coefficient and feature aggregation:

[0137]

[0138] The aggregated features of each head are:

[0139]

[0140] Finally, the output features of all heads are concatenated or averaged to obtain the final node feature representation:

[0141]

[0142] (5) Stacking multiple layers of GAT

[0143] To capture more complex long-range dependencies, GATs typically stack multiple GAT layers. Each GAT layer is computed based on the output of the previous layer, thus progressively aggregating information from more distant neighbors. Through multi-layer stacking, nodes can gradually integrate more layers of semantic information.

[0144] (6) Final output

[0145] Ultimately, the GAT model transforms the feature representation of each node into a high-dimensional semantic vector h. i This vector represents the semantic information carried by node i in the graph. These vectors will serve as input for subsequent tasks.

[0146] Knowledge storage: Knowledge modeled using GAT is stored uniformly. Specialized graph databases (such as Neo4j and OrientDB) are used to store symbolic layer knowledge triples (for precise queries) and vector layer entity / relation embeddings (for advanced tasks such as semantic similarity calculation and link prediction). The database provides efficient query interfaces through languages ​​such as SPARQL or Cypher.

[0147] 3. Semantic completion and dynamic updating of process information model

[0148] This module aims to ensure that the process information model built in Module 2 possesses self-improvement and self-adaptation capabilities, making it not merely a static knowledge snapshot, but a "living" model capable of learning, reasoning, and evolving in sync with the physical production line. This is primarily achieved through two mechanisms: semantic completion and dynamic evolution.

[0149] 3.1 Semantic completion based on vector representation

[0150] The goal of semantic completion is to use the existing knowledge of the model to automatically reason and fill in the potentially missing information, thereby enriching and improving the entire process information model.

[0151] Link prediction: Using the knowledge vector representation generated in step 2.2, a link prediction model is trained. This model (e.g., a scoring function based on tensor decomposition or a graph neural network) learns patterns from existing knowledge triples and calculates a confidence score for a new link (i.e., a new relationship) between any two entities. When the score exceeds a preset threshold, the link is added to the graph as inferred new knowledge, thereby enriching the knowledge base.

[0152] Entity alignment: At the same time, entity alignment technology is used to automatically discover and merge different entities that refer to the same physical object by calculating the similarity between entity vectors, thus ensuring the uniqueness of knowledge.

[0153] 3.2 Dynamic updating of knowledge

[0154] To enable the process information model to reflect the latest status of the production line in real time, an event-driven dynamic evolution mechanism was designed, giving it the ability to change synchronously with the physical world.

[0155] Incremental update mechanism: Deploy an update module based on Temporal Graph Network (TGN). This module listens to the real-time data stream, and when it detects a new event (such as a production change operation or equipment maintenance completion), it triggers a transaction, incrementally adding the event and its related information as new nodes or edges to the graph database. Simultaneously, the TGN model efficiently updates the vector representations of the local nodes affected by this event, rather than retraining the entire system.

[0156] Knowledge Validation and Calibration: Simultaneously with incremental knowledge updates, a validation engine based on SHACL (Shapes Constraint Language) or SPARQL CONSTRUCT runs. This engine performs consistency checks on newly added or inferred knowledge according to predefined ontology constraints and business rules. Conflicting or inconsistent knowledge is flagged or automatically corrected according to preset strategies, thereby ensuring the logical consistency and high quality of the knowledge graph.

[0157] Through the above mechanism, the process information model constructed by this invention is no longer a static database, but a knowledge graph that can continuously improve and grow through reasoning completion and event-driven processes, providing truly dynamic and reliable knowledge support for upper-level applications.

[0158] In summary, the technical solution of this embodiment aims to transform raw multimodal assembly data into a dynamic and intelligent process information model through a systematic process, with the core being the use of Graph Attention Networks (GAT) for deep modeling and intelligent mapping. Through GAT's unique self-attention mechanism, this embodiment can dynamically assign weights to the relationships between different information nodes, thereby enabling more accurate knowledge reasoning and representation. This systematically addresses the problems of knowledge fragmentation, rigid management, lack of semantic understanding, and inefficient knowledge utilization in existing technologies.

[0159] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for constructing and updating an assembly process knowledge graph based on graph attention network mapping, characterized in that, Includes the following steps: Phase 1: Multimodal Data Fusion under Human-Machine Collaborative Assembly Line Changeover Step S1: Construction of Standardization System Design: In order to solve the problems of diverse data sources and inconsistent standards, we first conduct a target and demand practicality analysis of the target assembly field, and on this basis, we carry out standard system structure design and standard compilation. Step S2: Data Acquisition: Extract information from three major data sources—text documents, business systems, and expert experience—using relevant interfaces and parsers according to the standards of Step S1. Step S3: Data Classification: The data extracted in the above steps is classified into static data or dynamic data types; static data refers to relatively fixed data that serves as a benchmark or historical record; dynamic data refers to data that changes in real time and reflects the current state. Step S4: Multimodal data preprocessing: Clean, mine, and analyze the classified data to obtain corresponding rules, mechanisms, and cases, and use these as a high-quality knowledge source for subsequent steps; Phase Two: Graph Attention-Based Modeling of Process Relationships and Semantic Mapping Step S5: Multi-source information extraction and graph node feature initialization: Different extraction techniques are used for different data modalities to extract key information from the data of different modalities, and then alignment and mapping are performed. Finally, the mapped data is used as the initial feature vector of the corresponding graph node. Step S6: Semantic Modeling and Vector Space Mapping Driven by Graph Attention Network: This step is the core of "modeling and mapping". The initial graph structure and node features generated in step S5 are used for end-to-end deep learning through a graph attention network to generate a high-dimensional semantic vector representation. This representation adopts a two-layer representation architecture, including symbol layer representation and vector layer representation. Step S7: Knowledge Storage: Store the knowledge modeled by the graph attention network in a unified manner; use a graph database to store the knowledge triples of the symbolic layer and the entity / relation embeddings of the vector layer, and provide users with an efficient query interface through the database; Step S8: Construct an assembly process knowledge graph using the knowledge data stored in Step S7, including intelligent process Q&A, predictive maintenance, fault cause analysis, and production process traceability; Phase 3: Semantic Completion and Dynamic Update of the Process Information Model Step S9: By using semantic completion based on vector representation and dynamic knowledge updates, the process information model constructed in step S6 can continuously improve and grow through reasoning completion and event-driven processes, providing truly dynamic and reliable knowledge support for upper-level applications.

2. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 1, characterized in that, The semantic completion based on vector representation in step S9 includes: Link prediction: Using the knowledge vector representation generated in step S6, a link prediction model is trained; the model learns the pattern of existing knowledge triples and calculates the confidence score of forming a new link between any two entities; when the score exceeds a preset threshold, the link will be added to the graph as inferred new knowledge, thereby enriching the knowledge base; Entity alignment: At the same time, entity alignment technology is used to automatically discover and merge different entities that refer to the same physical object by calculating the similarity between entity vectors, thus ensuring the uniqueness of knowledge.

3. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 1, characterized in that, The knowledge dynamic update method in step S9 includes: Incremental update mechanism: Deploy an update module based on a temporal graph network; this module listens to the real-time data stream, and when a new event is detected, it triggers a transaction, adding the event and its related information as new nodes or edges incrementally to the graph database; at the same time, the temporal graph network model updates the vector representations of the local nodes affected by this event, instead of retraining the entire system. Knowledge Verification and Calibration: While knowledge is being incrementally updated, a verification engine based on SHACL or SPARQL CONSTRUCT runs simultaneously. This engine performs consistency checks on newly added or inferred knowledge according to predefined ontology constraints and business rules. Conflicting or inconsistent knowledge will be marked or automatically corrected according to preset strategies, thereby ensuring the logical consistency and high quality of the knowledge graph.

4. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 1, characterized in that, The construction of the standardization system design in step S1 includes the construction of a standardized terminology dictionary, the construction of a standardized ontology design, the construction of a standardized service technology system, and the construction of standards for typical application scenarios; The construction of the standardized terminology dictionary refers to the analysis and construction of a standardized terminology dictionary for the field of complex equipment assembly, defining core entities, synonyms, and abbreviations. The construction of the standardized ontology refers to designing and constructing a standardized ontology that describes the concepts, attributes and their interrelationships in the assembly domain, and studying the fusion method of multi-source heterogeneous ontology to provide a unified semantic framework for the structured representation of knowledge. The construction of the standardized service technology system refers to establishing technical standards and specifications for data collection, cleaning, storage, management, and external service provision. This system forms a closed-loop management system through promotion, feedback, maintenance, and improvement. The construction of typical application scenario standards refers to the process of pre-defining and standardizing the input, output, interaction logic, and evaluation criteria of the core applications that are planned to be supported after the knowledge graph is built.

5. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 1, characterized in that, The data cleaning in step S4 refers to the automated cleaning of the collected static data, including removing irrelevant information, unifying the data format, filling in missing values, and correcting errors based on predefined rules. The data mining and analysis refers to the preliminary offline or online analysis of dynamic data; including using association rule mining algorithms to analyze the potential correlation between process parameters, and the strong association rules mined can be regarded as the direct basis for constructing "candidate edges" in the graph; using clustering algorithms to classify equipment operating conditions, and the classification results can be used as dynamic state characteristics of equipment nodes, greatly enriching the information content of the nodes; and using regression analysis to establish a preliminary mathematical model between parameters and quality indicators.

6. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 1, characterized in that, Step S5 further includes: Step S51: Structured extraction of text data: Entity recognition: Deploy a named entity recognition model that integrates a pre-trained language model and is fine-tuned with assembly domain corpus. This model can automatically and efficiently identify core process entities from unstructured texts such as maintenance logs and operation manuals. Relationship modeling: The entities identified by the named entity recognition model are used as input and fed into a relation extraction model based on graph attention network for modeling, thereby forming the initial skeleton of the graph; Step S52: Feature vectorization for non-textual multimodal data: To integrate non-textual information from images and time-series signals, this method employs a multimodal collaborative learning framework, transforming it into numerical feature vectors usable by graph attention networks. These vectors include: image data, which extracts visual features through convolutional neural networks to generate feature vectors representing image content; and time-series signals, which extract time-series features through recurrent neural networks or their variants to capture dynamic patterns of device operation. Step S53: Perform cross-modal feature alignment. Through a joint embedding space, the feature vectors extracted from the non-text data are aligned and mapped with the entities extracted from the text. Finally, these vectors will serve as the initial feature vectors of the corresponding graph nodes, greatly enriching the information dimension of the nodes and providing a data foundation for graph attention networks to perform deeper reasoning.

7. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 1, characterized in that, The two-layer representation architecture in step S6 includes: Symbolic layer representation: Around the production factors of people-machine-material-method-environment, all extracted knowledge triples are mapped onto the domain ontology defined in step S1, following the resource description framework standard; this step ensures that all knowledge has clear semantics that conform to industry standards. Vector layer representation: Modeling is performed using a graph attention network. The process takes the initial graph structure containing nodes and edges constructed in step S1, as well as the initial feature vectors of each node that incorporate multimodal information, as input.

8. The method for constructing and updating an assembly process knowledge graph based on graph attention network mapping according to claim 7, characterized in that, The vector layer representation in step S6 includes the following processing steps: Step S61: Feature Transformation and Attention Coefficient Calculation: For each node i and its neighbor node j in the graph, feature transformation is first performed; assuming the initial feature vectors of node i and node j are h and h respectively. i and h j And map these features to a higher-dimensional space: h i =Wh i ,h′ j =Wh j In Equation 1, W is a learnable weight matrix; Next, the feature vectors of node i and node j are concatenated and fed into a single-layer fully connected neural network to calculate the unnormalized attention coefficient e. ij : e ij =LeakyReLU(a T [h′ i ||h′ j In Equation 2, a is a learnable attention weight vector, || represents the vector concatenation operation, and LeakyReLU represents the activation function; Step S62: Normalization of attention weights: To ensure the comparability of attention coefficients between nodes, the Softmax function is used to normalize the unnormalized attention coefficients for all neighboring nodes j of node i: In the formula, N(i) represents the set of neighbors of node i, and a ij It is the attention weight of the relationship between node j and node i; the masking mechanism in this step ensures that only the coefficients of neighboring nodes are involved in the calculation; Step S63: Weighted aggregation of neighbor features: After calculating the attention weights of all neighboring nodes, the neighbor features are aggregated; for each neighbor j, its feature vector h is used. j Perform a linear transformation and multiply by the corresponding attention weight 'a'. ij Then sum all the weighted features: In the formula, h′ i It is the aggregated feature vector of node i in the current layer; Step S64: Multi-head attention mechanism: To capture the relationship information between nodes from different perspectives, a multi-head attention mechanism is adopted; for each head k, a different weight matrix W is used. k and attention weight vector a k To calculate the attention coefficient and feature aggregation: The aggregated features of each head are: Finally, the output features of all heads are concatenated or averaged to obtain the final node feature representation: Step S65: Stacking multiple layers of graph attention networks: To capture more complex long-range dependencies, multiple GAT layers need to be stacked. Each GAT layer is calculated based on the output of the previous layer, thereby gradually aggregating information from more distant neighbors. Through multi-layer stacking, nodes can gradually integrate more layers of semantic information. Step S66: Finally, the graph attention network model transforms the feature representation of each node into a high-dimensional semantic vector h′. i This vector represents the semantic information carried by node i in the graph and serves as input for subsequent steps.

Images