Industrial control system communication fault prediction method for enhancing interpretability of large language model based on dynamic time sequence knowledge graph

Abstract

The invention discloses an industrial control system communication fault prediction method for enhancing large language model interpretability based on a dynamic time sequence knowledge graph. The method comprises the following steps: step 1, processing multi-scale data based on wavelet transform; 2, constructing a time sequence knowledge graph; step 3, training a dynamic graph neural network and constructing a dynamic time sequence knowledge graph; and step 4, fusing the dynamic time sequence knowledge graph and the large language model to carry out real-time online prediction on the communication fault. According to the method, the advantages that a large language model has wide knowledge and flexible natural language interaction can be played, meanwhile, the interpretability of large language model reasoning can be enhanced, the illusion problem is reduced, and the method has the characteristics that the industrial control system has a highly-complex topological structure, multiple communication protocols and multiple data sources, and fault data needs to be analyzed in a time domain; and the communication fault prediction of the industrial control system can be effectively carried out.

Description

Technical Field

[0001] This invention belongs to the field of deep learning and relates to a method for predicting communication faults in industrial control systems. Specifically, it relates to a method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models. Background Technology

[0002] Industrial control systems (ICS) are a collective term for specialized hardware and software systems used to monitor and control industrial processes. Their core objective is to ensure the safe, reliable, and efficient operation of production facilities. ICS achieves information exchange and collaborative control between devices through communication mechanisms. During operation, internal faults such as device application failures and device outages, as well as external faults such as DDoS attacks, can cause significant damage to ICS, affecting normal industrial production and even resulting in severe economic losses or casualties. Establishing a real-time, efficient fault prediction method to predict faulty devices, fault types, and fault durations is crucial for addressing these issues. The uneven distribution of sensors, actuators, and other devices often leads to highly complex topologies in ICS. Furthermore, communication mechanisms involve multiple communication protocols and complex data sources, exhibiting multi-dimensional and multi-type characteristics. Communication faults typically evolve over time, requiring analysis of fault characteristics in the time domain. Constructing accurate communication fault prediction methods for ICS faces these challenges.

[0003] Temporal knowledge graphs introduce a time dimension to knowledge graphs, capturing the changes in entities and relationships over time. The combination of graph neural networks and recurrent neural networks has become a mainstream method for processing dynamic graph data in recent years, as this combination allows for better extraction of spatiotemporal relationships within graph data. Large language models are deep neural networks trained on massive amounts of data, enabling flexible natural language interaction.

[0004] The static nature of traditional knowledge graphs makes it impossible to capture the evolution of entities and relationships; although temporal knowledge graphs introduce timestamps, traditional temporal models struggle to efficiently capture complex time dependencies and evolutionary patterns; dynamic graph neural networks excel at handling dynamic graph structures but lack explicit, structured knowledge representations of facts; large language models lack an intrinsic understanding of factual truth, logical consistency, and event causality, and their decision-making processes are untraceable, leading to insufficient interpretability; content based on probability splicing is prone to factual errors, i.e., creating "illusions." Summary of the Invention

[0005] To address communication faults in industrial control systems (ICS), this invention provides a method for predicting ICS communication faults based on a dynamic temporal knowledge graph to enhance the interpretability of large language models. This method, aimed at overcoming the insufficient interpretability of large language models in ICS communication fault prediction scenarios and their susceptibility to "illusion" problems, proposes a dynamic temporal knowledge graph that efficiently captures the evolution patterns of entities and relationships over time and possesses explicit structured knowledge representation. This graph, which integrates a deep temporal knowledge graph and a dynamic graph neural network, can capture the temporal evolution patterns of faults in ICS communication scenarios. Because its predictions are based on rigorous mathematical calculations and data associations, it effectively enhances the interpretability of large language model reasoning. This invention leverages the advantages of large language models—their extensive knowledge and flexible natural language interaction—while also enhancing the interpretability of their reasoning and reducing the "illusion" problem. Given the highly complex topologies of ICS, the multiple communication protocols, and the diverse data sources, requiring analysis of fault data in the time domain, this method can effectively predict ICS communication faults.

[0006] The objective of this invention is achieved through the following technical solution:

[0007] A method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models includes the following steps:

[0008] Step 1: Process multi-scale data based on wavelet transform:

[0009] Based on wavelet transform, the historical and current communication performance indicators of normal and fault conditions are decomposed in the time and frequency domain, extracting important features at multiple scales and discretizing them to multiple time points.

[0010] Step 2: Construct a time-series knowledge graph:

[0011] Step 21: Customize and adjust the large language model to better suit the characteristics of the industrial control system domain. Then, use the large language model to convert the unstructured data of the industrial control system into structured data. The specific steps are as follows:

[0012] Step 211: Add a custom adjustment scheme to the large language model processing process so that each processing step has clear domain logic support and can handle common problems in the industrial control field such as fuzzy descriptions and inconsistent terminology.

[0013] Step 212, Command Guidance:

[0014] (1) Task description, which clearly requires the extraction of specific information from given unstructured data;

[0015] (2) Specify the structure template and provide a detailed description of the output format;

[0016] (3) Case training: Provide training cases to show unstructured data input and corresponding ideal structured data output, so that the large language model can learn the mapping rules;

[0017] Step 213, Large Language Model Processing:

[0018] (1) Domain knowledge-guided data understanding: embedding a three-layer context understanding mechanism, the first layer is a structured data anchoring mechanism, the second layer is a communication protocol-specific understanding mechanism, and the third layer is a time series context association mechanism;

[0019] (2) Semantic hierarchical recognition of industrial control: Implement a domain-oriented three-layer recognition system. The first layer is the entity recognition framework, the second layer is the rule-guided attention mechanism, design the triggering rules of industrial control keywords, and the third layer is the block partitioning strategy adjustment.

[0020] (3) Domain adaptive normalization: includes three aspects of normalization and formatting processing. First, the industrial control data is normalized; then, the domain semantics are normalized; and finally, adaptive adjustment is performed when the structured output is output.

[0021] Step 22: Construct a time-series knowledge graph for the industrial control system based on historical data. The specific steps are as follows:

[0022] Step 221: Extract quadruples from historical data and structured data, and add features extracted by discrete wavelet transform to the entity attributes, including but not limited to continuous communication performance indicators such as throughput, packet loss rate, and load.

[0023] Step 222: Separate the management of time-invariant triples from those that change over time. This involves constructing a static knowledge graph as a globally shared template, while the temporal knowledge graph primarily stores the relationship between changes in node attributes and timestamps. This temporal knowledge graph can intuitively represent the internal topology of the industrial control system and the evolution of the communication performance indicators of a node over discrete time series during communication. Its formal notation is defined as:

[0024] Entity collection: , of which each ( () represents an entity;

[0025] Relation set: , of which each ( () represents a relationship between entities;

[0026] Time set: , of which each ( () represents a point in time or a period of time;

[0027] Time-series knowledge graph: , where each quadruple Represents a piece of knowledge, indicating that in At that time, entity and entity Interval Existence relation;

[0028] Step 23: Construct a real-time subgraph based on real-time data. The specific steps are as follows:

[0029] Step 231: Define a sliding time window for each data source;

[0030] Step 232: When each sliding window is triggered, perform discrete wavelet transform on the continuous communication performance indicators within the window, extract the features of the communication performance indicators, and output them.

[0031] Step 233: Extract the entity identifier of each data source, quickly align it with the static knowledge graph template, confirm the node corresponding to the communication performance indicator feature, add the feature to the attribute of the node, and thus efficiently construct a real-time subgraph.

[0032] Step 3: Train a dynamic graph neural network and construct a dynamic temporal knowledge graph:

[0033] Step 31: Construct a dynamic graph neural network model consisting of a contrastive attention graph neural network, a graph attention network, a gated recurrent unit, and an incremental memory unit. The dynamic graph neural network comprises one contrastive attention graph neural network layer, two graph attention network layers, one gated recurrent unit layer, one incremental memory unit, and one fully connected layer. Specifically: the contrastive attention graph neural network layer uses eight attention heads to acquire hidden embedding representations; the first graph attention network layer uses four attention heads to process the features enhanced by the contrastive attention graph neural network; the second graph attention network layer aggregates the learned representations to generate a comprehensive node representation for prediction; the gated recurrent unit layer accepts the temporal features of each node, capturing the state evolution over time; the incremental memory unit records the compact features of encountered faulty subgraphs, and when a new subgraph is input, it quickly calculates its similarity to existing patterns. If highly similar, the prediction is strengthened; if it is a new pattern, a parameter fine-tuning process is triggered and the memory bank is updated; the fully connected layer, as the last layer, completes the final decision and is often used in conjunction with an activation function to output the final prediction result.

[0034] Step 32: Train a dynamic graph neural network model on the temporal knowledge graph obtained in Step 2. The specific steps are as follows:

[0035] Step 321, Data Preprocessing: Divide the time series knowledge graph into training set, validation set, and test set according to time order, with the test set selecting the most recent time period;

[0036] Step 322, Training: Represent the temporal knowledge graph data as a graph sequence that evolves over time, perform message passing and aggregation on the graph sequence to learn the dynamic embedding representation of entities and relations in time, and design standard negative sampling loss and temporal smoothness constraints;

[0037] Step 323, Testing and Evaluation: The tasks are link prediction and time prediction. Link prediction is to predict another entity given an entity, relationship and timestamp. Time prediction is to predict the event time given the first and last entities and relationship. The evaluation metrics are average reciprocal ranking, hit rate and time prediction error.

[0038] Step 33: Deeply fuse the temporal knowledge graph and the dynamic graph neural network into a dynamic temporal knowledge graph. The specific steps are as follows:

[0039] The entity, relation, and time information in the temporal knowledge graph are encoded and mapped to a dynamic graph neural network. The entity attributes are mapped to node feature vectors, and the entity topology is mapped to an adjacency matrix. The protocol type and communication quality in the relation are mapped to edge feature vectors, and the relation topology is mapped to an adjacency matrix. The time information is mapped to time feature encoding.

[0040] Step 4: Integrate dynamic temporal knowledge graphs and large language models for real-time online prediction of communication faults.

[0041] Step 41: Combine the real-time subgraph and dynamic time-series knowledge graph obtained in Step 2 to perform real-time fault prediction. The specific steps are as follows:

[0042] The prediction task is performed based on the dynamic temporal knowledge graph and the real-time subgraph obtained in step S2. The fault evolution mode is simulated, key nodes are identified through attention weights, and the prediction results include faulty equipment, fault type, fault occurrence time, confidence level, and key evidence chain.

[0043] Step 42: The large language model performs fault prediction reasoning based on knowledge, and the reasoning is strictly anchored to the prediction obtained in step 41.

[0044] Step 43: Output the prediction results through natural language and interact with the system to complete the task of fault tracing, generating a formatted report, and performing multi-level semantic interpretation.

[0045] Compared with the prior art, the present invention has the following advantages:

[0046] (1) Using a time-series knowledge graph to represent the communication process of an industrial control system with a complex topology allows for the definition of unique attributes and constraints of different protocols. It also enables the structured integration of heterogeneous data from multiple sources such as PLCs, SCADA systems, and logs, achieving a unified representation and effectively addressing the complex data sources of industrial control systems. The added time dimension of the time-series knowledge graph compared to a standard knowledge graph allows it to record the temporal evolution patterns of communication faults in industrial control systems, serving as a prerequisite for analyzing communication fault evolution in the time domain. The real-time subgraph constructed from real-time data collected via a sliding time window also effectively addresses the real-time requirements for predicting communication faults in industrial control systems.

[0047] (2) A dynamic temporal knowledge graph was constructed, which deeply integrates the temporal knowledge graph and the dynamic graph neural network. It can simultaneously capture the topological structure and time evolution pattern between devices, and can also explicitly model the fault propagation and evolution process. Its prediction is based on rigorous mathematical calculation and data association, and has good interpretability.

[0048] (3) It combines large language model reasoning. The advantages of large language models are that they have a wide knowledge base and can interact with natural language. Based on knowledge, they can predict communication faults in industrial control systems and improve the prediction accuracy of atypical faults. Anchoring the large language model reasoning on the prediction results of dynamic time-series knowledge graphs can enhance the interpretability of large language models, effectively solve the illusion problem, and also use large language models to process unstructured data. Customized adjustment schemes are added to the processing to make it more suitable for the field of industrial control systems. Attached Figure Description

[0049] Figure 1 A block diagram of a method for predicting communication faults in industrial control systems;

[0050] Figure 2 A flowchart for a method to predict communication faults in industrial control systems;

[0051] Figure 3 This is a diagram of the architecture of a dynamic temporal knowledge graph. Detailed Implementation

[0052] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0053] This invention provides a method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, such as... Figure 1 As shown, the prediction method includes the following four steps:

[0054] Step 1: Processing multi-scale data based on wavelet transform: Decompose the communication performance indicators of normal and faulty systems in the time and frequency domains using multiple wavelet bases, extract important multi-scale features, and discretize them to multiple time points.

[0055] The second step is to construct a time-series knowledge graph: The large language model is customized to better suit the characteristics of the industrial control system domain. Then, the unstructured data of the industrial control system, such as fault log text and equipment maintenance reports, is converted into structured data through the large language model. Finally, a time-series knowledge graph is constructed based on the structured data of the industrial control system and the performance indicators at discrete time points obtained from the first step. This involves constructing a time-series knowledge graph using historical data containing fault data and a real-time subgraph using current data.

[0056] The third step is to train a dynamic graph neural network and construct a dynamic temporal knowledge graph: First, a dynamic graph neural network model is constructed, consisting of a contrastive attention graph neural network, a graph attention network, a gated recurrent unit, and an incremental memory unit. Then, it is trained on a temporal knowledge graph containing historical fault data to obtain model parameters for predicting communication faults in industrial control systems. Finally, the temporal knowledge graph and the dynamic graph neural network are deeply integrated into a dynamic temporal knowledge graph. The dynamic graph neural network is used as the core inference engine to form a knowledge representation and computation framework for unified modeling and learning of structural evolution and time dependence in the form of a dynamic graph.

[0057] The fourth step is to integrate dynamic temporal knowledge graphs with large language models for real-time online prediction of communication faults: Fault prediction is performed on real-time subgraphs using a trained dynamic temporal knowledge graph. The reasoning of the large language model based on knowledge is strictly anchored to the facts and prediction results of the dynamic temporal knowledge graph, resulting in fault prediction results based on knowledge and facts. Thanks to the large language model's support for flexible natural language interaction, it is possible to interact with the prediction results and complete tasks such as fault tracing, generating formatted reports, and performing multi-level semantic interpretation.

[0058] like Figure 2 As shown, the specific steps are as follows:

[0059] Step S1: Perform time-frequency domain decomposition of historical and current communication performance indicators based on wavelet transform, where:

[0060] The communication performance indicators include, but are not limited to, the communication latency, load, and throughput of the communication nodes of the industrial control system;

[0061] The communication performance indicators are efficiently collected by distributed probes deployed on various devices or communication links in the industrial control system, and are continuous signals. Distributed probe technology is of great significance for meeting the real-time and reliability requirements of industrial control network performance measurement and fault detection. The probe is mainly composed of a data acquisition module, a storage module, a communication interface, a communication protocol stack, a power supply module, etc., and has data acquisition and transmission functions. Generally, clock synchronization or internal network synchronization is used to ensure that the timestamps of each node are consistent when collecting data.

[0062] The specific method of wavelet transform is to perform time-frequency domain decomposition on the communication performance indicators of normal and faulty systems using multiple wavelet bases, extract important features at multiple scales, and discretize them to multiple time points.

[0063] The wavelet transform mentioned is mainly the discrete wavelet transform method. For each fixed scale, the discrete wavelet transform outputs a discrete time series, revealing the evolution of the signal characteristics at that frequency over time. At fine time scales, it captures sudden and transient anomalies, and at coarse-grained time scales, it analyzes slowly changing trends or periodic disturbances.

[0064] The communication performance indicators are structured data both before and after discretization;

[0065] The formula for the continuous wavelet transform is:

[0066]

[0067] in, For continuous input signals; The mother wavelet is an oscillatory function with good locality in both the time and frequency domains. It has various forms, and the appropriate function can be selected as needed. For scale parameters, smaller The higher the frequency; For translation parameters, control exist The center position on the timeline; for Subwavelet obtained after scaling and time shifting; The transformation coefficients represent and The degree of similarity;

[0068] The discrete wavelet transform is derived from the continuous wavelet transform formula. , Obtained through discrete sampling, commonly using binary discretization:

[0069]

[0070] in, The usual value is 2; , representing the scale index, indicating discrete scale levels. The bigger The larger; For translation exponent, in a specific The following represents the discrete time-shifted position; The sampling interval is determined by the sampling frequency;

[0071] The formula for the discrete wavelet transform is:

[0072]

[0073] Step S2: Customize and adjust the large language model to better suit the characteristics of the industrial control system domain. Then, use the large language model to convert the unstructured data of the industrial control system into structured data, where:

[0074] The structured data is highly standardized in format, including but not limited to equipment information, sensor readings, process parameters, historical databases, etc.

[0075] The unstructured data refers to data that contains rich information but has no fixed format, including but not limited to fault log text, equipment maintenance reports, expert experience, etc.

[0076] The aforementioned custom adjustment of the large language model refers to adding a custom adjustment scheme to the large language model processing process, so that the large language model is no longer a general natural language processing, but a professional understanding of industrial control communication fault scenarios, so that each processing step has clear domain logic support, and can handle common problems in the industrial control field such as ambiguous descriptions and inconsistent terminology.

[0077] The process of converting unstructured data into structured data using a large language model mainly consists of two parts: the instruction guidance process and the large language model processing process. The instruction guidance process includes three steps:

[0078] (1) Task description, which clearly requires the extraction of specific information from given unstructured data;

[0079] (2) Specify the structure template and provide a detailed description of the output format;

[0080] (3) Case training: Provide a small number of training cases to show unstructured data input and corresponding ideal structured data output, so that the large language model can learn the mapping rules.

[0081] After incorporating a custom adjustment scheme into the large language model processing process, the large language model processing process specifically includes three steps:

[0082] (1) Domain knowledge-guided data understanding: a three-layer context understanding mechanism is embedded. The first layer is the structured data anchoring mechanism, which allows the large language model to extract entities from unstructured data, including device names, protocol names, parameter names, etc., and to query the type, subsystem, physical location, historical fault mode, etc. of these entities in the existing industrial control domain structured data, and use these as context prompts for the entities. Then there is the communication protocol specific understanding mechanism, which predefines each protocol in the industrial control system, such as Modbus / TCP, Profinet, OPC UA, etc., and predefines the key parameter fields, typical fault modes, normal value ranges, etc. of these protocols. When the large language model processes the data, it first identifies the protocol type involved in the text, and then dynamically loads the feature template of the protocol as the understanding background. Finally, there is the time series context association mechanism, which extracts the performance index trend of relevant devices in the past specified time period from the time series database, and converts these time series features into natural language descriptions as supplementary context.

[0083] (2) Industrial Control Semantic Hierarchical Recognition: Implement a domain-oriented three-layer recognition system. The first layer is the entity recognition framework, which includes the recognition of five types of entities: basic entities, industrial control semantic entities, relational entities, composite event entities, and confidence-labeled entities. Basic entities mainly include equipment names, timestamps, parameters, etc.; industrial control semantic entities mainly include communication quality, fault modes, and degree of impact, etc.; relational entities mainly include causal relationships, spatiotemporal relationships, and logical relationships, etc.; composite event entities mainly include fault chains and fault evolution, etc.; and confidence-labeled entities mainly include degree of certainty, source of evidence, and conflict markers, etc.; the second layer is... Based on a rule-guided attention mechanism, we designed a keyword triggering rule for industrial control systems. When specific words appear in the text, the large language model is guided to pay attention to the relevant context. The third layer is the adjustment of the block partitioning strategy. We proposed an adaptive block partitioning algorithm, which is no longer simply block by paragraph or word count, but by device type, time window, communication ring, etc. Blocking by device means that all descriptions of the same device are processed together. Blocking by time window means that events in the same time period are analyzed together. Blocking by communication ring means that device information in the same control ring is processed together. We proposed cross-validation partitioning, and information in different partitions is automatically cross-referenced to detect inconsistencies.

[0084] (3) Domain Adaptive Normalization: This includes three aspects of normalization and formatting. First, it normalizes industrial control data, including automatically identifying and converting different units, such as ms / s, % / ratio, dB / mV, etc.; mapping raw values ​​to standard levels, such as normal / abnormal / warning / danger, etc.; and encoding time series features into trend descriptions, such as rising / falling / fluctuating / stable, etc. Then, it performs domain semantic normalization, including establishing a synonym mapping table, such as "communication interruption → communication interruption", "PLC no response → PLC communication timeout", etc.; and establishing a fault coding mapping table to normalize natural language descriptions. The description is mapped to standard fault codes, such as "Equipment stops working → F_001: Complete interruption of equipment communication" and "Signal is interrupted from time to time → F_002: Intermittent communication failure". Finally, there is adaptive adjustment when the structured output is displayed, including dynamic template selection. Different output templates can be selected according to the type and severity of the fault. If the fault is minor, a simple report format is provided. If the fault type involves certain indicators, the historical records and related information of the indicators are added to the template. If the fault is severe, detailed information such as fault evolution records, impact assessment, fault tracing and standard handling procedures for similar faults are provided.

[0085] Step S3: Construct a time-series knowledge graph of the industrial control system based on historical data, wherein:

[0086] The historical data mainly consists of the baseline communication performance indicators during normal system operation and the communication failure performance indicators during the period when the system experienced a failure.

[0087] The knowledge graph is a structured form of information representation that depicts real-world entities (such as people, places, devices, and events) and the various relationships between them (such as location, connection, routing path, data dependency, etc.) in the form of a graph. Triples are frequently used to represent knowledge in knowledge graphs. The temporal knowledge graph adds a time dimension to the knowledge graph, more accurately describing the changes of entities and relationships over time. In a temporal knowledge graph, each triple is additionally associated with one or more timestamps, forming a quadruple, indicating that the relationship is valid at a certain point in time or within a certain period, for example (PLC_001, data transmission, Switch_Core_001, 20151220143024).

[0088] The formal notation of the time-series knowledge graph is defined as follows:

[0089] Entity collection: , of which each ( () represents an entity;

[0090] Relation set: , of which each ( () represents a relationship between entities;

[0091] Time set: , of which each ( () represents a point in time or a period of time;

[0092] Time-series knowledge graph: , where each quadruple Represents a piece of knowledge, indicating that in At that time, entity and entity Interval Existence relation;

[0093] The process of constructing the temporal knowledge graph is as follows: Quadruples are extracted from historical and structured data, such as (PLC_001, data transmission, Switch_Core_001, 20151220143024). Entity attributes include, but are not limited to, features extracted from continuous communication performance indicators such as throughput, packet loss rate, and load through discrete wavelet transform. In the example, the timestamp represents 14:30:24 on December 20, 2015, and can be adjusted to the millisecond level as needed. When constructing the temporal knowledge graph, triples that do not change over time (main...) can be... To separate the management of the topology, physical connections, and communication connections of an industrial control system (which are unchanging over time and unaffected by faults) from the time-varying triples (which mainly describe the entity parts whose attributes and relationships change after a fault), a static knowledge graph is constructed as a globally shared template, while the temporal knowledge graph mainly stores the association between changes in node attributes and timestamps. This can greatly improve the construction speed. This temporal knowledge graph can intuitively reflect the internal topology of the industrial control system and the evolution of the communication performance indicators of a node in the discrete time series during the communication process.

[0094] The storage and display of the time-series knowledge graph is handled by HugeGraph, a database that natively supports additional timestamps and is well-suited for recording dynamic changes. When the amount of time-series data is small, Neo4j can also be used, and time-series queries can be achieved through attribute filtering.

[0095] Step S4: Construct a real-time subgraph based on real-time data, where:

[0096] The real-time data mainly consists of real-time communication performance indicators on communication devices. A sliding time window is defined for each data source, for example, a window length of 5 seconds and a sliding step size of 1 second. When each sliding window is triggered, discrete wavelet transform is performed on the continuous communication performance indicators within the window to extract the communication performance indicator features and output them. The entity identifier of each data source is extracted and quickly aligned with the static knowledge graph template constructed in step S3 to confirm the node corresponding to the communication performance indicator feature. Features are added to the attributes of the node to efficiently construct a real-time subgraph.

[0097] Step S5: Construct a dynamic graph neural network model consisting of a contrastive attention graph neural network, a graph attention network, a gated recurrent unit, and an incremental memory unit, wherein:

[0098] The dynamic graph neural network mainly consists of a contrastive attention graph neural network layer, two graph attention network layers, a gated recurrent unit layer, a memory increment unit, and a fully connected layer. The contrastive attention graph neural network layer uses eight attention heads to acquire hidden embedding representations, ensuring the quality of the input features. The first graph attention network layer uses four attention heads to process the features enhanced by the contrastive attention graph neural network, extracting deeper features. The second graph attention network layer aggregates the learned representations to generate a comprehensive node representation for prediction; the two-layer graph attention network can deeply understand the relationships in the graph structure. The gated recurrent unit layer accepts the temporal features of each node, capturing the evolution of the state over time. The increment memory unit, an auxiliary structure of the gated recurrent unit, records the compact features of encountered faulty subgraphs. When a new subgraph is input, this unit quickly calculates its similarity to existing patterns; if highly similar, the prediction is strengthened; if it is a new pattern, a parameter fine-tuning process is triggered and the memory bank is updated, achieving experience accumulation and rapid adaptation. The fully connected layer, as the last layer, completes the final decision and is often used in conjunction with activation functions (such as Softmax or Sigmoid) to output the final prediction result. The core similarity metric is the comprehensive similarity, which includes structural similarity (weight 40%), feature embedding similarity (weight 35%), and time series pattern similarity (weight 25%). Structural similarity is normalized using graph editing distance, feature embedding similarity is a weighted average of node embedding cosine similarity, and time series pattern similarity uses dynamic time-warped distance. After calculating and normalizing the comprehensive similarity, a threshold of 0.85 is set as the high similarity threshold.

[0099] This dynamic graph neural network is a model specifically designed for processing and predicting temporal knowledge graphs. It can capture hidden structural evolution patterns and temporal dependencies that are crucial for predicting failures from a series of graph snapshots. It can remember failure patterns that have occurred in the past and make different, smarter responses when it encounters similar or new patterns. The final output is usually a prediction of the state of a node at some point in the future.

[0100] Step S6: Train a dynamic graph neural network model on the temporal knowledge graph obtained in step S3, wherein:

[0101] The training process includes three steps. First, in the data preprocessing stage, the temporal knowledge graph is divided into training, validation, and test sets according to time sequence, with the test set selecting the most recent time period. Second, in terms of training strategy, the temporal knowledge graph data is represented as a graph sequence evolving over time. Message passing and aggregation are performed on the graph sequence to learn the dynamic embedding representation of entities and relations in time sequence. Standard negative sampling loss and temporal smoothness constraints are designed. The significance of standard negative sampling loss is that, after artificially constructing negative samples, the model learns discriminative features by assigning higher scores to true facts and lower scores to false facts. The significance of temporal smoothness constraints is to prevent overfitting in the time dimension and enhance the robustness of the model. Finally, there is testing and evaluation. The tasks are link prediction and time prediction. Link prediction involves predicting another entity given an entity, relation, and timestamp. Time prediction involves predicting the event time given the first and last entities and relations. The evaluation metrics are mean reciprocal rank (MRR), hit rate (Hit@k), and time prediction error.

[0102] Step S7: Deeply fuse the temporal knowledge graph and the dynamic graph neural network into a dynamic temporal knowledge graph. Combine the real-time subgraph obtained in step S4 with the dynamic temporal knowledge graph to perform real-time fault prediction, wherein:

[0103] The architecture diagram of the deep fusion of temporal knowledge graph and dynamic graph neural network into a dynamic temporal knowledge graph is as follows. Figure 3 The dynamic temporal knowledge graph is not a simple mechanical superposition of the two, but has a two-way closed-loop information flow and an integrated architecture that can be jointly optimized in terms of learning methods. The trained dynamic graph neural network can feed the prediction results back to the temporal knowledge graph, thereby updating the structure of the temporal knowledge graph. The updated temporal knowledge graph can continue to train the dynamic graph neural network, forming a closed-loop reinforcement system.

[0104] The dynamic temporal knowledge graph has the following advantages: graph construction guides model training, and model training optimizes graph construction; the graph structure changes dynamically with learning; explicit structured knowledge representation reduces uncertainty; strong interpretability, and the decision-making processes of graph construction and training can explain each other; this deep integration enables the dynamic temporal knowledge graph to transcend the traditional paradigm of building first and then training, and achieve true intelligent integration.

[0105] The key steps in deeply integrating temporal knowledge graphs and dynamic graph neural networks into a dynamic temporal knowledge graph are as follows: Encoding and mapping entity, relation, and time information from the temporal knowledge graph to the dynamic graph neural network; mapping entity attributes to node feature vectors, where elements include, but are not limited to, latency level, load level, throughput level, packet loss rate level, device type encoding, protocol support vector, criticality weight, and location encoding; mapping the topology of entities to an adjacency matrix; mapping protocol type and communication quality in relations to edge feature vectors, where elements include, but are not limited to, communication protocol encoding, communication direction, dependency type, communication frequency, data packet size, and time evolution characteristics; mapping the topology of relations to the adjacency matrix; and mapping time information to time feature encoding, where elements include, but are not limited to, absolute time encoding, relative time encoding, and periodic time encoding.

[0106] The fault prediction process is as follows: based on the dynamic temporal knowledge graph and the real-time subgraph obtained in step S4, the prediction task is executed to simulate the fault evolution mode, key nodes are identified through attention weights, and the prediction results are output, including faulty equipment, fault type, fault occurrence time, confidence level, key evidence chain, etc.

[0107] Step S8: The large language model performs fault prediction inference based on knowledge, and the inference is strictly anchored to the prediction obtained in step S7. Finally, the prediction result is output through natural language and interactive processing is performed.

[0108] The reasoning process of the large language model for fault prediction based on knowledge is as follows: The prediction results of the dynamic temporal knowledge graph, the entities related to the prediction and their attributes and relationships near the prediction time point, the dynamic process leading to the prediction conclusion, industrial control system domain rules, historical fault cases, task instructions, etc., are transformed into structured text and input into the large language model. Reasoning is then performed based on its extensive built-in knowledge. The large language model can understand the rich semantics in unstructured data, associating specific data with abstract concepts and historical experience. Combined with its extensive knowledge base, it can handle not only typical fault scenarios but also atypical, vaguely described, or fault scenarios caused by complex indirect factors, enhancing its generalization ability. The reasoning generation process is strictly anchored to the facts and reasoning results of the dynamic temporal knowledge graph. The dynamic temporal knowledge graph accurately models the spatiotemporal evolution of the industrial control system, and the prediction is based on strict... Mathematical computation and data association offer high accuracy in fault mode identification, specifically in the following ways: First, temporal graphs explicitly structure entities, relationships, and their changes over time, making the knowledge background upon which the model relies transparent and verifiable. Second, the paths and association rules in the graph provide a clear logical chain for the reasoning process, supporting the tracing back from the result to relevant facts. Furthermore, the dynamic temporal mechanism ensures that knowledge is updated in real time with new evidence, avoiding reliance on outdated or static fragments for inference and improving the timeliness and reliability of conclusions. Finally, the graph-based reasoning steps can be explained layer by layer through visualization or natural language, organically combining symbolic computation with semantic reasoning. This makes it easier for users to understand how the model derives predictive conclusions from data associations. Therefore, the reasoning of large language models is generated based on reliable facts, fundamentally constraining the reasoning of large language models and effectively solving the illusion problem.

[0109] To enhance the interpretability of predictions generated by the large language model, the following measures are taken: each prediction generated by the large language model is associated with specific entities, relationships, and temporal facts in the temporal knowledge graph; the probability confidence level output by the dynamic temporal knowledge graph serves as a constraint condition when the large language model generates predictions; based on the structural differences of the temporal knowledge graph, a comparative analysis of fault types generated by the large language model is conducted to explain "why it is fault A instead of fault B";

[0110] The key features of the dynamic temporal knowledge graph are converted into decision rules. The large language model outputs the rule "At time t, when [feature 1] satisfies ... and [feature 2] satisfies ..., it is predicted that fault A will occur".

[0111] The aforementioned interaction with the prediction results via natural language output refers to the ability to interact with the prediction results after obtaining knowledge-based and fact-based communication fault predictions. Benefiting from the flexible natural language interaction capabilities of the large language model, users can perform tasks such as fault tracing and generating formatted reports for multi-level semantic interpretation. The specific process for generating the formatted report is as follows: the large language model integrates the prediction results with relevant context, such as recent anomaly indicators of related nodes, historical similar cases, attention weights of all nodes pointing to the central node in the graph attention network layer, and important neighboring nodes and their contributions, to form a readable decision-making basis. A formatted report is then generated based on a pre-set template.

[0112] Example:

[0113] This embodiment provides a method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models. The method includes the following steps:

[0114] Step S1: Perform time-frequency domain decomposition of historical and current communication performance indicators based on wavelet transform.

[0115] In this embodiment, communication performance indicators include node communication latency, load, throughput, etc., which are efficiently collected by distributed probes deployed on various devices or communication links of the industrial control system, and are continuous signals.

[0116] The wavelet base strategy used in this embodiment is as follows: Load and throughput are analyzed using db6, sym6, and Haar. The db6 wavelet achieves the best balance between mutation detection and trend separation. The sym6 wavelet has better statistical properties for maintaining traffic distribution in symmetrical structures. The Haar wavelet has a simple structure and can achieve fast real-time detection. Communication delay is analyzed using db8, sym8, and coif3. The db8 wavelet is suitable for accurately separating deterministic and random delay components at high-frequency resolution. The sym8 wavelet is suitable for maintaining the time correlation of delay sequences in symmetrical structures. The coif3 wavelet has more vanishing moments, providing better trend separation capabilities. Multi-index comprehensive analysis uses bior3.3 and rbio3.3. The linear phase characteristics of the bior3.3 wavelet maintain the time synchronization relationship between multiple indices. The rbio3.3 wavelet's inverted structure optimizes the emphasis analysis of different indices.

[0117] Step S2: Customize and adjust the large language model to better suit the characteristics of the industrial control system field, and then use the large language model to convert the unstructured data of the industrial control system into structured data.

[0118] In this embodiment, the unstructured data includes two years' worth of fault logs and maintenance reports from a certain industrial control system. The structured data input to the large language model includes equipment basic information tables, relational database tables, I / O device data, historical signal data, fault type library, physical network topology data, bus system topology, control system hierarchical topology, signal flow topology, power supply system topology, etc. After two stages—instruction guidance and large language model processing with added custom adjustments—structured data is obtained. Specifically, the extracted content includes time, equipment entity, numerical value, fault type, protocol type, fault characteristics, fault process, fault resolution process, control logic, and relationships.

[0119] Step S3: Construct a time-series knowledge graph of the industrial control system based on historical data.

[0120] In this embodiment, data from a certain industrial control system over two years is used to construct a time-series knowledge graph. To control capacity, data from 10 normal operation periods and all communication failure periods are selected, with the data from the normal operation periods serving as the baseline.

[0121] The process of constructing a time-series knowledge graph in this embodiment is as follows: cleaning, deduplicating, and unifying the format of structured data; performing entity recognition and disambiguation; extracting relationships and aligning them in time sequence; creating nodes, connecting edges, and attaching attributes to corresponding data times under different timestamps; performing time consistency checks and resolving conflicts; and fusing multi-source data to eliminate conflicts.

[0122] This embodiment constructs a static knowledge graph that fully reflects the topology of the industrial control system and sets it as a knowledge graph template. Subsequent attribute additions at different timestamps and the creation of new edges can be generated based on the template, improving the efficiency of constructing time-series knowledge graphs and real-time subgraphs.

[0123] In this embodiment, the storage and display of the time-series knowledge graph are accomplished by HugeGraph.

[0124] Step S4: Construct a real-time subgraph based on real-time data.

[0125] In this embodiment, real-time data acquisition is achieved through distributed probes within the industrial control system. Internal network synchronization ensures that the timestamps of each node are consistent during data acquisition, and data is collected when each sliding time window arrives.

[0126] In this embodiment, the sliding time window is designed with a window length of 5 seconds and a sliding step size of 1 second. When each sliding window is triggered, the continuous communication performance indicators collected within the window are subjected to discrete wavelet transform to extract the communication performance indicator features and output them. The entity identifier of each data source is extracted and quickly aligned with the knowledge graph template obtained in step S3 to identify the node corresponding to the communication performance indicator feature. The feature is added to the attribute of the node, thereby efficiently constructing a real-time subgraph.

[0127] Step S5: Construct a dynamic graph neural network model consisting of a contrastive attention graph neural network, a graph attention network, a gated recurrent unit, and an incremental memory unit.

[0128] The dynamic graph neural network consists of a contrastive attention graph neural network layer, two graph attention network layers, a gated recurrent unit layer, a memory increment unit, and a fully connected layer. The contrastive attention graph neural network layer uses eight attention heads to acquire hidden embedding representations; the first graph attention network layer uses four attention heads to process the features enhanced by the contrastive attention graph neural network, extracting deeper features; the second graph attention network layer aggregates the learned representations to generate a comprehensive node representation for prediction; the gated recurrent unit receives temporal features from each node, capturing state evolution patterns; the increment memory unit, an auxiliary structure of the gated recurrent unit layer, records the compact features of encountered faulty subgraphs. When a new subgraph is input, this unit quickly calculates its similarity to existing patterns; if highly similar, the prediction is strengthened; if it is a new pattern, a parameter fine-tuning process is triggered and the memory is updated; the fully connected layer, in conjunction with the Softmax activation function, outputs the final prediction result.

[0129] Step S6: Train a dynamic graph neural network model on the temporal knowledge graph obtained in step S3.

[0130] This embodiment divides the temporal knowledge graph into training, validation, and test sets according to time sequence, resulting in an overall division of 60% training set, 20% validation set, and 20% training set. The test set selects data from the most recent time period. In terms of training strategy, the temporal knowledge graph data is represented as a graph sequence evolving over time. A dynamic graph neural network is used to perform message passing and aggregation on the graph sequence to learn the dynamic embedding representation of entities and relations in time sequence. Standard negative sampling loss and temporal smoothness constraints are designed to enable the model to learn discriminative features, prevent overfitting in the time dimension, and enhance model robustness. Finally, in the testing and evaluation part, the tasks are set as link prediction and time prediction. Link prediction involves predicting another entity given an entity, relation, and timestamp. Time prediction involves predicting the event time given the first and last entities and relations. The evaluation metrics are mean reciprocal rank (MRR), hit rate (Hit@k), and time prediction error.

[0131] Step S7: Deeply integrate the temporal knowledge graph and the dynamic graph neural network into a dynamic temporal knowledge graph, and combine the real-time subgraph obtained in step S4 with the dynamic temporal knowledge graph to perform real-time fault prediction.

[0132] In this embodiment, real-time subgraph data is encoded into graph sequences and communication fault prediction is performed using a dynamic graph neural network. The prediction results are then imported into a time-series knowledge graph for updating after structured parsing and confidence evaluation, thus achieving a closed loop.

[0133] Step S8: The large language model performs fault prediction reasoning based on knowledge, and the reasoning is strictly anchored to the prediction obtained in step S7. Finally, the prediction result is output through natural language and interaction is performed.

[0134] In this embodiment, the prediction results in the form of structured data, including real-time communication performance index features, device topology data, fault logs, maintenance reports, and dynamic temporal knowledge graph output, are input into the large language model. Then, the prediction result template is input into the large language model, and the output of the large language model is strictly anchored to the prediction results of the dynamic temporal knowledge graph.

[0135] In this embodiment, the prediction results output by the large language model are interacted with in natural language, with questions such as "What is the reason for this conclusion?", "Please give the cause of the failure?", and "Please provide historical reports of this type of failure?" are raised. The large language model's responses are clear, logical, and reasonable.

Claims

1. A method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, characterized in that... The method includes the following steps: Step 1: Process multi-scale data based on wavelet transform: Based on wavelet transform, the historical and current communication performance indicators of normal and fault conditions are decomposed in the time and frequency domain, extracting important features at multiple scales and discretizing them to multiple time points. Step 2: Construct a time-series knowledge graph: Step 21: Customize and adjust the large language model to better suit the characteristics of the industrial control system field, and then use the large language model to convert the unstructured data of the industrial control system into structured data. Step 22: Construct a time-series knowledge graph of the industrial control system based on historical data; Step 23: Construct a real-time subgraph based on real-time data; Step 3: Train a dynamic graph neural network and construct a dynamic temporal knowledge graph: Step 31: Construct a dynamic graph neural network model consisting of a contrastive attention graph neural network, a graph attention network, a gated recurrent unit, and an incremental memory unit; Step 32: Train a dynamic graph neural network model on the temporal knowledge graph obtained in step 2; Step 33: Deeply integrate the temporal knowledge graph and the dynamic graph neural network into a dynamic temporal knowledge graph; Step 4: Integrate dynamic temporal knowledge graphs and large language models for real-time online prediction of communication faults. Step 41: Combine the real-time subgraph and dynamic temporal knowledge graph obtained in Step 2 to perform real-time fault prediction; Step 42: The large language model performs fault prediction reasoning based on knowledge, and the reasoning is strictly anchored to the prediction obtained in step 41. Step 43: Output the prediction results through natural language and interact with the system to complete the task of fault tracing, generating a formatted report, and performing multi-level semantic interpretation.

2. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... The specific steps of step 21 are as follows: Step 211: Add a custom adjustment scheme to the large language model processing process so that each processing step has clear domain logic support and can handle common problems in the industrial control field such as fuzzy descriptions and inconsistent terminology. Step 212, Command Guidance: (1) Task description, which clearly requires the extraction of specific information from given unstructured data; (2) Specify the structure template and provide a detailed description of the output format; (3) Case training: Provide training cases to show unstructured data input and corresponding ideal structured data output, so that the large language model can learn the mapping rules; Step 213, Large Language Model Processing: (1) Domain knowledge-guided data understanding: embedding a three-layer context understanding mechanism, the first layer is a structured data anchoring mechanism, the second layer is a communication protocol-specific understanding mechanism, and the third layer is a time series context association mechanism; (2) Semantic hierarchical recognition of industrial control: Implement a domain-oriented three-layer recognition system. The first layer is the entity recognition framework, the second layer is the rule-guided attention mechanism, design the triggering rules of industrial control keywords, and the third layer is the block partitioning strategy adjustment. (3) Domain adaptive normalization: This includes three aspects of normalization and formatting processing. First, the industrial control data is normalized. Then, the domain semantics are normalized. Finally, the structured output is adaptively adjusted.

3. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... The specific steps of step 22 are as follows: Step 221: Extract quadruples from historical data and structured data, including features extracted by discrete wavelet transform for throughput, packet loss rate, and load continuous communication performance indicators in entity attributes. Step 222: Separate the management of triples that do not change over time from those that change over time. That is, construct a static knowledge graph as a globally shared template, while the temporal knowledge graph mainly stores the relationship between changes in node attributes and timestamps.

4. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... The specific steps of step 23 are as follows: Step 231: Define a sliding time window for each data source; Step 232: When each sliding window is triggered, perform discrete wavelet transform on the continuous communication performance indicators within the window, extract the features of the communication performance indicators, and output them. Step 233: Extract the entity identifier of each data source, quickly align it with the static knowledge graph template, identify the node corresponding to the communication performance indicator feature, add the feature to the attribute of the node, and thus efficiently construct a real-time subgraph.

5. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... The specific steps of step 32 are as follows: Step 321, Data Preprocessing: Divide the time series knowledge graph into training set, validation set, and test set according to time order, with the test set selecting the most recent time period; Step 322, Training: Represent the temporal knowledge graph data as a graph sequence that evolves over time, perform message passing and aggregation on the graph sequence to learn the dynamic embedding representation of entities and relations in time, and design standard negative sampling loss and temporal smoothness constraints; Step 323, Testing and Evaluation: The tasks are link prediction and time prediction. Link prediction is to predict another entity given an entity, relationship, and timestamp. Time prediction is to predict the event time given a head and tail entity and relationship. The evaluation metrics are average reciprocal ranking, hit rate, and time prediction error.

6. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... The specific steps of step 33 are as follows: The entity, relation, and time information in the time-series knowledge graph are encoded and mapped to a dynamic graph neural network, the entity attributes are mapped to node feature vectors, and the entity topology is mapped to an adjacency matrix. Map the protocol type and communication quality in the relationship to edge feature vectors, and map the topology of the relationship to the adjacency matrix; Time information is mapped to time feature codes.

7. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... The specific steps of step 41 are as follows: The prediction task is performed based on the dynamic temporal knowledge graph and the real-time subgraph obtained in step S2. The fault evolution mode is simulated, key nodes are identified through attention weights, and the prediction results include faulty equipment, fault type, fault occurrence time, confidence level, and key evidence chain.

8. The method for predicting communication faults in industrial control systems based on dynamic temporal knowledge graphs to enhance the interpretability of large language models, as described in claim 1, is characterized in that... In step 42, the reasoning process of the large language model for fault prediction based on knowledge is as follows: the prediction results of the dynamic temporal knowledge graph, the entities related to the prediction and their attributes and relationships near the prediction time point, the dynamic process leading to the prediction conclusion, the rules of the industrial control system domain, historical fault cases, and task instructions are converted into structured text and input into the large language model. Based on its built-in extensive knowledge, the large language model performs reasoning. It understands the rich semantics in unstructured data, associates specific data with abstract concepts and historical experience, and combines its knowledge base to process typical fault scenarios and atypical, vaguely described, or fault scenarios caused by indirect factors.

Images