Cyber physical control system using graph retrieval augmented generation and neuro symbolic reasoning

Abstract

The present invention relates to a cyber-physical control system using graph search augmentation generation and neural symbol inference, comprising: a graph generation unit that generates a knowledge graph based on sensor data collected from industrial equipment and equipment-related documents; a graph inference unit that extracts contextual information based on the knowledge graph and inputs the contextual information into a language model to generate a control command; a neural symbol verification unit that verifies the control command based on the knowledge graph and blocks the control signal output of the industrial equipment when a violation of a constraint stored in the knowledge graph is detected; and an NPU acceleration unit that accelerates the traversal operation of the knowledge graph and the neural network operation of the language model.

Description

Technology Field

[0001] The present invention relates to a cyber-physical control system using graph search augmentation generation and neural symbol inference. Background Technology

[0002] Recently, there has been a surge in demand in the manufacturing industry to utilize Large Language Models (LLMs) for equipment control and fault diagnosis in industrial settings. However, language models are inherently probabilistic and suffer from the hallucination problem, which can generate different outputs for the same input each time; consequently, they face limitations in that they are difficult to apply directly to industrial control systems that require high reliability.

[0003] To mitigate these hallucination issues and improve the response quality of language models, Retrieval-Augmented Generation (RAG) technology is being introduced, which retrieves relevant information from external knowledge bases and provides it to the models. However, existing vector-based RAG technologies have structural limitations in that the logical context running through the entire document and the causal relationships between different documents are lost during the process of dividing and vectorizing documents into fixed sizes. Consequently, there is a risk of inaccurate diagnostic results in situations requiring multi-level causal reasoning, such as complex equipment failures in manufacturing plants.

[0004] To overcome the limitations of such vector-based search-augmented generation technologies, Graph Retrieval-Augmented Generation (GAG) technology has been proposed, which represents structural relationships between data as a Knowledge Graph and utilizes it for search. GAG technology is attracting attention as a technology suitable for industrial control environments because it enables multi-level inference by representing causal relationships between equipment entities as nodes and edges.

[0005] However, existing graph search augmented generation technologies suffer from the problem of lacking a means to deterministically guarantee physical safety during the process of converting the probabilistic inference results of language models into industrial control commands. In other words, there is no deterministic verification mechanism to preemptively block situations where control commands generated by the language model violate physical constraints of equipment or cause conflicts, such as the simultaneous operation of interlocked devices, which can lead to safety accidents in industrial settings.

[0006] Furthermore, graph traversal operations have irregular memory access patterns and high data dependencies, leading to frequent cache misses on conventional general-purpose graphics processing units (GPUs). This presents a hardware limitation in that the ultra-low latency processing required for real-time control is impossible.

[0007] Therefore, there is a need to develop an industrial cyber-physical control system capable of structurally representing complex equipment causal relationships in industrial sites and performing multi-stage reasoning, while also being equipped with deterministic verification means to prevent safety violations caused by the probabilistic output of language models in advance, and capable of processing graph traversal and neural network operations in real time. Prior art literature

[0008] Republic of Korea Registered Patent Publication No. 10-2582779 The problem to be solved

[0009] The present invention was devised to solve the aforementioned conventional problems and aims to provide a cyber-physical control system using graph search augmented generation and neural symbol inference that can resolve the risk of safety accidents caused by hallucinations and computational delay issues resulting from the application of probabilistic artificial intelligence to physical control environments by deriving and deterministically verifying control commands of a language model based on an industrial knowledge graph and accelerating the processing using dedicated hardware. means of solving the problem

[0010] The above objective is achieved by a cyber-physical control system using graph search augmentation generation and neural symbol inference, comprising, according to the present invention: a graph generation unit that generates a knowledge graph based on sensor data collected from industrial equipment and equipment-related documents; a graph inference unit that extracts contextual information based on the knowledge graph and inputs the contextual information into a language model to generate a control command; a neural symbol verification unit that verifies the control command based on the knowledge graph and blocks the control signal output of the industrial equipment when a violation of a constraint stored in the knowledge graph is detected; and an NPU acceleration unit that accelerates the traversal operation of the knowledge graph and the neural network operation of the language model.

[0011] Additionally, the graph generation unit includes a data collection module that collects and preprocesses sensor data and equipment-related documents from the industrial equipment; an ontology mapping module that generates the knowledge graph by extracting entities and relationships between entities from the preprocessed equipment-related documents, mapping them to a standard ontology schema, and converting them into nodes and edges; and a dynamic mapping module that updates the knowledge graph by mapping the preprocessed sensor data to the dynamic attributes of the nodes.

[0012] Additionally, the graph inference unit comprises a community detection module that clusters nodes of the knowledge graph based on density to form a community and generates summary information of the community, a node identification module that identifies an anchor node by performing a vector similarity search based on the summary information, a traversal search module that extracts context information by performing a traversal operation based on the anchor node, and a command generation module that inputs the context information into the language model to generate the control command.

[0013] Additionally, the neural symbol verification unit includes a constraint verification module that verifies whether the control command violates the constraint by comparing it with the Shapes Constraint Language (SHACL)-based constraint stored in the knowledge graph, a simulation module that simulates the state change resulting from the execution of the control command and verifies whether there is a topological violation through reachability analysis on the knowledge graph for the control command that has passed the verification of the constraint verification module, and a blocking module that blocks the control signal output of the industrial equipment when the constraint violation or the topological violation is detected.

[0014] In addition, the NPU acceleration unit includes a neural network acceleration module that accelerates the neural network operation by accelerating the dense matrix operation of the language model based on a systolic array structure, and a traversal acceleration module that accelerates the traversal operation by processing irregular memory accesses that occur during the traversal operation of the knowledge graph based on a gather-scatter method. Effects of the invention

[0015] According to the present invention, by integrating fragmented unstructured equipment documents and real-time sensor data to construct a knowledge graph and extracting structural context through multi-stage graph traversal operations, it is possible to prevent the hallucination phenomenon that occurs when a language model relies on fragmentary keywords and to accurately identify the root cause of complexly intertwined equipment failures.

[0016] In addition, since the probabilistic control commands generated by the language model are not immediately applied to physical equipment, and the neural symbol verification unit performs deterministic verification based on constraints and topological reachability predefined in the knowledge graph, the risk of physical safety accidents such as mechanical collisions or exceeding limits is prevented in advance, making it safe to apply to industrial control environments where strict reliability is required.

[0017] In addition, by integrating the irregular traversal operations of a knowledge graph and the dense neural network operations of a large-scale language model on-chip in a dedicated NPU acceleration unit based on heterogeneous computation cores, memory bottlenecks and latency caused by external cloud communication can be drastically resolved, thereby enabling ultra-low latency real-time equipment control even in edge environments and preventing the leakage of corporate industrial secret data.

[0018] Meanwhile, the effects of the present invention are not limited to those mentioned above, and various effects may be included within the scope obvious to a person skilled in the art from the contents described below. Brief explanation of the drawing

[0019] FIG. 1 is a block diagram illustrating, overall, a cyber-physical control system using graph search augmentation generation and neural symbol inference according to an embodiment of the present invention, and FIG. 2 is a block diagram illustrating the detailed configuration of a graph generation unit of a cyber-physical control system using graph search augmentation generation and neural symbol inference according to an embodiment of the present invention, and FIG. 3 is a block diagram illustrating the detailed configuration of a graph inference unit of a cyber-physical control system using graph search augmentation generation and neural symbol inference according to an embodiment of the present invention, and FIG. 4 is a block diagram illustrating the detailed configuration of a neural symbol verification unit of a cyber-physical control system using graph search augmentation generation and neural symbol inference according to an embodiment of the present invention, and FIG. 5 is a block diagram illustrating the detailed configuration of an NPU acceleration unit of a cyber-physical control system using graph search augmentation generation and neural symbol inference according to an embodiment of the present invention. Specific details for implementing the invention

[0020] Hereinafter, some embodiments of the present invention will be described in detail with reference to the exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings.

[0021] In addition, when describing embodiments of the present invention, if it is determined that a detailed description of related known configurations or functions would hinder understanding of the embodiments of the present invention, such detailed description is omitted.

[0022] In addition, terms such as first, second, A, B, (a), (b), etc., may be used when describing the components of the embodiments of the present invention. These terms are used merely to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by the terms used.

[0023] Additionally, in the specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms “comprising” and / or “comprising” as used in the specification do not exclude the presence or addition of one or more other components in addition to the mentioned components.

[0024] In addition, in describing embodiments of the present invention, a “part,” “module,” or “step” as each component may be implemented using one or more processors and memory. Here, a processor may be interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a microcontroller (MCU), a digital signal processor (DSP), a controller, or a state machine, etc.

[0025] The processor may be composed of a single processor or a processor including multiple processing cores, and may include various forms of processing devices, such as, for example, a combination of multiple microprocessors, a combination of a microprocessor and a DSP, or a single processor including multiple processing cores having different characteristics. Additionally, depending on the specific implementation environment, the processor may be configured to include an Application-Specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), or similar hardware logic circuits.

[0026] Additionally, memory may be interpreted as including any electronic storage means capable of storing electronic information, and may include various forms of processor-readable media including random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), programmable read-only memory (PROM), eraseable programmable read-only memory (EPROM), electrically eraseable read-only memory (EEPROM), flash memory, magnetic or optical storage devices, registers, etc.

[0027] When a processor can read information from memory or write information to memory, the processor and memory are interpreted as being in a state where they can communicate electronically, and each “part,” “module,” or “step” according to an embodiment of the present invention may be implemented by a program, firmware, or application stored in the processor and memory in such an electronically communicable state.

[0028] Furthermore, in the present invention, Artificial Intelligence (AI) refers to a technology that imitates human learning ability, reasoning ability, and perceptual ability, and implements them on a computer, and may include concepts such as machine learning and symbolic logic. Machine Learning (ML) is an algorithmic technology that classifies or learns the characteristics of input information on its own. AI technology can analyze input information as a machine learning algorithm, learn from the results of that analysis, and make judgments or predictions based on the results of that learning. Additionally, technologies that mimic the functions of the human brain, such as cognition and judgment, by utilizing machine learning algorithms can also be understood as falling within the category of AI. For example, technological fields such as linguistic understanding, visual understanding, reasoning / prediction, knowledge representation, and motion control may be included.

[0029] Artificial intelligence learning models or neural network models can be designed to implement the structure of the human brain on a computer and may include multiple network nodes that have weights and simulate neurons of a human neural network. These multiple network nodes may have interconnected relationships by simulating the synaptic activity of neurons, where neurons exchange signals through synapses. An artificial intelligence learning model may be, for example, an Artificial Neural Network (ANN) or a Convolutional Neural Network (CNN). An artificial intelligence learning model may be machine-learned according to methods such as supervised learning, unsupervised learning, and reinforcement learning. Machine learning algorithms used to perform machine learning may include Decision Trees, Bayesian Networks, Support Vector Machines, Perceptrons, and Clustering.

[0031] From now on, a cyber-physical control system (100) using graph search augmentation generation and neural symbol inference according to one embodiment of the present invention will be described in detail with reference to the attached drawings.

[0032] As illustrated in FIG. 1, a cyber-physical control system (100) using graph search augmentation generation and neural symbol inference according to one embodiment of the present invention includes a graph generation unit (110), a graph inference unit (120), a neural symbol verification unit (130) and / or an NPU acceleration unit (140).

[0033] The graph generation unit (110) generates a knowledge graph based on sensor data collected from industrial equipment and equipment-related documents, and is electrically connected to the graph inference unit (120) and / or the neural symbol verification unit (130).

[0034] The graph generation unit (110) integrates unstructured data, semi-structured data, and structured data generated at the industrial site to construct a knowledge graph that represents the relationship between equipment entities and entities as nodes and edges.

[0035] Here, the Knowledge Graph is a graph-based data structure in which facility entities are represented as nodes and physical connection and causal relationships between entities are represented as edges. The graph inference unit (120), which will be described later, can be utilized as a structural basis for exploring multi-stage causal relationship paths, and the neural symbol verification unit (130), which will be described later, can be utilized for constraint reference to verify control commands.

[0036] Unstructured data may include text-based data such as equipment manuals, standard operating procedures, maintenance history and / or failure reports, semi-structured data may include converted data of piping and instrumentation drawings, bills of materials and circuit diagrams, and structured data may include register maps of programmable logic controllers and / or real-time sensor tag values ​​of SCADA systems.

[0037] As illustrated in FIG. 2, the graph generation unit (110) includes a data collection module (111), an ontology mapping module (112) and / or a dynamic mapping module (113).

[0038] The data collection module (111) is configured to collect and preprocess sensor data and equipment-related documents from industrial equipment, and is electrically connected to the ontology mapping module (112) and / or dynamic mapping module (113).

[0039] The data collection module (111) can collect real-time sensor data from a programmable logic controller and / or SCADA system through communication protocols such as Modbus and OPC-UA used in industrial sites, for example, and can collect equipment-related documents such as equipment manuals, standard operating procedures, maintenance history and / or failure reports.

[0040] The data collection module (111) can perform text chunking processing on the collected equipment-related documents to divide them into text chunks that can be processed by a language model.

[0041] Text chunking is a preprocessing step that divides long, unstructured text documents into units of a size manageable within the input limits of a language model; these units can be determined based on sentence boundaries, paragraph structure, and semantic completeness. Through this process, even massive documents spanning hundreds of pages, such as equipment manuals, can be converted into units that the language model can process without losing context.

[0042] The data collection module (111) can preprocess the collected sensor data by performing outlier filtering and / or normalization processing.

[0043] Outlier filtering is a process that removes abnormal measurements caused by sensor malfunctions or communication errors, and functions as a preprocessing step to ensure the reliability of the subsequent Grandeur causality analysis.

[0044] Normalization processing is a process that converts heterogeneous sensor data having different physical units and measurement ranges, such as temperature, pressure, and rotational speed, into a common scale, enabling the dynamic mapping module (113), which will be described later, to update the dynamic attributes of the sensor node in a consistent format.

[0045] The divided text chunks are passed to the ontology mapping module (112) and used as input for entity extraction, and at the same time can be stored in the knowledge graph as chunk nodes for original context reference.

[0046] A chunk node is a representation of an original text chunk as a node within a knowledge graph, and an ontology mapping module (112), to be described later, can be connected to an entity node extracted from the chunk via an edge.

[0047] Through this, when the graph inference unit (120) requires detailed information about a specific entity during the process of performing a graph traversal operation, it can immediately refer to the context of the original document through the chunk node connected to the entity node.

[0048] Next, the ontology mapping module (112) is configured to extract entities and relationships between entities from facility-related documents preprocessed by the data collection module (111), and to map the extracted relationships between entities to a standard ontology schema to generate a static structure of a knowledge graph, and is electrically connected to the data collection module (111) and / or the dynamic mapping module (113).

[0049] The ontology mapping module (112) can input text chunks received from the data collection module (111) into a large language model (LLM) for information extraction to extract facility entities and causal relationships between entities.

[0050] A Large Language Model (LLM) for information extraction is a model specialized for information extraction tasks that extract structured knowledge from unstructured text, and can be implemented in a form fine-tuned for Named Entity Recognition and / or Relation Extraction tasks.

[0051] When the ontology mapping module (112) inputs text chunks into a large-scale language model for information extraction, it inputs a predefined prompt for the type of entity to be extracted and the type of relationship so that entities and relationships specialized in the facility domain are extracted in a consistent format.

[0052] A large-scale language model for information extraction can analyze the context of input text chunks to identify equipment entities, process entities, sensor entities, etc., and extract physical connection relationships, causal relationships, and / or control relationships between the identified entities.

[0053] For example, from the text chunk "If the coolant temperature exceeds 60 degrees, the chiller efficiency drops sharply," the coolant temperature entity, the chiller efficiency reduction entity, and the causal relationship between them can be extracted.

[0054] At this time, the ontology mapping module (112) can structure the data extracted through the language model into semantic triples in the form of subject, predicate, and object.

[0055] Here, a triple refers to the smallest unit representing knowledge, signifying a data format that represents two entities and the relationship between them as a single structured unit. For example, the relationship between extracted coolant temperature and chiller efficiency degradation is structured as a triple of “Coolant Temperature (Subject), Triggered by Exceeding (Predicate), Chiller Efficiency Degradation (Object),” which can be utilized as the basic unit for generating nodes and edges in a knowledge graph.

[0056] In this way, the ontology mapping module (112) can convert facility knowledge extracted from unstructured text into a structured form that can be processed by a machine through a triple structure.

[0057] Additionally, the ontology mapping module (112) can connect the extracted entity nodes to the original text chunk nodes split by the data collection module (111) via edges. This allows the graph inference unit (120), which will be described later, to immediately refer to the detailed context of the original document that is the source of the entity when performing a graph traversal operation.

[0058] The ontology mapping module (112) can ensure data interoperability by mapping the extracted entities and relationships to a standard ontology schema.

[0059] For example, the ontology mapping module (112) can use Basic Formal Ontology (BFO) as the top-level ontology and hierarchically apply Industrial Ontologies Foundry (IOF) and Semantic Sensor Network (SSN) ontologies below it.

[0060] Basic Formal Ontology (BFO) is a high-level ontology that provides a top-level conceptual framework independent of domains, such as material entities, processes, and attributes, and can function as a common foundation that allows lower-level ontologies to be defined on a consistent conceptual framework.

[0061] The Industrial Ontologies Foundry (IOF) is a domain ontology that standardizes and defines concepts of industrial domains, such as manufacturing equipment, processes, and maintenance activities, based on basic format ontology, and plays a role in ensuring conceptual compatibility between equipment from different manufacturers.

[0062] A Semantic Sensor Network (SSN) is an ontology that defines the relationships between sensors, measurement targets, and observations in a standardized manner, and can provide a semantic basis for connecting sensor data with node attributes of a knowledge graph.

[0063] In this way, the ontology mapping module (112) applies a hierarchical ontology structure to define equipment, processes, sensors, and failure types in a standardized conceptual framework, thereby ensuring data compatibility between equipment from different manufacturers and maintaining a consistent semantic structure throughout the knowledge graph.

[0064] The ontology mapping module (112) can primarily generate a static structure that forms the backbone of the knowledge graph by converting entities mapped to a standard ontology schema into nodes and relationships between entities into edges.

[0065] Nodes included in the static structure contain static attributes that do not change over time, such as the unique identifier, type, installation location, and model name of the facility entity, and each edge may contain information regarding the type of relationship and directionality between nodes.

[0066] Next, the dynamic mapping module (113) is electrically connected to the data collection module (111) and / or the ontology mapping module (112) and is configured to update the knowledge graph by mapping sensor data preprocessed in the data collection module (111) to the dynamic attributes of nodes generated in the ontology mapping module (112).

[0067] The dynamic mapping module (113) can update the knowledge graph by mapping sensor data collected in real time from the data collection module (111) to the dynamic attributes of the corresponding entity node created as a static structure in the ontology mapping module (112).

[0068] The dynamic attributes of a node refer to attributes that change in real time, such as current temperature, pressure, and rotational speed, and are updated on the high-speed memory of the NPU acceleration unit (140) described later, so that the physical state of the industrial equipment and the digital state of the knowledge graph can be synchronized.

[0069] In addition, the dynamic mapping module (113) can perform Granger causality analysis on multiple sensor data collected through the data collection module (111).

[0070] Granger causality analysis is an analytical technique that verifies whether specific time series data makes a significant contribution to statistically predicting the future values ​​of other time series data. When applied to the tag values ​​of sensor data in industrial facilities, it can be utilized to detect potential causal relationships based on the temporal sequence of data, even among sensors where physical connections such as piping or electrical wiring are not specified in the facility drawings.

[0071] For example, if a pattern is detected in which the vibration sensor value of the cooling water pump statistically precedes the change in the outlet temperature sensor value of the heat exchanger after a few minutes, the dynamic mapping module (113) can create an edge between the two sensor nodes and add it to the knowledge graph.

[0072] Through this, the knowledge graph can be expanded into a structure that includes not only explicit physical connection relationships established by the ontology mapping module (112) from equipment-related documents, but also potential causal relationships inductively derived from actual operation data, and the graph inference unit (120) can explore hidden influence paths between equipment not specified in the design drawings, thereby improving the accuracy of root cause analysis of complex equipment failures.

[0073] Meanwhile, the dynamic mapping module (113) can calculate the rate of change of sensor data values ​​per unit time and store it as a dynamic attribute of the corresponding node.

[0074] The dynamic mapping module (113) can calculate a rate of change in the form of a first difference value by dividing the difference between the sensor data value at the current sampling point and the sensor data value at the previous sampling point by the sampling period, and can perform a difference operation after applying a moving average filter over multiple sampling periods to block data distortion caused by short-term noise.

[0075] The dynamic mapping module (113) adjusts the window size of the moving average filter according to the dynamic response characteristics of the equipment type, and can apply different window sizes to the pressure sensor with a fast response speed and the temperature sensor with a slow response speed.

[0076] For example, if the sampling period of the reactor pressure sensor is 100ms and the pressure value at the current time is 85kPa and the pressure value at the previous time is 84kPa, the dynamic mapping module (113) can calculate the pressure change rate by applying a moving average filter and then performing a difference operation.

[0077] For example, if the sampling period of the reactor pressure sensor is 100ms and the current pressure value is 85kPa and the previous pressure value is 84kPa, the dynamic mapping module (113) can store the calculated pressure change rate of 10kPa / s as a dynamic attribute of the reactor pressure node. In this way, even when the current pressure value of 85kPa satisfies static constraints as being less than the allowable maximum pressure value of 100kPa, if the pressure change rate of 10kPa / s approaches the allowable maximum pressure rise rate constraint defined by shape constraint language rules, the dynamic mapping module (113) can identify this from the dynamic attribute of the node.

[0078] Additionally, the dynamic mapping module (113) can store a change rate abnormality flag as a dynamic attribute in the knowledge graph by adding it to the node if the calculated change rate exceeds a predefined change rate threshold.

[0079] The change rate threshold can be defined as a shape constraint language rule based on the permissible transient response characteristics for each facility extracted from facility-related documents by the ontology mapping module (112) and stored in the shape constraint language rule of the corresponding facility entity node.

[0080] The rate of change threshold can be set as a certain ratio relative to the allowable upper limit of the static constraint or as the maximum allowable rate of change specified in the facility manual.

[0081] The constraint verification module (131) of the neural symbol verification unit (130) utilizes a change rate anomaly flag to classify control commands for the corresponding node as targets for additional verification based on the change rate, and can verify the possibility of preemptive constraint violation based on the change trend even in a normal state section where the current value satisfies static constraint conditions.

[0082] The dynamic mapping module (113) can construct a change rate vector between multiple sensor nodes and calculate a Pearson correlation coefficient between the change rate vectors to analyze the change rate correlation.

[0083] The dynamic mapping module (113) identifies pairs of sensor nodes in which the calculated Pearson correlation coefficient exceeds a predefined correlation threshold, and can use the correlation of the rate of change of the identified pairs of sensor nodes as a selection criterion for analysis targets of Granger causal analysis.

[0084] The dynamic mapping module (113) can use the Pearson correlation coefficient between the rate of change vectors as a pre-filtering criterion for Granger causality analysis to select pairs of sensor nodes with high rate of change correlation as priority analysis targets for Granger causality analysis.

[0085] Through this, the dynamic mapping module (113) can rapidly identify pairs of sensor nodes in which potential causal relationships are being formed in a transient state using only short-term time series data, and intensively perform Granger causal relationship analysis on the identified pairs of nodes to update edges and reflect them in the knowledge graph.

[0086] In this way, the graph generation unit (110) can generate and update a knowledge graph through a series of processing flows in which, based on sensor data and equipment-related documents collected and preprocessed by the data collection module (111), the ontology mapping module (112) extracts entities and relationships to generate a static structure of the knowledge graph, and the dynamic mapping module (113) reflects real-time sensor data as dynamic attributes of the nodes.

[0088] The graph inference unit (120) extracts contextual information based on a knowledge graph received from the graph generation unit (110) and inputs the contextual information into a language model to generate control commands, and is electrically connected to the graph generation unit (110), the neural symbol verification unit (130) and / or the NPU acceleration unit (140).

[0089] The graph inference unit (120) can generate control commands that correspond to the current state and causal relationship of the equipment by utilizing a knowledge graph as a search target to explore structural relationships and causal paths between nodes and providing the explored structural context to a language model through Graph Retrieval-Augmented Generation (GraphRAG), which explores multi-stage causal relationship paths of industrial equipment from a knowledge graph constructed by the graph generation unit (110), extracting contextual information, and providing the extracted contextual information to a language model.

[0090] The generated control command can be transmitted to the neural symbol verification unit (130) and, after undergoing verification against the constraints defined in the knowledge graph, output to the industrial equipment.

[0091] As illustrated in FIG. 3, the graph inference unit (120) includes a community detection module (121), a node identification module (122), a traversal search module (123) and / or a command generation module (124).

[0092] The community detection module (121) is configured to cluster nodes of the knowledge graph based on density to form communities and generate summary information of the communities, and is electrically connected to the node identification module (122).

[0093] The community detection module (121) can analyze the edge density between nodes across the entire knowledge graph and cluster nodes with high interconnection into one community.

[0094] A community refers to a set of nodes within a knowledge graph whose interconnectedness is relatively denser than their connections with external nodes; from the perspective of industrial facilities, this can correspond to groups of functionally related facility entities, such as cooling systems, drive systems, and control systems.

[0095] The community detection module (121) can apply the Leiden Algorithm as a clustering algorithm.

[0096] The Leiden algorithm is an algorithm that iteratively refines communities in a way that maximizes the modularity of connections between nodes, and it can derive stable and high-quality community structures from large-scale knowledge graphs.

[0097] In this context, modularity is an indicator representing how high the edge density within a community is compared to a random graph; a higher modularity value signifies that the community structure is more clearly distinguishable.

[0098] The community detection module (121) can organize the formed communities into a hierarchical structure to create a hierarchical community structure composed of an upper community and a lower community.

[0099] For example, the top-level community can be hierarchized by encompassing the entire cooling system, and the lower-level communities can be subdivided into cooling water circulation loops and heat exchanger groups.

[0100] The community detection module (121) can generate summary information for each formed community by integrating the attributes of nodes and edges within the community.

[0101] Summary information is information that compresses the functional roles, major causal relationships, and operating constraints of facility entities belonging to the community into a natural language format, and can be used as a search target for the node identification module (122) to perform vector similarity search.

[0102] Since the clustering and summary information generation of the community detection module (121) is performed and stored in advance at the time the knowledge graph is built, the stored summary information can be utilized immediately without a separate clustering operation during the real-time control command generation process.

[0103] Next, the node identification module (122) is configured to identify anchor nodes by performing a vector similarity search based on summary information generated by the community detection module (121), and is electrically connected to the community detection module (121) and / or the traversal search module (123).

[0104] The node identification module (122) receives a control request or anomaly detection signal received from an industrial facility as input, converts it into an embedding vector, and calculates the cosine similarity with the embedding vector of the community summary information pre-generated by the community detection module (121) to preferentially identify the most relevant community.

[0105] The node identification module (122) can identify the node most relevant to the input within the community as an anchor node based on summary information of the identified community.

[0106] An anchor node is a node that serves as the starting point for traversing a knowledge graph, and refers to an equipment entity node most directly associated with a control request. For example, when an overload abnormal signal of a cooling water pump is input, the node identification module (122) first identifies a cooling system community and can identify a cooling water pump node within that community as an anchor node.

[0107] In this way, by performing a hierarchical search at the community level, the search scope can be significantly reduced compared to performing a vector similarity search across the entire knowledge graph, thereby ensuring the response speed required for real-time control even in large-scale knowledge graphs.

[0108] Next, the traversal search module (123) is configured to extract context information by performing a traversal operation based on the anchor node identified by the node identification module (122), and is electrically connected to the node identification module (122) and the command generation module (124).

[0109] The traversal search module (123) can perform a hybrid search that combines edge-based graph traversal from anchor nodes identified based on the vector similarity search results performed by the node identification module (122).

[0110] The traversal search module (123) can explore indirect connection relationships between facilities without direct keyword matching by supplementing the multi-stage causal relationship paths between facilities that cannot be explored by vector similarity search alone through hybrid search by graph traversal.

[0111] A knowledge graph traversal operation is a graph search algorithm that systematically visits the nodes and edges of a graph. In the present invention, this can be applied by starting from an anchor node and repeatedly visiting adjacent nodes from the cause node to the result node, following the directionality of the causal relationship edges.

[0112] The traversal search module (123) can explore a multi-stage causal path by repeatedly performing a traversal operation starting from an anchor node and moving to an adjacent node along the causal edge of the knowledge graph.

[0113] At this time, the traversal search module (123) may apply a search depth limit and a relevance threshold to control the depth and range of the search.

[0114] The search depth limit is a parameter that defines how many steps of adjacent nodes to traverse from an anchor node; as the search depth increases, a broader causal path is explored, but the likelihood of including nodes with low relevance to the control request may also increase.

[0115] In addition, the relevance threshold is a filtering criterion that measures the semantic relevance between a node visited during the traversal process and an anchor node using cosine similarity, including only nodes with a level above a certain threshold in the search target; it can be utilized for pruning to exclude nodes with low relevance from the search path.

[0116] The traversal search module (123) may be configured to include types of edges that are the subject of the traversal operation, such as causal edges generated by the ontology mapping module (112) from facility-related documents and edges generated by the dynamic mapping module (113) through Granger causal analysis.

[0117] This allows for the exploration of not only physical connection relationships specified in design drawings but also potential causal paths derived from actual operating data, thereby enabling the detection of complex propagation paths stemming from an anomaly in a single facility.

[0118] For example, when a cooling water pump node is identified as an anchor node for an overload abnormal signal of a cooling water pump, the traversal search module (123) can start from the cooling water pump node and traverse the causal path of the cooling water pump, cooling water flow rate, heat exchanger inlet temperature, and reactor temperature in multiple stages, and collect real-time measurements of the cooling water flow rate, heat exchanger inlet temperature, and reactor temperature, which are the current dynamic attribute values ​​of each node, and the types of causal relationships between nodes.

[0119] In addition, the dynamic mapping module (113) can explore potential causal paths along the edge created between the coolant pump vibration sensor and the heat exchanger outlet temperature sensor through Grandeur causal analysis.

[0120] The traversal search module (123) can generate a subgraph including static attributes, dynamic attributes, and types of causal relationships between nodes of nodes traversed through traversal operations.

[0121] A subgraph is a partial graph formed by selecting only the nodes and edges related to a control request from the entire knowledge graph, and can contain structural information compressed to a size that can be processed within the context window of a language model.

[0122] Additionally, the traversal search module (123) can include the relevant context of the facility manual and standard operating procedure in the subgraph by referencing the original text chunk node connected to the explored entity node together.

[0123] The traversal search module (123) can serialize the configured subgraph into a natural language format to generate context information and transmit the generated context information to the command generation module (124).

[0124] Contextual information refers to information generated from a subgraph extracted by the traversal search module (123) traversing the knowledge graph, and includes structured information such as static and dynamic attributes of facility entity nodes related to control requests, types and directions of causal relationships between nodes, and related operating constraints.

[0125] Contextual information is distinguished from the contextual information provided by existing vector-based search augmentation generative technologies in that it is not text fragments based on simple keyword similarity, but rather structural information combining real-time sensor data and causal relationship paths between facilities represented by nodes and edges of a knowledge graph.

[0126] Contextual information can be converted into a prompt for a large-scale language model for inference and input into a command generation module (124) to be described later, and can function as basis information that enables the language model to generate control commands by referencing the current physical state of the equipment and the causal relationship between the equipment. Through this, the hallucination problem that may occur when the language model generates control commands relying only on training data is suppressed, and control commands based on the real-time equipment state reflected in the knowledge graph can be generated.

[0127] Meanwhile, the traversal search module (123) can generate context information by including the rate of change of sensor data values ​​and / or a rate of change abnormality flag, which the dynamic mapping module (113) has stored as dynamic attributes of the node, in the subgraph.

[0128] Large-scale language models for inference can generate control commands that reflect the direction and speed of state changes by referencing the rate of change of sensor data values ​​included in context information.

[0129] For example, even if the current value of a reactor temperature node does not approach the allowable upper limit, if the rate of change is trending rapidly, a large-scale language model for inference can recognize this from contextual information and generate a control command to preemptively suppress the temperature rise.

[0130] This allows for a transition from ex post control based on current values ​​to preemptive control based on change trends, thereby reducing the ripple effect beyond the equipment.

[0131] Next, the command generation module (124) is configured to generate control commands by inputting context information extracted from the traversal search module (123) into a language model, and is electrically connected to the traversal search module (123) and / or the neural symbol verification unit (130).

[0132] The command generation module (124) structures the context information received from the traversal search module (123) into a prompt format specialized for generating control commands and inputs it into a large-scale language model for inference.

[0133] The prompt is configured to include dynamic attribute values ​​representing the current equipment status, causal relationship paths between equipment, related operating constraints, and control request content, enabling the language model to generate control commands by referencing the equipment's current physical state and causal relationships together.

[0134] A control command is a specific operation instruction for industrial equipment generated by a large-scale language model for inference based on contextual information, and may include commands that change the physical state of the equipment, such as starting or stopping specific equipment, opening or closing valves, adjusting pump rotational speed, or changing temperature or pressure set values.

[0135] Control commands can be generated not only as simple operation commands for a single facility, but also as complex control commands including sequential or simultaneous operations for multiple facilities.

[0136] Since the control command is output as a control signal for industrial equipment after undergoing a comparison verification with the constraints defined in the knowledge graph in the neural symbol verification unit (130) to be described later, the probabilistic output generated by the language model can be designed to pass deterministic safety verification before being directly applied to the industrial site.

[0137] The command generation module (124) can generate control commands by analyzing the current facility status and causal relationship path based on the prompt input into the large-scale language model for inference.

[0138] For example, when an abnormal signal indicating an overload of the cooling water pump is input, the command generation module (124) constructs a prompt including a causal path leading from the cooling water pump node to the cooling water flow rate node, the heat exchanger inlet temperature node, and the reactor temperature node, the current dynamic attribute values ​​of each node, and the operating constraints of the cooling water pump, and inputs this to a large-scale language model for inference. Subsequently, the large-scale language model for inference analyzes the causal path indicating that the overload condition of the cooling water pump can propagate to a rise in reactor temperature through a reduction in cooling water flow rate, and can generate a complex control command to gradually reduce the rotational speed of the cooling water pump and open the auxiliary cooling water line.

[0139] In addition, the command generation module (124) can extract context information through Map-Reduce and generate control commands when a global query across the entire system, such as a summary of the state of the entire process or an analysis of energy consumption patterns, is input.

[0140] Map-Reduce refers to a distributed processing paradigm that divides large-scale data into multiple parallel processing units for processing and then integrates the results.

[0141] The command generation module (124) can apply a map-reduce method to perform a Map step in which multiple hierarchical community summary information pre-generated in the community detection module (121) is searched in parallel to generate partial inference results for each community, and a Reduce step in which a large-scale language model for inference synthesizes the partial inference results for each community to generate a final inference result that encompasses the entire system.

[0142] Through this, the command generation module (124) can generate control commands that reflect the overall operating status of the process from equipment information distributed throughout the knowledge graph.

[0143] Additionally, the large-scale language model for inference of the command generation module (124) can be implemented as a generative large-scale language model and can be optimized into a form that can be directly executed on the NPU acceleration unit (140) through quantization processing that converts the model's weights and activation values ​​from floating-point to integer. Through this, real-time control commands can be generated in the edge environment of an industrial site without relying on a cloud server.

[0144] In this way, the graph inference unit (120) performs a series of processing flows in which a community detection module (121) pre-generates community structure and summary information of a knowledge graph, a node identification module (122) identifies anchor nodes related to control requests through vector similarity search, a traversal search module (123) traverses multi-stage causal relationship paths from anchor nodes to extract context information, and a command generation module (124) inputs the extracted context information into a large-scale language model for inference to generate control commands.

[0145] The control command generated through the graph inference unit (120) is transmitted to the neural symbol verification unit (130) and, after undergoing verification against the constraints defined in the knowledge graph, can be output to the industrial equipment.

[0147] Next, the neural symbol verification unit (130) verifies the control command received from the graph inference unit (120) based on the knowledge graph and blocks the control signal output of the industrial equipment when a violation of the constraints stored in the knowledge graph is detected, and is electrically connected to the graph generation unit (110) and / or the graph inference unit (120).

[0148] The neuro-symbolic verification unit (130) can verify control commands generated by the large-scale language model for reasoning of the graph reasoning unit (120) through neuro-symbolic reasoning.

[0149] Neuro-Symbolic Reasoning is a method that combines neural network-based reasoning based on probabilistic pattern recognition with symbolic logic reasoning based on predefined rules and constraints. In the present invention, this can be applied to verify probabilistic control commands generated by a large-scale language model for reasoning against deterministic constraints defined in a knowledge graph.

[0150] Through this, safety in industrial sites can be ensured by deterministically blocking the control signal output when the probabilistic output of the language model violates physical safety constraints in the industrial site.

[0151] As illustrated in FIG. 4, the neural symbol verification unit (130) includes a constraint verification module (131), a simulation module (132) and / or a blocking module (133).

[0152] The constraint verification module (131) is configured to verify whether a control command is violated by comparing it with Shapes Constraint Language (SHACL)-based constraints stored in a knowledge graph, and is electrically connected to the simulation module (132) and / or the blocking module (133).

[0153] Shapes Constraint Language (SHACL) is a standard language for defining validation rules for graph data based on the Resource Description Framework (RDF), which allows for the declarative definition of node attribute value ranges, allowed types of relationships between nodes, and cardinality constraints between nodes.

[0154] In the present invention, the shape constraint language-based constraints are defined as shape constraint language rules for the nodes and edges of a knowledge graph, wherein the ontology mapping module (112) of the graph generation unit (110) defines the operating constraints extracted from equipment-related documents. These constraints may include an allowable operating range per equipment, interlock conditions, and safety cutoff conditions.

[0155] Since the shape constraint language-based constraint is stored by being directly connected to the nodes of the knowledge graph, the constraint verification module (131) can perform verification by immediately referencing the shape connected to the corresponding node during the process of exploring the knowledge graph without referring to a separate rule database.

[0156] Through this, when the graph generation unit (110) updates the knowledge graph, the constraints are also updated at the same time, so that changes in the equipment operating conditions can be immediately reflected in the verification criteria of the constraint verification module (131).

[0157] The constraint verification module (131) can identify the target facility entity node for each operation instruction constituting the control command received from the command generation module (124) in the knowledge graph, and compare whether the operation instruction is outside the allowable operation range or violates the interlock condition by referring to the shape connected to the identified node.

[0158] For example, if a control command containing an operation instruction to set the rotational speed of a cooling water pump to a value exceeding the allowable maximum value is input, the constraint verification module (131) can detect a violation by comparing it with the rotational speed upper limit constraint of the shape connected to the cooling water pump node and transmit a blocking signal to the blocking module (133).

[0159] If a violation of a constraint condition is detected as a result of verification by the constraint verification module (131), the blocking module (133) can block the control signal output of the industrial equipment by referring to the verification result.

[0160] Additionally, if no constraint violation is detected, the constraint verification module (131) transmits the corresponding control command to the simulation module (132) to perform additional verification.

[0161] Meanwhile, the constraint verification module (131) can perform a change rate-based constraint verification by comparing the change rate stored as a dynamic attribute of the node by the dynamic mapping module (113) with a change rate constraint defined by a shape constraint language rule.

[0162] A rate of change constraint is a constraint that defines the allowable range of change of a state value of an equipment entity per unit time using shape constraint language rules. It refers to a condition for detecting in advance a transient state that may lead to equipment damage or process instability if the rate of change exceeds the allowable range, even if the current sensor data value is within the allowable operating range.

[0163] Rate of change constraints are used in conjunction with existing constraints based on the absolute value of state values ​​to simultaneously verify the static and dynamic stability of the equipment state.

[0164] For example, a static constraint of an allowable maximum pressure value of 100 kPa and a rate of change constraint of an allowable maximum pressure rise rate of 10 kPa / s per unit time can be defined and stored as a shape for a reactor pressure node, and the constraint verification module (131) can determine that the rate of change constraint is violated if the pressure rise rate exceeds 10 kPa / s even if the current pressure value is less than 100 kPa.

[0165] The change rate constraint can be defined as a shape constraint language rule based on the permissible transient response characteristics for each facility extracted from the facility-related document by the ontology mapping module (112) and stored in the shape of the corresponding facility entity node.

[0166] Through this, a cyber-physical control system (100) using graph search augmentation generation and neural symbol inference according to one embodiment of the present invention can extend the declarative constraint expression capability of a shape constraint language to a time-series dynamic data domain, thereby enabling a transition from a retrospective safety block based on current sensor data values ​​to a preemptive safety block based on change trends.

[0167] Next, the simulation module (132) is electrically connected to the constraint verification module (131) and / or blocking module (133) and is configured to simulate the execution of a control command based on a knowledge graph for a control command that has passed verification by the constraint verification module (131) and to re-verify whether there is a violation of the constraint.

[0168] Constraints based on configuration constraint language are suitable for verifying static constraints, such as the attribute value range of individual equipment nodes, but they have limitations in pre-verifying the ripple effects that occur sequentially across multiple equipment when control commands are executed.

[0169] To compensate for these limitations, the simulation module (132) can perform a topological reachability analysis that simulates the ripple effects occurring on the knowledge graph in advance when executing a control command.

[0170] Topological reachability analysis refers to an analysis method that explores the path along which state changes of nodes affected by the execution of a control command propagate along the causal edges of a knowledge graph, and simulates all reachable states along that path to verify in advance the possibility of a constraint violation state occurring.

[0171] The simulation module (132) can apply a graph traversal algorithm to simulate state changes of all nodes reachable along causal relationship edges from the target node of a control command execution, and can re-verify whether the simulated state changes violate the constraints defined in the knowledge graph.

[0172] For example, even if a specific valve opening command passes verification by the constraint verification module (131), the simulation module (132) can re-verify whether the flow rate change caused by the valve opening can reach a state where it violates the upper limit constraint of the water level of the downstream tank by simulating it on a knowledge graph.

[0173] Additionally, when a command for simultaneous operation is input for two devices with interlocks set, the simulation module (132) can re-verify through simulation whether the command violates interlock constraints by exploring mutually exclusive relationship edges defined in the knowledge graph.

[0174] For example, in the case of a complex situation where a control command to 'increase conveyor speed' to increase production is transmitted from the graph inference unit (120), the constraint verification module (131) can identify the conveyor belt node that is the target of the operation instruction constituting the control command, and verify whether the allowable operating range is exceeded from a single node perspective by comparing it with the maximum allowable speed constraint (e.g., 110 mm / s) connected to the node by shape constraint language rules. Subsequently, even if the control command passes the first verification of the constraint verification module (131), the simulation module (132) can secondarily simulate the ripple effect by exploring the nodes of downstream processes connected to the conveyor belt node by causal edge on the knowledge graph.

[0175] For example, if the simulation module (132) predicts that when the conveyor speed increases, it will exceed the shooting cycle of the downstream vision inspection node and reach a Forbidden State, which is a defect detection error, it can determine this as a topological violation. In this way, by sequentially combining the constraint verification module (131), which performs static constraint verification of a single node, and the simulation module (132), which performs dynamic ripple effect verification between multiple nodes, the risk of complex physical safety accidents that may be caused by the probabilistic language model can be blocked.

[0176] If a violation of a constraint is detected as a result of re-verification of the simulation module (132), the blocking module (133) can block the control signal output of the industrial equipment by referring to the re-verification result.

[0177] Next, the blocking module (133) is electrically connected to the constraint verification module (131) and / or simulation module (132) in a configuration that blocks the control signal output of the industrial equipment when a constraint violation or topological violation is detected.

[0178] The blocking module (133) can monitor the verification results of the constraint verification module (131) and the simulation module (132) in real time and block the output of the control signal corresponding to the control command as soon as a constraint violation or topological violation is detected.

[0179] The blocking module (133) can selectively block only the control signal corresponding to the operation instruction for which a violation is detected among the plurality of operation instructions constituting the control command, or block the control signal output for the entire control command collectively when the scope of the violation extends to the entire system. By distinguishing between selective blocking and collective blocking according to the scope and severity of the violation, the control signals for the remaining operation instructions for which no violation is detected can be output normally.

[0180] The blocking module (133) receives verification results from the constraint verification module (131) and the simulation module (132) and can identify whether a violation is detected, the type of violation, and / or the violation target facility entity node.

[0181] When a violation target facility entity node is identified, the blocking module (133) can calculate the range of propagation of adjacent nodes connected to the violation target node by causal relationship edges by referring to the knowledge graph, and determine the range of the blocking target control signal based on the calculated propagation range.

[0182] When the range of the control signal to be blocked is determined, the blocking module (133) can block the output of the control signal and record a blocking history including the blocked control command, the type of violation, and the information of the facility to be violated in the knowledge graph. The blocking history recorded in the knowledge graph can be used as input data for Granger causal relationship analysis of the dynamic mapping module (113) and can be used to update potential causal relationship edges.

[0183] As an example, the blocking module (133) can be implemented in a hard gating manner by installing a hardware gate in the output path of the control signal to directly block the control signal output path at the hardware level.

[0184] The hard gating method ensures the reliability of safety functions because physical blocking is performed independently of software processing delays or errors. This allows for the deterministic blocking of constraint violation control commands originating from the probabilistic output of a language model before they are transmitted to industrial equipment.

[0185] Meanwhile, the blocking module (133) can block the control signal output and, at the same time, generate a feedback signal including the violated constraint and / or simulated topological violation result and transmit it to the graph inference unit (120).

[0186] The feedback signal may include specific operation instruction parameters of a blocked control command, the violating facility entity node, allowable threshold values ​​of shape constraint language rules defined in the knowledge graph, and / or prohibited state information derived through topological reachability analysis.

[0187] The command generation module (124) of the graph inference unit (120) can configure a new prompt that reflects the violation history based on the received feedback signal.

[0188] For example, if a control command sets the conveyor belt speed to exceed an allowable limit value, the command generation module (124) can add the allowable limit value information defined in the knowledge graph and the vision inspection error prediction information of the downstream process to the prompt and re-enter it into a large-scale language model for inference.

[0189] Large-scale language models for inference can analyze re-entered prompts to generate alternative control commands that satisfy safety constraints defined in a knowledge graph.

[0190] For example, a large-scale language model for inference can derive and output an alternative control command that readjusts the conveyor belt speed to a value within a safe limit by referring to the allowable limit value and expected error state of the feedback signal. Through this, a cyber-physical control system (100) using graph search augmentation generation and neural symbol inference according to an embodiment of the present invention can implement a closed-loop control mechanism that searches for a safe control alternative and restores the equipment state by linking the neural symbol verification unit (130) and the graph inference unit (120).

[0191] On the other hand, if neither constraint violation nor topological violation is detected, the blocking module (133) can keep the output path of the control signal open to transmit the verified control signal to the industrial equipment.

[0192] In this way, the neural symbol verification unit (130) performs a series of processing flows in which the constraint verification module (131) verifies whether the control command violates static constraints through comparison with shape constraint language-based constraints, the simulation module (132) re-verifies the possibility of violating constraints due to cascading ripple effects occurring during the execution of the control command through topological reachability analysis, and the blocking module (133) blocks the control signal output if a violation is detected.

[0193] Through the neural symbol verification unit (130), a control command for a constraint violation resulting from the probabilistic output of a language model is deterministically blocked before being transmitted to industrial equipment, thereby ensuring safety at the industrial site.

[0195] The NPU acceleration unit (140) accelerates the traversal operation of the knowledge graph and the neural network operation of the language model, and is electrically connected to the graph generation unit (110), the graph inference unit (120), and / or the neural symbol verification unit (130).

[0196] The NPU acceleration unit (140) can process traversal operations on nodes and edges of a knowledge graph and neural network operations required for inference of a language model within a single hardware acceleration structure.

[0197] The NPU acceleration unit (140) processes two operations having different computational characteristics through a dedicated acceleration structure optimized for each, thereby securing the computational performance required for graph search augmentation generation and neural symbol inference in an independent on-premise environment and reducing the total processing delay from the time of knowledge graph construction to the time of control signal output, thereby enabling real-time cyber-physical control in industrial sites.

[0198] As shown in FIG. 5, the NPU acceleration unit (140) includes a neural network acceleration module (141) and a traversal acceleration module (142).

[0199] The neural network acceleration module (141) is configured to accelerate neural network operations by accelerating dense matrix operations of the language model based on a systolic array structure, and is electrically connected to the traversal acceleration module (142).

[0200] The neural network acceleration module (141) can accelerate the neural network operation in which a large-scale language model for information extraction extracts entities and relationships from text chunks in the ontology mapping module (112) of the graph generation unit (110), and the neural network operation in which a large-scale language model for inference generates control commands based on context information in the command generation module (124) of the graph inference unit (120).

[0201] For example, neural network operations in language models may include large-scale dense matrix multiplication operations occurring in the attention mechanism of a Transformer structure and the feed-forward layer, and have a large amount of computation and regular repeating patterns.

[0202] The neural network acceleration module (141) includes a matrix unit, and the matrix unit can accelerate dense matrix operations based on a systolic array structure.

[0203] A systolic array is a hardware structure that arranges multiple processing elements in a grid shape and performs operations in parallel by regularly transferring data between adjacent processing elements, and can perform large-scale matrix multiplication operations with high throughput.

[0204] The matrix operation unit can accelerate neural network operations by processing in parallel the multiplication operations between query, key, and value matrices that occur in the attention mechanism operations of the language model through a systolic array structure.

[0205] The neural network acceleration module (141) enables real-time operation of the language model in an edge environment by converting the weights and activation values ​​of the language model from floating-point to integer types through quantization processing, thereby reducing computational load and memory usage. Through this, the total processing delay from receiving a control request to generating a control command can be reduced to a level required for real-time control in industrial sites.

[0206] Next, the traversal acceleration module (142) is configured to accelerate traversal operations by processing irregular memory accesses occurring during traversal operations in a gather-scatter manner, and is electrically connected to the neural network acceleration module (141).

[0207] The traversal acceleration module (142) may include a graph streaming processor.

[0208] The graph streaming processor can process a traversal operation in which the community detection module (121) of the graph inference unit (120) forms a community over the entire knowledge graph through the Leiden algorithm, a traversal search module (123) searches for a multi-stage causal relationship path from an anchor node, and a traversal operation in which the simulation module (132) of the neural symbol verification unit (130) performs a topological reachability analysis.

[0209] In addition, the graph streaming processor can handle irregular memory access that occurs during traversal operations by applying the Gather-Scatter method.

[0210] Gather-Scatter is a memory access method that collects or distributes data distributed across non-contiguous memory addresses in a single operation, enabling the efficient collection of node data distributed along the sparse connection structure of a knowledge graph.

[0211] Specifically, the Gather operation collects the attribute values ​​of adjacent nodes to be visited from non-contiguous memory addresses all at once during a traversal operation, and the Scatter operation can record the intermediate results of the traversal operation by distributing them across the memory addresses of the corresponding nodes.

[0212] Additionally, the traversal acceleration module (142) can perform vertex tiling based on the community structure pre-generated in the community detection module (121).

[0213] Vertex tiling is a memory optimization technique that arranges data from nodes belonging to the same community in tile units across adjacent areas of on-chip high-speed memory. By placing node data that is likely to be accessed together during traversal operations due to dense connectivity between nodes within the community into adjacent areas of high-speed memory, it can improve the cache hit rate and reduce memory access latency.

[0214] Through this, memory access delays are reduced during the process in which the traversal search module (123) traverses the community and the process in which the simulation module (132) performs topological reachability analysis in a large-scale facility environment, thereby ensuring a processing speed that satisfies the requirements for real-time control response.

[0215] Additionally, the traversal acceleration module (142) can process the node and edge data required for the traversal operation through data pre-loading and / or pipelining within the on-chip memory to eliminate transmission delay through the PCIe (Peripheral Component Interconnect Express) interface with external memory.

[0216] The traversal acceleration module (142) can reduce the waiting time of the traversal operation by preloading the node data to be visited in the next traversal step into the on-chip memory while the graph streaming processor processes the adjacent node data of the node currently being visited.

[0217] Since the large-scale traversal operation of the knowledge graph is performed continuously within the on-chip memory independently of external memory access delay, the multi-stage causal path search of the traversal search module (123) and the topological reachability analysis of the simulation module (132) can be performed without processing delay.

[0218] The traversal acceleration module (142) can directly update the dynamic attribute values ​​of the nodes that the dynamic mapping module (113) updates in real time on the on-chip high-speed memory, thereby synchronizing the digital state of the knowledge graph and the physical state of the industrial equipment without delay.

[0219] Since dynamic attribute values ​​are directly updated on the on-chip high-speed memory, the traversal search module (123) and the simulation module (132) can perform traversal operations by immediately referencing a knowledge graph that reflects the latest equipment status without accessing external memory.

[0220] In this way, the NPU acceleration unit (140) can provide a heterogeneous computation hardware architecture that integrates a neural network acceleration module (141) specialized for dense matrix operations and a traversal acceleration module (142) specialized for sparse connection structure search into a single chip.

[0221] The NPU acceleration unit (140) can create a hardware-software co-design synergy by extending the physical coupling of heterogeneous hardware so that the algorithmic computation result of the graph inference unit (120) directly controls the physical memory layout structure of the hardware.

[0222] Additionally, the NPU acceleration unit (140) inputs the logical community detection structure derived by the graph inference unit (120) into the vertex tiling operation to maximize spatial locality, which forces related data to be physically adjacent cache lines. This signifies an organic control mechanism in which the software's cognitive clustering model dynamically reorganizes the hardware's data placement structure, and can reduce the cache miss rate of irregular graph operations beyond the hardware's structural limit compared to the simple sum of independent modules.

[0224] As described above, according to a cyber-physical control system (100) using graph search augmentation generation and neural symbol inference according to one embodiment of the present invention, a knowledge graph is constructed by integrating fragmented unstructured equipment documents and real-time sensor data, and structural context is extracted through multi-stage graph traversal operations, thereby preventing the hallucination phenomenon that occurs when a language model relies on fragmentary keywords and having the effect of accurately identifying the root cause of complexly intertwined equipment failures.

[0225] In addition, since the probabilistic control commands generated by the language model are not immediately applied to physical equipment, and the neural symbol verification unit performs deterministic verification based on constraints and topological reachability predefined in the knowledge graph, the risk of physical safety accidents such as mechanical collisions or exceeding limits is prevented in advance, making it safe to apply to industrial control environments where strict reliability is required.

[0226] In addition, by integrating the irregular traversal operations of a knowledge graph and the dense neural network operations of a large-scale language model on-chip in a dedicated NPU acceleration unit based on heterogeneous computation cores, memory bottlenecks and latency caused by external cloud communication can be drastically resolved, thereby enabling ultra-low latency real-time equipment control even in edge environments and preventing the leakage of corporate industrial secret data.

[0228] The operation and effects of the invention will be explained in more detail below through specific embodiments. However, these are presented merely as examples of the invention and do not limit the scope of the invention in any way.

[0229] Example 1

[0230] In this embodiment, a cyber-physical control system utilizing graph search augmentation generation and neural symbol inference was implemented for the injection molding domain of an automotive parts manufacturing process.

[0231] The equipment subject to the experiment was an electric injection molding machine (ENGEL e-motion 180 / 110) with a clamping pressure of 180 tons, and a total of 38 sensor tags, including 8 temperature sensor channels, 6 pressure sensor channels, and 4 position sensor channels, were collected at 10ms intervals via the Modbus TCP protocol.

[0232] To construct the knowledge graph, the equipment manuals (3 types, 612 pages total), maintenance history (18 months, 847 records), FMEA document (1 type), and collected sensor time-series data of the relevant facility were used as raw data. Unstructured documents were split into chunks of 512 tokens, and Z-score-based outlier filtering and Min-Max normalization were performed on the time-series data. 1,134 entity nodes and 3,412 relationship edges were extracted from the text chunks, and Granger causality analysis (maximum lag of 10 steps, significance level p < 0.05) was performed on the sensor time-series data to generate 43 additional latent correlation edges not specified in the design drawings. Neo4j 5.18 was used as the knowledge graph database.

[0233] The inference and generation models used LLaMA-3.1-70B with 4-bit quantization in an on-premises environment, and the embedding model used text-embedding-3-small (1,536 dimensions). A Graph-Native NPU (based on Groq LPU, 230MB on-chip SRAM) was equipped for graph computation acceleration, and community detection was configured into a 4-layer cluster using the Leiden algorithm.

[0234] Constraints for safety verification were defined in SHACL format (76 in total), and all inference and control command outputs were configured to be completed on-premises without an external network.

[0235] The experimental environment was configured with a Dell PowerEdge R750 server (Intel Xeon Gold 6338, 32 cores, 256GB RAM), Ubuntu 22.04 LTS, and CUDA 12.1.

[0236] Comparative Example 1 - PLC Rule-Based Control System

[0237] In Comparative Example 1, a conventional PLC rule-based control system was configured for the same injection molding machine as in Example 1.

[0238] Data was collected from 38 identical sensor channels via Modbus TCP at 10ms intervals, and a total of 76 threshold-based IF-THEN rules for each channel were implemented in the ladder diagram of a Siemens S7-1500 PLC to output alarm and control signals. History referencing for anomaly diagnosis was performed by manually querying static CSV log files.

[0239] The experimental environment was configured with the same Dell PowerEdge R750 server and Ubuntu 22.04 LTS as in Example 1.

[0240] Comparative Example 2 - Vector RAG-based LLM control system

[0241] In Comparative Example 2, a vector-based RAG (Vector RAG) system was configured for the same injection molding machine as in Example 1.

[0242] The same raw data (equipment manual, maintenance history, FMEA document) as in Example 1 was split into 512-token chunks to generate a FAISS vector index. Real-time sensor data was converted into natural language sentences and indexed in the same manner, with the index update cycle set to 60 seconds. The embedding model and generation model used were text-embedding-3-small and LLaMA-3.1-70B, respectively, as in Example 1.

[0243] The experimental environment was configured in the same way as in Example 1.

[0245] Experimental Example 1 - Comparative Evaluation of Root Cause Diagnosis Accuracy (RCA Accuracy)

[0246] For the system of Example 1 and the systems of Comparative Example 1 and Comparative Example 2, the accuracy of root cause diagnosis was compared and measured in the same injection molding machine environment.

[0247] As experimental conditions, 50 complex failure scenarios that may occur during the injection molding process were defined in advance.

[0248] Each scenario consisted of a single cause-single symptom type (Simple, 20), a single cause-multiple symptom type (Multi-symptom, 15), and a multi-cause-multiple symptom type (Multi-hop, 15), and the correct answer labels were determined in advance through the consensus of three skilled maintenance technicians of the relevant equipment.

[0249] Each scenario was input identically into the systems of Example 1 and Comparative Examples 1 and 2, and the accuracy was calculated by comparing the root cause diagnosis results output by each system with the correct label, and the measurement results were as shown in Table 1 below.

[0250] Failure type Example 1 Comparative Example 1 Comparative Example 2 Simple 18 / 20 (90.0%) 15 / 20 (75.0%) 16 / 20 (80.0%) Multi-symptom 13 / 15 (86.7%) 6 / 15 (40.0%) 9 / 15 (60.0%) Multi-hop 12 / 15 (80.0%) 2 / 15 (13.3%) 5 / 15 (33.3%) entire 43 / 50 (86.0%) 23 / 50 (46.0%) 30 / 50 (60.0%)

[0251] As shown in Table 1, the system of the example achieved a diagnostic accuracy of 86.0% for the entire scenario. In particular, it recorded 80.0% in the Multi-hop type, which requires multi-stage causal reasoning, whereas the PLC rule-based system of Comparative Example 1 only achieved 13.3% in the same type.

[0252] The Vector RAG system of Comparative Example 2 recorded 60.0% overall, but remained at 33.3% in the Multi-hop type, confirming that there are limitations to multi-level relationship inference using only vector similarity-based search.

[0253] Example 1 showed an accuracy improvement of 40.0%p compared to Comparative Example 1 and 26.0%p compared to Comparative Example 2.

[0255] Experimental Example 2 - Comparative Evaluation of Safety Violation Control Command Blocking Rates

[0256] For the system of Example 1 and the systems of Comparative Examples 1 and 2, the blocking performance of control commands violating safety constraints was compared and measured.

[0257] As experimental conditions, 30 safety violation control command scenarios that could lead to equipment damage or worker safety accidents were defined in advance.

[0258] The scenarios consisted of single parameter limit exceedance types (15) and indirect violation types caused by chained actions (15), and the measurement results were as shown in Table 2 below.

[0259] Types of violations Example 1 Comparative Example 1 Comparative Example 2 Exceeding single parameter limit 14 / 15 (93.3%) 12 / 15 (80.0%) 8 / 15 (53.3%) Indirect violation of chain action 12 / 15 (80.0%) 3 / 15 (20.0%) 3 / 15 (20.0%) entire 26 / 30 (86.7%) 15 / 30 (50.0%) 11 / 30 (36.7%)

[0260] As shown in Table 2, the system of Example 1 achieved an overall blocking rate of 86.7%. In addition, the system of Example 1 recorded 80.0% in the indirect violation type of chained operation, whereas the PLC system of Comparative Example 1 and the Vector RAG system of Comparative Example 2 each recorded only 20.0% in the same type.

[0261] Comparative Example 1 is analyzed to have no structure for tracking chain effects other than predefined threshold rules in advance, and Comparative Example 2 is analyzed to have failed to detect indirect violations due to the absence of a topological constraint verification function.

[0262] Example 1 showed an improvement in blocking rate of 36.7%p compared to Comparative Example 1 and 50.0%p compared to Comparative Example 2.

[0264] Experimental Example 3 - Comparative Evaluation of Graph Operation Latency by Graph-Native NPU Acceleration

[0265] For the system of Example 1 and the system of Comparative Example 2, the graph traversal operation and end-to-end inference delay times were compared and measured. In this experimental example, Comparative Example 1 (PLC) was excluded from the measurement subjects because it lacked a graph operation structure itself and therefore did not qualify as a subject for comparison under the same conditions.

[0266] As experimental conditions, 15 multi-hop scenarios requiring a graph traversal depth of 3 or more were selected from the 50 scenarios used in Experimental Example 1, and the average and maximum values ​​were calculated by repeating measurements 20 times for each scenario.

[0267] Example 1 was measured in a Graph-Native NPU (Groq LPU-based, on-chip SRAM 230MB) environment, and Comparative Example 2 was measured using the same Neo4j 5.18 graph database and LLaMA-3.1-70B model, but configured to perform graph computations on an NVIDIA A100 80GB GPU and LLM inference via the AWS Seoul Region Cloud API. The measurement results are shown in Table 3 below.

[0268] Measurement items Example 1 Comparative Example 2 Graph traversal average latency (ms) 8 143 Graph traversal maximum delay (ms) 19 312 End-to-end average latency (ms) 34 1,820 End-to-end maximum latency (ms) 67 3,240 Response rate within 100ms (%) 94.6 0.3 Cash Miss Rate (%) 11.3 58.7

[0269] As shown in Table 3, Example 1 recorded an average graph traversal delay of 8ms and an end-to-end average delay of 34ms. In the general-purpose GPU environment of Comparative Example 2, the cache miss rate reached 58.7% due to irregular memory access patterns during pointer tracking operations following the adjacent list of the graph, and as a result, the average graph traversal delay was measured to be 143ms, which is about 17.9 times higher than Example 1.

[0270] With the addition of network delay due to the cloud API, the end-to-end average delay was measured to be 1,820ms, which is about 53.5 times higher than Example 1.

[0271] Example 1 was confirmed to satisfy the real-time control requirements of the injection molding process by suppressing the cache miss rate to 11.3% and achieving a response rate of 94.6% within 100ms through vertex tiling technology by relocating graph data to adjacent addresses in physical memory.

[0273] Although all components constituting the embodiments of the present invention have been described above as being combined or operating in combination, the present invention is not necessarily limited to such embodiments. That is, within the scope of the purpose of the present invention, all components may be selectively combined in one or more ways to operate.

[0274] Furthermore, terms such as "include," "compose," or "have" described above, unless specifically stated otherwise, mean that the relevant component may be inherent; therefore, they should be interpreted as allowing for the inclusion of additional components rather than excluding them. All terms, including technical or scientific terms, have the same meaning as generally understood by those skilled in the art to which the present invention pertains, unless otherwise defined. Commonly used terms, such as those defined in advance, should be interpreted in accordance with their meaning in the context of the relevant technology and should not be interpreted in an ideal or overly formal sense unless explicitly defined in the present invention.

[0275] Furthermore, the above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications and variations within the scope of the essential characteristics of the present invention.

[0276] Accordingly, the embodiments disclosed in this invention are intended to illustrate, not limit, the technical concept of the invention, and the scope of the technical concept of the invention is not limited by these embodiments. The scope of protection of this invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of this invention. Explanation of the symbols

[0277] 100: Cyber-physical control system using graph search augmented generation and neural symbol reasoning 110: Graph generation section 111: Data Collection Module 112: Ontology Mapping Module 113: Dynamic Mapping Module 120: Graph Inference Unit 121: Community Detection Module 122: Node Identification Module 123: Traversal Search Module 124: Command generation module 130: Neural Symbol Verification Unit 131: Constraint Validation Module 132: Simulation Module 133: Blocking module 140: NPU Acceleration Unit 141: Neural Network Acceleration Module 142: Traversal Acceleration Module

Claims

Claim 1 A graph generation unit that generates a knowledge graph based on sensor data collected from industrial equipment and equipment-related documents; a graph inference unit that extracts contextual information based on the knowledge graph and inputs the contextual information into a language model to generate a control command; and a neural symbol verification unit that verifies the control command based on the knowledge graph and blocks the control signal output of the industrial equipment if a violation of a constraint stored in the knowledge graph is detected. and includes an NPU acceleration unit that accelerates the traversal operation of the knowledge graph and the neural network operation of the language model, and the graph generation unit includes a data collection module that collects and preprocesses the sensor data and equipment-related documents from the industrial equipment, an ontology mapping module that extracts entities and relationships between entities from the preprocessed equipment-related documents, maps them to a standard ontology schema, converts them into nodes and edges to generate the knowledge graph, and a dynamic mapping module that updates the knowledge graph by mapping the preprocessed sensor data to the dynamic attributes of the nodes, and the dynamic mapping module calculates the rate of change of the sensor data value per unit time by dividing the difference between the value of the sensor data at the current sampling point and the value of the sensor data at the previous sampling point by the sampling period and stores it as the dynamic attribute of the node, wherein if the rate of change exceeds a predefined rate of change threshold, a rate of change abnormality flag is added as the dynamic attribute of the node and stored in the knowledge graph, and the neural symbol verification unit, the control command is stored in the knowledge graph using Shapes Constraint Language, A constraint verification module that verifies whether the said constraint is violated by comparing it with SHACL-based constraints, and for a control command that has passed the verification of the said constraint verification module, simulates the state change resulting from the execution of the said control command through reachability analysis on the said knowledge graph, andA cyber-physical control system using graph search augmented generation and neural symbol inference, comprising a simulation module for verifying whether a topological violation has occurred, and a blocking module for blocking the control signal output of the industrial equipment when a violation of the constraint or the topological violation is detected. The constraint includes a static constraint that defines the range of attribute values ​​of the node, and a rate of change constraint that is defined based on the shape constraint language, stored in the node, and defines an allowable range for the value of the sensor data to change per unit time. The constraint verification module performs a rate of change-based constraint verification by comparing the rate of change stored as a dynamic attribute of the node by the dynamic mapping module with the rate of change constraint. Even if the value of the sensor data satisfies the static constraint, if the rate of change exceeds the rate of change constraint, it is determined to be a violation of the constraint. Claim 2 delete Claim 3 A cyber-physical control system using graph search augmentation generation and neural symbol inference according to claim 1, wherein the graph inference unit comprises: a community detection module that clusters nodes of the knowledge graph based on density to form a community and generates summary information of the community; a node identification module that performs a vector similarity search based on the summary information to identify an anchor node; a traversal search module that performs a traversal operation based on the anchor node to extract context information; and a command generation module that inputs the context information into the language model to generate the control command. Claim 4 delete Claim 5 A cyber-physical control system using graph search augmented generation and neural symbol inference, wherein the NPU acceleration unit comprises a neural network acceleration module that accelerates the neural network operation by accelerating the dense matrix operation of the language model based on a systolic array structure, and a traversal acceleration module that accelerates the traversal operation by processing irregular memory access occurring during the traversal operation of the knowledge graph based on a gather-scatter method.

Images