AI-based System and Method for Converting Domain-specific Logic Diagrams to Executable Code

Abstract

The automatic code conversion system of the present invention includes an input unit that receives input data including a control logic diagram image, a processor that analyzes the input data received through the input unit and converts it into programmable code based on an artificial intelligence-based code conversion model, and an output unit that outputs the program code converted by the processor.

Description

Technology Field

[0001] The present invention relates to an automatic code conversion system and an automatic code conversion method for automatically converting a logic diagram (LD) into programmable code. Background Technology

[0002] In modern industrial sectors, various forms of logic diagrams (CLD, P&ID, HMI, electrical circuit diagrams, etc.) are utilized for the design of control systems in power plants, chemical plants, manufacturing facilities, and commercial buildings; most of these drawings are provided in unstructured image formats such as PDF, TIFF, and DWG. Converting these into control code requires manual work by skilled engineers, and this process carries a high risk of work delays and human error.

[0003] In addition, as the symbol notations and control system standards used vary by industry (IEC 61131-3, IEC 61499, Modbus, BACnet, etc.), there is an increasing demand for automated conversion technology for the integration of heterogeneous systems. However, to date, conversion technologies limited to specific industries or formats exist, and research on universal and accurate automatic conversion technology is lacking. The problem to be solved

[0004] The problem that the present invention aims to solve is to provide an automatic code conversion system and an automatic code conversion method that can resolve the issues of work delay and human error that occur when programmable code for controllers is manually written based on logic diagrams generated in power plants, chemical plants, manufacturing facilities, commercial buildings, IoT systems, etc.

[0005] In particular, the invention provides an automatic code conversion system and an automatic code conversion method capable of automatically generating program code by accurately recognizing complex control logic symbols and connection relationships from unstructured image data in various formats such as PDF / TIFF / DWG / CAD using artificial intelligence-based object recognition and analysis techniques.

[0006] Third, it provides an automatic code conversion system and an automatic code conversion method that resolve the difficulties of universal conversion caused by differences in unique symbol systems and notations across various industries, such as automation and commercial control systems. means of solving the problem

[0007] The automatic code conversion system of the present invention includes an input unit that receives input data including a control logic diagram image, a processor that analyzes the input data received through the input unit and converts it into a programmable code based on an artificial intelligence-based code conversion model, and an output unit that outputs the program code converted by the processor.

[0008] The processor may include an object detection module that detects elements constituting the diagram based on a vision-based object detection algorithm, a graph analysis module that analyzes relationships between elements in the diagram based on a graph-based neural network, and a semantic interpretation module that generates the programmable code based on the analysis results of the object detection module and the graph analysis module.

[0009] The above input data may include at least one of an instrument specification sheet, a control logic diagram (CLD), and a parameter table.

[0010] The above input data may include image-based control logic diagrams and text-based data, and the processor may include a multimodal model that converts the diagrams and text data input through the input unit into programmable code.

[0011] The processor may further include a verification module that verifies the accuracy of the programmable code, a learning module that learns errors based on the verification results of the verification module to improve the code conversion model, and an adapter module that supports the standard of the diagram.

[0012] The processor may include at least one analysis model among an object detection and graph analysis model, a multimodal model, a reinforcement learning model, an evolutionary computation model, and a neural symbolic inference model, and a code generation model that generates the programmable code based on the results analyzed by the analysis model.

[0013] It may further include a selection unit that receives a selection for at least one analysis model.

[0014] An automatic code conversion method according to one embodiment of the present invention comprises: (a) receiving input data including a control logic diagram image by an input unit; (b) selecting an artificial intelligence-based analysis model for analyzing the input data by a selection unit; (c) analyzing the input data by the analysis model selected in step (b); and (d) generating programmable code based on the results analyzed in step (c).

[0015] The above step (b) may include evaluating at least one of the resolution, complexity, and noise level of the diagram, determining available computing resources and required processing time based on the evaluation result, and selecting the analysis model according to the determination result. Effects of the invention

[0016] First, the automatic code conversion system and automatic code conversion method of the present invention have the effect of significantly reducing work delays and human errors occurring during the design and implementation stages by automatically analyzing control logic diagrams and converting them into code.

[0017] Second, different types of diagrams such as P&ID, ILD, C&E, FBD, LD, and SFC can also be standardized through adapter modules, which has the effect of enabling universal code generation.

[0018] Third, by supporting various analysis models and multimodal inputs, it offers great scalability and applicability across industries.

[0019] Fourth, by automatically receiving a control logic diagram image through the input unit, converting it into programmable code for the processor, and providing it through the output unit, manual work in the process of writing control code can be significantly reduced and development efficiency can be improved.

[0020] Fifth, by utilizing an AI-based code conversion model, it is possible to prevent misunderstandings or omissions that may occur when interpreted by humans, and to enhance the accuracy and reliability of the program code.

[0021] Sixth, since the converted program code can be output in various control language formats such as Structured Text (ST), Ladder Diagram (LD), and Function Block Diagram (FBD), it offers excellent compatibility with controllers and standards used in industrial settings. Therefore, this invention automates the generation of control code based on control logic diagrams in various industrial fields, such as power plants, chemical plants, manufacturing facilities, and building automation, thereby reducing time and costs in the design and implementation process and minimizing errors. Brief explanation of the drawing

[0022] FIG. 1 is a configuration diagram of an automatic code conversion system according to one embodiment of the present invention. FIG. 2 is a flowchart showing an automatic code conversion method according to an embodiment of the present invention. FIG. 3 is a configuration diagram of an automatic code conversion system according to another embodiment of the present invention. Figure 4 illustrates a P&ID diagram as an example of data input to the input section. Figure 5 shows an example of code output through the output section. Specific details for implementing the invention

[0023] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0024] FIG. 1 is a configuration diagram of an automatic code conversion system according to one embodiment of the present invention. FIG. 2 is a flowchart showing an automatic code conversion method according to one embodiment of the present invention. FIG. 3 is a configuration diagram of an automatic code conversion system according to another embodiment of the present invention. FIG. 4 illustrates a P&ID diagram as an example of data input to an input unit. FIG. 5 illustrates an exemplary code output through an output unit.

[0025] Referring to FIGS. 1 and 2, an artificial intelligence-based automatic code conversion system according to one embodiment of the present invention includes an input unit (100), a processor (200), and an output unit (300).

[0026] The input unit (100) can receive input data including a control logic diagram image. The above control logic diagram may include at least one of the following: a Piping and Instrumentation Diagram (P&ID) that illustrates equipment or instruments such as piping, tanks, pumps, and valves, and the flow of control signals; a Control Logic Diagram (e.g., Interlock Diagram) that illustrates logical control relationships such as equipment operation and shutdown and alarm generation under specific conditions; a Cause & Effect Diagram (C&E) that expresses cause and effect in a matrix form; a Function Block Diagram (FBD, IEC 61131-3) that expresses control logic using blocks and signal lines as one of the international standard control languages ​​for controllers; a Ladder Diagram (LD) based on the IEC 61131-3 language that expresses control logic in the form of relay circuits; and a Sequential Function Chart (SFC) that expresses sequential control logic based on steps and transitions.

[0027] The above diagram may be an image in various formats such as PDF, TIFF, DWG, CAD, etc.

[0028] According to an embodiment, the input data may be text-based data, for example, receiving data such as instrumentation specifications or parameter tables. Here, the text-based data may be not only data directly input as characters but also data detected or identified from images.

[0029] The processor (200) may process data input through the input unit (100). The processor (200) may include an artificial intelligence processing unit (210) that processes data based on an artificial neural network.

[0030] The artificial intelligence processing unit (210) analyzes the data input through the input unit (100), specifically, it can identify elements in the diagram, determine the meaning of the identified elements and the relationships between them, and generate programmable code based on this.

[0031] Here, programmable code may be a control language such as Structured Text (ST), Ladder Diagram (LD), Function Block Diagram (FBD), and Sequential Function Chart (SFC) as defined in IEC 61131-3, and may also be in the form of a script language such as JSON, XML, or Python.

[0032] According to an embodiment, the artificial intelligence processing unit (210) may include an object detection module (211), a graph analysis module (212), and / or a semantic interpretation module (213).

[0033] The object detection module (211) can detect equipment and instrument symbols within the diagram and extract tag information.

[0034] For example, the object detection module (211) can detect symbols such as pumps, valves, and sensors included in the input diagram and extract tag numbers (e.g., P-101, CV-101, FT-101).

[0035] The graph analysis module (212) can identify the graph structure by identifying the connection lines and directions between the detected objects.

[0036] The semantic interpretation module (213) can generate programmable code, such as conditional statements, control statements, and interlock logic, by semantically analyzing the object and relationship graph.

[0037] For example, the semantic interpretation module (213) can generate programmable code consisting of conditions (If) and actions (Then), etc., based on object detection by the object detection module (211) and graph analysis results by the graph analysis module (212).

[0038] The output unit (300) can output program code generated by the processor (200). The output by the output unit (300) can be in various formats such as IEC 61131-3 Structured Text (ST), Ladder Diagram (LD), Function Block Diagram (FBD), JSON / XML, etc.

[0039] Referring to FIG. 2, a code generation method according to one embodiment of the present invention may include the following steps.

[0040] Data input step (S1): Input data including images such as P&ID diagrams and ILDs can be input into the input unit (100). According to the embodiment, the input data may further include text data, etc.

[0041] Object detection step (S2): The processor (200) can recognize symbols and / or tags corresponding to elements (or equipment or instruments) such as pumps, valves, sensors, and alarm blocks from a diagram included in the input data through a vision (image recognition)-based object detection algorithm. This object detection step (S2) can be performed by an object detection module (211).

[0042] Referring to Fig. 4, in step S2, TK-101 (tank), P-101 (pump), CV-101 (control valve), FT-101 (flow meter), TT-101 (thermometer), HS-201 (AUTO / LOCAL switch), FAL-101 (low flow alarm), TAH-101 (high temperature alarm), etc. can be recognized from the P&ID input diagram.

[0043] Relationship analysis step (S3): By analyzing the diagram based on a graph recognition-based neural network, the connection relationships (pipes, signal lines, control lines, etc.) between symbols (or elements) can be analyzed.

[0044] The graph analysis module (212) can analyze, for example, fluid flow leading to TK-101, P-101, CV-101, FT-101, TT-101, user system, control signal connected from HS-201 to P-101 / CV-101, and alarm path of FT-101==>FAL-101, TT-101==>TAH-101.

[0045] Code generation step (S4): Programmable code can be generated based on the results derived from the object detection step (S2) and the relationship analysis step (S3). Step S4 can be performed by the semantic interpretation module (213).

[0046] For example, the semantic interpretation module (213) can generate the following control logic code.

[0047] [Control Logic Code]

[0048] IF Flow < 500 L / min THEN FAL-101 := TRUE;

[0049] IF Temp > 40 °C THEN TAH-101 := TRUE;

[0050] IF MODE = AUTO THEN P-101 := ON;

[0051] CV-101 := OPEN; ELSE (LOCAL MODE) P-101, CV-101 := Field Command;

[0052] Here, reference values ​​such as "500 L / min" or "40 °C" are not shown in FIG. 4, but other data included in the input data can be obtained, for example, from the Instrument Specification Sheet, Control Logic Diagram (CLD), and parameter table.

[0053] Result output step (S5): The code generated in the code generation step (S4) can be output through the output unit (300).

[0054] Meanwhile, referring to FIG. 3, according to an embodiment, the artificial intelligence processing unit (210) may include a multimodal model (220). The multimodal model (220) combines and interprets image-based input (control diagram) and text-based input (instrumentation specification, parameter table, etc.). Through this, it is possible to generate program code applicable to actual operation by reflecting reference values ​​listed in the specification (e.g., alarm triggered when flow rate < 500 L / min, alarm triggered when temperature > 40℃) rather than simply recognizing symbols and connecting lines.

[0055] Meanwhile, according to an embodiment, the processor (200) may further include a verification module that verifies the accuracy of the code generated in the code generation step (S4), a learning module that learns errors based on the verification results of the verification module to improve the code conversion model, and an adapter module that supports the standard of the diagram.

[0056] The verification module can check for grammatical errors, executable status, and logical consistency of control logic in the generated program code. If errors or abnormal behavior are detected in the verification results, the learning module can reflect this as training data to improve the code conversion model.

[0057] In addition, the adapter module can ensure compatibility in various industrial environments by correcting differences between diagram standards and / or code formats defined by the industry, such as ISA and IEC.

[0058] Meanwhile, the above adapter module may standardize differences in the format of input data (e.g., diagram specifications).

[0059] For example, even if the input data includes diagrams with different representation methods and notation rules such as P&ID, Control Logic Diagram (including ILD), C&E, FBD, LD, and SFC, the adapter module can standardize them by correcting for differences according to the format. In this case, the object detection step (S2) and / or relationship analysis step (S3) may be performed based on the standardized input data. This has the advantage of providing high universality and scalability that can be applied to various control systems and equipment environments without being dependent on a specific industrial field or a specific control language.

[0060] In addition, the processor (200) in the present invention may include various artificial intelligence-based analysis models. For example, the processor (200) may include a reinforcement learning model that learns an optimal policy through a reward signal during the code conversion process, an evolutionary computation model that searches for an optimal structure by generating and evolving candidate interpretation models, and a neural symbolic inference model that interprets control logic more precisely by combining neural network-based learning and symbolic logic inference.

[0061] Additionally, according to an embodiment, an automatic code conversion system according to one embodiment of the present invention may further include a selection unit that receives a model selection so that a user or a system selects one or more of the analysis models.

[0062] The selection through the above selection unit can be made by the processor (200). Specifically, the processor (200) evaluates at least one of the resolution, complexity, and noise level of a diagram input to the input unit (100), determines available computing resources and required processing time based on the evaluation result, and can select a suitable analysis model according to the determination result.

[0063] For example, the processor (200) can select a simple CNN-based object detection model when the resolution or complexity of the diagram is low, and a multimodal model when the complexity is high or there is multiple input data.

[0064] Alternatively, the above selection unit may be configured to provide a selection menu that allows the user to directly select a specific analysis model.

[0065] Meanwhile, the term “processor” in this specification may be implemented by one or more computing units, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a microcontroller, or a combination thereof, and may be configured by hardware, software, firmware, or a combination thereof. A processor may be implemented as a single device or as multiple devices in a distributed network environment.

[0066] In this specification, the term “model” means a mathematical structure or algorithm learned to analyze or interpret input data, and may include, for example, a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Transformer, a Graph Neural Network (GNN), a reinforcement learning model, an evolutionary computation model, a neural symbolic inference model, etc. Such “models” may be implemented as software code, hardware acceleration circuits, or a combination thereof, and the specific form does not limit the scope of the invention.

[0067] In this specification, the term “module” means a logical unit designed to perform a specific function and may be implemented as software, hardware, firmware, or a combination thereof.

[0068] For example, an object detection module may be software code that performs image analysis algorithms, a deep learning model running on a GPU, or a dedicated hardware circuit such as an FPGA / ASIC. Accordingly, it is specified that the term “module” as used herein is not limited to a specific implementation method.

Claims

Claim 1 An automatic code conversion system comprising: an input unit receiving input data including a control logic diagram image; a processor that analyzes the input data received through the input unit and converts it into a programmable code based on an artificial intelligence-based code conversion model; and an output unit that outputs the programmable code converted by the processor, wherein the processor includes: an object detection module that detects elements constituting the diagram based on a vision-based object detection algorithm; a graph analysis module that analyzes the relationships between elements in the diagram based on a graph-based neural network; and a semantic interpretation module that generates the programmable code based on the analysis results of the object detection module and the graph analysis module. Claim 2 delete Claim 3 In claim 1, the input data is an automatic code conversion system comprising at least one of an instrument specification sheet, a control logic diagram, and a parameter table. Claim 4 An automatic code conversion system according to claim 1, wherein the input data includes an image-based control logic diagram and text-based data, and the processor includes a multimodal model that converts the diagram and text data input through the input unit into programmable code. Claim 5 An automatic code conversion system according to claim 1, wherein the processor further comprises: a verification module that verifies the accuracy of the programmable code; a learning module that learns errors based on the verification results of the verification module to improve the code conversion model; and an adapter module that supports the standard of the diagram. Claim 6 An automatic code conversion system according to claim 1, wherein the processor comprises at least one analysis model among an object detection and graph analysis model, a multimodal model, a reinforcement learning model, an evolutionary computation model, and a neural symbolic inference model; and a code generation model that generates the programmable code based on the results analyzed by the analysis model. Claim 7 An automatic code conversion system according to claim 6, further comprising a selection unit that receives a selection for at least one analysis model. Claim 8 (a) receiving input data including a control logic diagram image by an input unit; (b) selecting an artificial intelligence-based analysis model for analyzing the input data by a selection unit; (c) analyzing the input data by the analysis model selected in step (b); and (d) generating programmable code based on the results of the analysis in step (c), wherein step (b) includes evaluating at least one of the resolution, complexity, and noise level of the diagram, determining available computing resources and required processing time based on the evaluation results, and selecting the analysis model according to the determination results. Claim 9 delete

Images